The brother of Australia international Harry Souttar rose highest to nod home an 89th minute free-kick at Tynecastle, a goal that saw the newly-promoted side grab all three points in its first game back in the Scottish Premiership.
Postecoglou's Celtic had gone behind early after a defensive calamity allowed the ball to fall kindly for Gary Mackay-Steven, who smashed home the opening goal from about six yards in the eighth minute.
Celtic responded well enough, dominating possession for the remainder of the first half, but it went into the sheds with nothing to show for it.
A big improvement required for #AngePostecoglou's Hoops to get something out of this game.— beIN SPORTS (@beINSPORTS_AUS) July 31, 2021
⏱ #SPFL HALF-TIME: @JamTarts 0️⃣-0️⃣ @CelticFC
Watch LIVE on CONNECT @kayosports, @Foxtel & Fetch!! ▶️ https://t.co/dR3JmGUdy7#HEACEL pic.twitter.com/lY0EKMk0vu
But nine minutes after the break, Anthony Ralston put the smile back on Postecoglou's face with a stunning solo effort that saw him jink past the Hearts defence before firing home a left-foot strike from just inside the box that silenced the home supporters.
Celtic continued to dominate possession thereafter, but again showed little attacking thrust as Hearts kept the visitors at bay despite spending long periods without the ball.
As the game began heading to what seemed like a draw, Hearts found a second wind and began putting the shaky Celtic defence under serious pressure again, ultimately leading to a free kick from an innocuous position on the left touchline.
Michael Smith's delivery proved to be perfect, and Souttar rose as his brother has many times in the green and gold to bury the opportunity.
Celtic pushed frantically for an equaliser and forced two magnificent saves from former Hoops goalkeeper Craig Gordon in stoppage-time, and Postecoglou looked a frustrated figure on the touchline as he endured another disappointing defeat, barely a week after overseeing a UEFA Champions League exit that will have done little to endear him to the Hoops faithful.