Atalanta collapsed from 3-0 up to draw at Lazio on Saturday, missing the chance to move to the top of the Serie A table.
Gian Piero Gasperini's side have made a superb start to the domestic season, despite suffering consecutive defeats to begin their debut Champions League campaign.
The Bergamo outfit could have climbed to the Serie A summit with victory in Rome - at least until champions Juventus played later on Saturday - and were cruising at half-time.
Atalanta, who face Manchester City away on Tuesday, led as Luis Muriel scored twice in quick succession midway through the first half.
Captain Alejandro Gomez then stretched their advantage further heading into the break.
But the visitors fell apart in quite spectacular fashion, with their lead cut thanks to two goals in two minutes from Ciro Immobile and Joaquin Correa.
Atalanta still looked to have held on until Immobile, who won and dispatched a penalty for his first, again went down in the area and converted from the spot in the 92nd minute.
The Italy striker then comically failed in his attempts to take his shirt off in celebration, falling over but still earning a yellow card.
The moment of hilarity was of little comfort to Atalanta ahead of a daunting European trip to the Etihad Stadium.