Leading 2-1 from the first leg of their last-16 tie in February, City hit the front early on at the Etihad Stadium when Raphael Varane gift-wrapped Raheem Sterling his 100th goal for the club.
Karim Benzema's fine header restored parity in the 28th minute, only for Varane to again cost Madrid midway through the second half.
Jesus was the benefactor of the centre-back's second calamity of the game, with a quarter-final clash against Lyon the reward for City.
Who else but Benzema!— Champions League on CBS Sports (@UCLonCBSSports) August 7, 2020
The Real Madrid talisman heads home the opening score for a much needed away goal. Game on! pic.twitter.com/oaBs4kyF2p
Varane's nightmare started in the ninth minute, with the France international playing himself into trouble, Jesus robbing possession and teeing up Sterling for a simple finish.
Sterling very nearly had his 101st goal in sky blue moments after, but his chipped effort dropped just over the bar.
Ederson had to make two fine saves as Madrid fought back, but City did not heed the warning, and the visitors were on level terms when Benzema headed in from Rodrygo's cross.
Sterling could have done better with a shot into the side netting following the restart, before Thibaut Courtois denied Kevin De Bruyne direct from a corner.
Courtois was in action again with 25 minutes remaining, tipping Jesus' effort over, but City had their second soon after.
Having messed up his initial header, Varane put nowhere near enough power on his nod back to Courtois, with Jesus nipping in to prod home cleverly from a tight angle.
Handed an Etihad farewell in the final stages, David Silva was inches away from making it 3-1 with a dipping free-kick, but City ultimately settled for what they had as they knocked out the 13-time European champions.
What does it mean? Jekyll and Hyde City must sharpen up to break Champions League duck
Sterling's opener was City's 150th Champions League goal, in their 76th appearance in the competition, yet they then conceded their 100th goal in the tournament under 20 minutes later.
If they are to end their wait for Europe's top club trophy this season, City's defence must give more support to their stellar forwards.
Century up for Sterling
Though the goal was gaping, Sterling still showed real composure to nudge City ahead early on, and in doing so he became the 17th player to reach a century of goals for the Citizens and the first Englishman since Dennis Tueart in 1981.
Varane horror show proves decisive
With Sergio Ramos suspended, the pressure was on Eder Militao heading into Friday's clash, yet it was his experienced defensive partner Varane who was left red-faced after his two errors resulted in both of City's goals.
Lyon, who knocked out Juventus on away goals after a 2-2 aggregate draw, are next up for City on August 15 in Lisbon, while Madrid's focus will switch to the new season.