The world number three and top seed had no answer to an impressive performance from Rune, with the young Dane ultimately easing to a 6-3 6-2 victory on Thursday (AEST).
Rune set the tempo early on, moving Zverev around the court with plenty of drop shots and heading into a 4-2 lead, before saving two break points on his way to clinching the first set.
An early break in the second then seemed to give the 18-year-old the confidence to see out his first career victory against a top 10 opponent.
"It was a difficult match from the beginning," said Rune on-court after the win. "He's in the top three in the world and has been playing some unbelievable tennis, especially in the past year winning so many big titles.
"I have a lot of respect for him, and I obviously didn't expect to win in two sets, and with a score like this. But it was a really hard match, we played a lot of rallies and I'm really happy about my performance today."
Rune will play Emil Ruusuvuori in the quarter-finals after the Finn beat Maxime Cressy 6-3 6-3.
Elsewhere, fourth seed Nikoloz Basilashvili came from behind to overcome Ilya Ivashka 3-6 6-2 6-4, and will face Miomir Kecmanovic in the last eight after the seventh seed beat Daniel Altmaier 6-2 6-4.
There were also three round of 32 matches on Wednesday, with eighth seed Botic van de Zandschulp, Egor Gerasimov and Alejandro Tabilo all through to the last 16 in Munich.
Over at the Estoril Open, Alejandro Davidovich Fokina will play Frances Tiafoe in the quarter-finals after both advanced in Portugal.
Fourth seed Davidovich Fokina beat fellow Spaniard Bernabe Zapata Miralles 6-3 7-6 (7-5) while fifth seed Tiafoe got the better of Nuno Borges 7-6 (7-4) 4-6 6-0.
Fernando Verdasco is also through after a 6-2 6-3 win against Pablo Cuevas, and he will play fellow Spaniard and sixth seed Albert Ramos-Vinolas, who came from a set down against Kwon Soon-woo, 5-7 6-1 6-2.