Nadal, a five-time champion at this event, was a break up in the first set of his 15th quarter-final at Caja Magica but lost 6-4 6-4 in an hour and 44 minutes.
Zverev had won the previous two meetings with Nadal, both on indoor hard courts, after losing five consecutive matches against the 20-time Major champion including a 6-1 6-1 rout in Monte Carlo in their first clay contest four years ago.
The world No.6 has five clay-court titles to his name, including this event in 2018, but defeating 13-time French Open champion Nadal on his favourite surface in front of a home crowd ranks among Zverev's finest achievements.
It looked as though Nadal was assuming control when he broke to love to lead 4-2 in the first set, but Zverev responded with a break back at the first time of asking.
The German had spoken of the Madrid courts playing faster than other clay events and the conditions certainly seemed to suit his fearless approach as Nadal struggled to deal with his crosscourt backhand.
He won the next three games to clinch a first set in which he lost just four of 16 points behind his first serve.
While Zverev looked increasingly confident, mixing up his shot depth and frequently coming to the net, Nadal cut an unusually frustrated figure even as he showed customary battling qualities to ward off a break point at 1-1.
However, more powerful hitting from Zverev saw him break to lead 4-2, and although he missed two chances for a double break, he closed out the match with minimal fuss.
"Definitely one of the biggest wins of my career," Zverev, who will face Dominic Thiem in the semi-finals, said. "Beating Rafa on clay is one of the toughest things to do in our sport."