The world number one collapsed from a set and a break ahead as German fourth seed Zverev won 1-6, 6-3, 6-1 to set up a Sunday final against Russian Karen Khachanov.
The 34-year-old Djokovic has never won the Olympic singles title.
The Serbian star had already captured the Australian Open, French Open and Wimbledon trophies this year and needed Olympic gold and the US Open crown to emulate Steffi Graf, who achieved the feat in 1988, by winning the Golden Slam.
But his attempt to make history came to a close as Zverev powered 30 winners past the usually impregnable Djokovic in a stunning display.
It was yet more heartbreak for the 20-time Grand Slam champion at the Olympics, where his best result is a bronze medal in 2008.
He lost to eventual winner Andy Murray in the semi-finals in London nine years ago, and was in tears after a first-round exit to Juan Martin del Potro at the 2016 Rio Games.
Zverev is looking to become the first German to win singles gold since Graf in Seoul after winning 10 of the last 11 games against Djokovic.
- Djokovic makes fast start -
Djokovic, roared on by a large contingent of his Serbian Olympic teammates, took the first set in trademark fashion by winning most of the big points.
The score made it look comfortable, but he was taken to deuce in his first three service games and had to save an early break point.
Zverev again put Djokovic's serve under pressure in the early stages of the second set, but the top seed wriggled out of trouble in the fourth game and broke in the next.
Zverev dumped a simple volley into the net on break point and hammered the ball into the empty stands in frustration.
That looked to be the final nail in his coffin, but the 24-year-old bounced back immediately with a break to love as Djokovic uncharacteristically played an error-strewn game at a vital time.
The momentum had suddenly swung in Zverev's favour and he quickly broke again, before taking his second set point to force a decider with a run of four straight games.
Any thoughts that Zverev would feel the tension were dispelled in the opening game of the third set as he broke once more.
He dug deep to consolidate his breakthrough and make it six consecutive games, saving four break points, the third of which after a stunning rally, finished off by a backhand passing shot up the line.
That proved to be the decisive moment, as Zverev then sealed a double break to put one foot in the final, with Djokovic reacting angrily after shanking a groundstroke long.
He finally snapped an eight-game losing streak, but his Olympic dream was already in tatters, and Zverev sealed a final place on his second match point with a rasping backhand.
Djokovic could still win multiple medals, with the bronze-medal match against Pablo Carreno Busta to come on Saturday after his mixed doubles semi-final with Nina Stojanovic against Elena Vesnina and Aslan Karatsev later Friday.
- Khachanov sees off Carreno Busta -
Khachanov, a former top-10 player who has struggled for consistency in the last three seasons, produced an impressive display earlier to dispatch Carreno Busta 6-3, 6-3.
Khachanov is bidding to become the first Russian champion in the men's singles since Yevgeny Kafelnikov struck gold in Sydney 21 years ago.
"It's just pure happiness, pleasure to be here to live those moments, these kind of memories will stay forever," the 25-year-old, who reached the Wimbledon quarter-finals earlier this month, said.
Marcus Daniell and Michael Venus became New Zealand's first Olympic tennis medallists for 109 years, beating Americans Austin Krajicek and Tennys Sandgren 7-6 (7/3), 6-2 in the men's doubles bronze-medal match.
Women's world number one Ashleigh Barty's third and final attempt to win a gold medal was ended after defeat alongside John Peers in the mixed doubles semi-finals against Russians Anastasia Pavlyuchenkova and Andrey Rublev.