Young, who also contributed 11 assists and seven rebounds, scored 12 points in the fourth quarter to spark a rally that baffled the Bucks.
"We just keep fighting until the end, no matter what the score is," Young said. "That's what we're going to do. We're going to keep fighting."
Young had the most points by any player making his conference finals debut since the NBA adopted the conference format 50 years ago.
The best-of-seven series continues Friday at Milwaukee with the winner advancing to an NBA Finals matchup against either the Phoenix Suns or the Los Angeles Clippers.
John Collins scored 23 points and grabbed 15 rebounds while Clint Capela had 12 points and 19 rebounds.
"People overlook this team, take us for granted, so for us to come out here and get a win is big time," Collins said.
Greek star big man Giannis Antetokounmpo had 34 points, 12 rebounds and nine assists while Jrue Holiday had 33 points and 10 assists for the Bucks in a losing cause.
"We'll get better," Bucks coach Mike Budenholzer said. "We'll get smarter game after game."
It was Milwaukee's first home playoff loss this season.
"Young's a great player," Budenholzer said. "He had a great night. We feel like we can play better."
Collins scored eight points, six on dunks, and Young added seven in a 21-8 Atlanta run that gave the Hawks
a 79-73 lead late in the third quarter before the Bucks rallied.
Antetokounmpo scored six points to lead a 13-2 Bucks' run for a 105-98 lead, but Collins hit a jumper and Young, who hit 17-of-34 shots from the floor, added a 3-point play to lift the Hawks back within two points.
The game was poised on a knife's edge until Collins hit a 3-pointer and Capela scored on a rebound put-back that lifted the Hawks ahead 112-11 with 29.8 seconds remaining.
Young grabbed a loose ball and sank two more free throws, then he and Antetokounmpo exchanged free throws in the final seconds to create the final margin.
Khris Middleton, who went 0-of-9 from 3-point range, missed an attempt to lift the Bucks level from beyond the arc in the final seconds to seal Atlanta's victory.
"I believe in this team," Young said. "We knew we could get a stop. We just buckled down and got it.
"Our confidence never goes away. We put too much work in to stop believing in each other."
There more than 16,000 spectators in the Bucks' arena and about as many outside watching on a huge screen.
The Hawks, who had lost their seven prior games in Milwaukee since 2016, were based in Milwaukee for four seasons in the 1950s. They won their only NBA title in 1958 while based in St. Louis. They last reached
the NBA Finals in 1961 when based in St. Louis.
The Bucks lost to Toronto in the 2019 conference finals and to Miami in last year's conference semi-finals, with Antetokounmpo winning the NBA Most Valuable Player award both seasons.
The Bucks have not reached the NBA Finals since 1974, when they lost to Boston in seven games. The Bucks won their only NBA title in 1971 when they swept Baltimore in the finals.
- Leonard out, Paul maybe -
The Clippers, trailing Phoenix 0-2 in their series, said Wednesday that star Kawhi Leonard will not play in game three on Thursday in Los Angeles due to a right knee sprain, the fifth game in a row he has missed.
Leonard has averaged 30.4 points, 7.7 rebounds and 4.4 assists in 11 playoff games before suffering the injury in game four of the Western Conference semi-finals against Utah.
Phoenix listed guard Chris Paul as probable after he missed the first two games of the series while in the NBA's Covid-19 health and safety protocols.
Paul, a former Clipper, averaged 25.5 points, 10.2 assists and 5.0 rebounds a game for the Suns in the second round against Denver.