Portugal went ahead after just two minutes as Bernardo Silva capitalised on a defensive error to set up Renato Sanches, who opened the scoring with an unerring finish.
Dusan Tadic's deflected strike levelled the scores later in the first half, with Rui Patricio partially culpable for allowing the ball to cross the line.
Substitute Mitrovic completed a dramatic turnaround in the 90th minute as he headed in from Tadic's cross to send Serbia top of Group A and condemn Portugal to the playoffs.
Portugal enjoyed a dream start as Silva dispossessed Nemanja Gudelj before finding Sanches, who advanced into the area and rifled past Predrag Rajkovic.
Dusan Vlahovic thought he had equalised 10 minutes later as he struck towards the bottom-right corner, but his left-footed effort bounced off the post and away from danger.
Serbia did draw level in the 33rd minute when Tadic's fierce strike from the edge of the area took a deflection, causing Patricio to fumble into his own net.
The host came within inches of reclaiming the lead after the interval as Sanches' blocked effort was cleared against Rajkovic's back and rolled just wide.
Serbia appeared surprisingly conservative in the second half but Mitrovic, who headed wide only minutes before, grabbed a dramatic winning goal as he nodded in at the back post after Tadic's cross from the right flank.