Mads Pedersen put Augsburg in front, but Choupo-Moting scored either side of Kimmich's beautiful finish to put the Bundesliga champion 3-1 up.
An own goal from Dayot Upamecano caused some fears of a Bayern collapse, but Musiala and Davies struck late to make sure of the victory.
Bayern's return to form continues as they reach the next round of a competition they have won on 20 occasions, 14 more than any other team.
The host started brightly and took the lead after nine minutes, Pedersen rifling superbly into the bottom right corner from 25 yards out.
Davies missed a great chance to equalise when he fired straight at Tomas Koubek, before the Augsburg goalkeeper also denied efforts from Musiala and Serge Gnabry.
However, Koubek did not cover himself in glory for Bayern's leveller, as Choupo Moting's drive from a tight angle sneaked in at the near post to make it 1-1.
Koubek somewhat redeemed himself shortly after, making a brilliant save to paw away Sadio Mane's goal-bound shot as the visitors put increasing pressure on the Augsburg defence.
Gnabry hit the bar with a deflected effort just after half-time, before Kimmich's curler from the edge of the box flew in to give Bayern the lead.
Choupo-Moting then took advantage of a defensive mix-up to smash home Bayern's third, seemingly putting the match to bed.
And despite Augsburg pulling a goal back when Upamecano diverted into his own net, Davies produced some sublime skill to tee up Musiala to bend home before the Canadian turned goalscorer as Bayern made sure of their progress.