Neil Lennon's men have made a habit of slow starts this season and were given a wake-up call on 16 minutes when Kristoffer Ajer was penalised for handball and Jason Holt converted from the penalty spot.
However, the Scottish champion responded immediately with two goals in three minutes to turn the game around.
Callum McGregor's deflected strike brought up his 50th goal for the club before Ryan Christie continued his free-scoring form with a close range finish form Greg Taylor's cross.
Lennon left out star striker Odsonne Edouard ahead of Friday's (AEST) UEFA Europa League trip to Latvia to face Riga.
But Albian Ajeti is already proving an able deputy to the France Uunder-21 international as he fired home his fourth goal in five league appearances since arriving from West Ham United last month seven minutes into the second-half.
Celtic was made to sweat for the final 12 minutes when Julien Serrano's brilliant long-range strike cut Livie's deficit back to a single goal.
The host held out to record a fifth straight league win and move above Rangers – which travels to third-placed Hibs on Monday (AEST) – on goal difference.
Elsewhere, Scotland international Lawrence Shankland scored his first goal back in the top-flight for Dundee United as it beat St Mirren 2-1.
Kilmarnock was also a 2-1 victor over Hamilton despite Eamonn Brophy's first-half red card, thanks to goals from Greg Kiltie and Nicke Kabamba
Ross County, meanwhile, won 1-0 at St Johnstone.