After a sloppy start, The Wallabies powered to victory in the Pool D clash in Sapporo, with Tolu Latu scoring two tries during a much-improved performance in the second half.
They had trailed for most of the contest, taking their first lead with less than 20 minutes remaining.
Fiji capitalised on a nervous and sloppy start from Australia, but its winless run against the Wallabies stretched to 18 games.
The Wallabies were punished early, Ben Volavola landing the first points before Peceli Yato went over down the right after just seven minutes.
Australia needed a response and they got it through their skipper, Michael Hooper crashing over in the 17th minute after strong work from the forward pack.
Fiji had the Wallabies rattled, Australia committing four turnovers in the opening 25 minutes as the boot of Volavola helped Fiji into a 14-7 lead.
However, Australia capitalised on a penalty for a high tackle with its second try, Reece Hodge, who made a couple of vital defensive plays, crossing in the corner with five minutes left in the first half.
The Wallabies were punished for another error early in the second half, Fiji incredibly taking a 21-12 lead, Waisea Nayacalevu racing away after a poor Samu Kerevi pass.
Australia piled on the pressure to get Latu over, but Hodge, who had earlier converted a penalty, missed the chance to give his side its first lead of the game.
The Wallabies were in control by this point, showing far greater patience on their way to taking the lead after Levani Botia was sin-binned, Latu crashing over again from a driving maul.
Australia made its dominance count late and Kerevi ran through a Semi Radradra tackle to pull the Wallabies further clear, Marika Koroibete then sealing the win after a brilliant quick pass from Dane Haylett-Petty.
The Wallabies' campaign continues with a blockbuster clash against Wales on 29 September, while Fiji faces Uruguay on Wednesday.