Also pushed to five sets in the opening round, Nishikori was tested again before claiming a 6-3 7-6 (8-6) 5-7 5-7 7-6 (10-7) victory after three hours, 48 minutes on Margaret Court Arena.
It was just the third men's match to need a super tie-break, introduced at the tournament this year, at the Australian Open.
Karlovic, 39, served 59 aces but was unable to hold off Nishikori in an enthralling final-set tie-break.
Nishikori, the eighth seed, held from 0-40 down in the ninth game of the fifth set and it proved to be crucial.
A three-time quarter-finalist in Melbourne, the Japanese star will face either Philipp Kohlschreiber or Joao Sousa in the third round.