The world number one, competing in the Austrian event for the first time since winning it in 2007, saved a set point in a tie-break during the opener on his way to a 7-6 (8-6) 6-3 triumph.
He had to work hard as Krajinovic, who lost to Djokovic in last month's Italian Open at the last-16 stage, twice broke serve in a tight opener, helping establish a 5-3 lead at one point.
Djokovic hit back in the 10th game to break his opponent for a second time as the opener went to a tie-break, which he edged 8-6 after Krajinovic failed to convert an opportunity when 6-5 up.
The second set was far more straightforward for the 33-year-old, however, as he held serve throughout and broke Krajinovic once to progress through.
"We practiced a few times [together] before coming to Vienna," Djokovic revealed in his on-court interview.
"The draw was tough for us to face each other, but there is always extra pressure and importance to a match when you play someone that you know very well."
Borna Coric is up next for Djokovic after the world number 26 beat Taylor Fritz on the opening day of action.
Elsewhere on Tuesday, Grigor Dimitrov held off fellow top-20 player Karen Khachanov 7-6 (8-6) 6-3 in a competitive match, while Hubert Hurkacz beat Attila Balazs in straight sets.
Dan Evans also advanced to the last 16, where Jurij Rodionov awaits, after Aljaz Bedene withdrew with a thigh complaint when a set down.