After saving from Modric, Denmark goalkeeper Kasper Schmeichel produced two saves in the shoot-out but saw opposite number Danijel Subasic keep out three, including from Nicolai Jorgensen to set up Rakitic to seal the 3-2 win.
In a tournament packed with excitement this was no classic, although it looked set to be a thriller after Mathias Jorgensen and Mario Mandzukic scored at either end within the first four minutes.
There was an air of expectation that Croatia's 'golden generation' of players, including Modric, Rakitic and Ivan Perisic, would rise to the occasion but there was little urgency, spark or flair about Zlatko Dalic's team after the early action.
Dalic's side nevertheless booked its place in the last eight alongside Russia, who it will play in Sochi on Sunday (NZST), and it will need to find a more incisive cutting edge in attack if it is to break down Stanislav Cherchesov's resilient defence.
Denmark head coach Age Hareide may feel hard done by after his players showed more intent to win the game in normal time, but Feyenoord striker Jorgensen cracked at the vital moment before Rakitic sent them home.