Vincenzo Italiano's side, who are playing their first ever Serie A season, moved into the last eight thanks to two goals in extra-time after initially letting slip an early two-goal lead.
Roma came into the last 16 clash at the Stadio Olimpico reeling from a 3-0 hammering at the hands of local rivals Lazio at the weekend, and were two down by the 15th minute thanks to Andrey Galabinov's penalty and a sumptious finish from Riccardo Saponara.
However Lorenzo Pellegrini pulled one back for the hosts with a penalty of his own shortly before the break and after wasting a series of chances Roma finally drew level in the 73rd minute through Henrikh Mkhitaryan.
Borja Mayoral, who had already ballooned over one great opportunity, should have put Roma in what would have been a deserved lead with four minutes left of normal time when he pounced on a stray backpass only to somehow shoot wide with the goal at his mercy.
Roma then found themselves down to nine less than two minutes into extra time when Gianluca Mancini picked up a second booking and then goalkeeper Pau Lopez was shown a straight red card for clattering Roberto Piccoli when fluffing a clearance.
Daniele Verde put Spezia ahead in the 107th minute with a fine header against the club he played for as a boy before Saponara completed a famous night for the away side with a fabulous chip one minute from the end.
The final last eight match will be decided on Thursday when Lazio host Parma with the winner facing a clash with Atalanta.