Thomas Tuchel, absent from the game after a positive COVID-19 test, saw his side battle back from an early Macaulay Gillesphey header through the unlikely duo of Cesar Azpilicueta and Marcos Alonso.
Azpilicueta's stylish finish drew the Blues level but it was not until the 16th minute of the additional period that Alonso struck to send them into the fifth round.
Oh my word, @CesarAzpi that is fiiiiiiiilthy ๐๐คฉ@ChelseaFC #EmiratesFACup pic.twitter.com/ifKbj97SxS
โ Emirates FA Cup (@EmiratesFACup) February 5, 2022
Marcos Alonso finishes calmly for @ChelseaFC ๐#EmiratesFACup pic.twitter.com/3SoqKAZXcy
โ Emirates FA Cup (@EmiratesFACup) February 5, 2022
Plymouth nearly forced a shootout but Ryan Hardie's penalty was comfortably saved by Kepa Arrizabalaga in a dramatic finish, the Chelsea keeper sparing the blushes of Malang Sarr.
The visitors, whose last win over Chelsea was in the League Cup 42 years ago, had previously gone 13 consecutive FA Cup ties against top-flight teams without going through since they beat West Bromwich Albion in 1983-1984.
Their travelling fans were understandably in raptures when Gillesphey outmuscled Romelu Lukaku to glance in Jordan Houghton's cross in just the eighth minute to put them ahead.
WOULD YOU BELIEVE IT ๐ฑ@gillesphey HEADS HOME FOR @only1argyle AGAINST @ChelseaFC ๐คฏ#EmiratesFACup pic.twitter.com/26NBE9bJTQ
โ Emirates FA Cup (@EmiratesFACup) February 5, 2022
Luck then played its part in preserving Plymouth's lead, Mateo Kovacic and Callum Hudson-Odoi each hitting the crossbar and the former rifling a low shot off the left-hand post.
Chelsea got its equaliser four minutes before the break, though, Azpilicueta scoring with a brilliant backheel finish home Mason Mount's low cross from the right.
The Spain defender repeated the trick just before the hour mark, but he was clearly offside after a header from Lukaku, who had earlier failed to meet a teasing Mount cross.
Chelsea could not turn dominance of the ball into many meaningful chances, though, and Kepa Arrizabalaga had to be alert to block an effort from Hardie, who was through on goal.
Michael Cooper then flew to his right to tip over a Mount strike as Plymouth stayed resolute against an attacking quintet of Lukaku, Mount, Ziyech, Timo Werner and Kai Havertz to force the extra 30 minutes.
Just before the extra-time interval, Chelsea finally put a clinical move together, Havertz playing a one-two with Werner before squaring for Alonso to side-foot home.
That looked like being the end of the drama but, in the 117th minute, Hardie won a penalty from Sarr only to see his poor attempt saved by Kepa.
๐งค @kepa_46 with a save from the spot!#EmiratesFACup pic.twitter.com/LGzpj7qqUy
โ Emirates FA Cup (@EmiratesFACup) February 5, 2022