City was 2-0 down after 29 minutes as Swansea's excellent first-half performance put it on course for a famous upset.
But Bernardo Silva's fine outside-of-the-boot effort pulled one back 21 minutes from the end, before Raheem Sterling contentiously won a penalty despite Cameron Carter-Vickers appearing to get the ball, Sergio Aguero's effort going in off goalkeeper Kristoffer Nordfeldt.
Aguero then completed the turnaround at the end with a header, despite appearing to stray offside, and had VAR been in use, City's second and third goals may have been disallowed.
The technology is in use in the FA Cup and was available at Liberty Stadium last season, but the Football Association is reported to have opted against equipping the ground with VAR since its relegation to the Championship and Guardiola was rueful.
Speaking in his post-match news conference, Guardiola said: "All around Europe there is VAR, here there is not. I don't like to win in that way. I am sorry."
The win saw City alive hopes of winning a famous quadruple, although Guardiola is not getting carried away thinking about such achievements.
"Ask me again in April, the end of April, ask me this question and I will answer," Guardiola said when asked if his side can complete the quadruple.
"I want to congratulate the team and everybody. We are satisfied to be in contention. I know I'll be a failure if I don't win all the titles, I'm sorry," the City boss quipped.