The host was behind for the bulk of the contest after Cenk Tosun's 39th minute opener but Sergio Aguero punctuated the frustration with his 250th goal in City colours eight minutes from time.
Aguero duly headed home Benjamin Mendy's cross to put Guardiola's men ahead but there was a sting in the tail and Wilfried Zaha tore past the timid efforts of John Stones and forced Fernandinho to put through his own goal.
"We have to avoid the mistake we have made on the second goal," Guardiola told a post-match news conference.
"Just do not allow Zaha to run, it’s that simple. And we allowed it.
"No [it was not a loss of concentration], it was a decision. Concentration was always there. We are going to solve it and improve it.
"In those positions we need to have the back four there and not let them run on this counter-attack."
Although he did not name the England international specifically, this felt like a damning indictment of Stones' efforts in what has been a difficult campaign for a centre-back frequently short of form and fitness.
At the end of his forlorn pursuit of Zaha, Stones appeared to jar a knee but Guardiola could not offer a fitness update.
"I didn't see the doctors. Now we are going to see," he said.
Champions City are 13 points behind leaders Liverpool having played two games more and Guardiola gave a weary response to a question over whether the title race was still alive – suggesting he might be better served preparing for his team's Tuesday assignment at Sheffield United as opposed to watching Manchester United's trip to Anfield on Sunday.
"I answered the question millions of times in the past few weeks," he added.
"Maybe I will watch it, maybe I will see Sheffield United. I don't know."
City thought they had a penalty at 1-0 down, only for referee Graham Scott to overturn his decision after a VAR review of the incident when Jairo Riedewald deflected Raheem Sterling's cross on to his own arm.
Guardiola said of the incident. "I don’t know the rule of the hands, honestly. It looks like hands, but…"