After Hakan Calhanoglu put the Nerazzurri in front in first-half stoppage time at San Siro, Barcelona thought it had levelled when Pedri turned home from close range in the 66th minute.
However, the goal was disallowed after replays showed Inter goalkeeper Andre Onana had tipped Ousmane Dembele's cross onto Ansu Fati's arm before Pedri tapped in.
Inter was again fortunate late on, when VAR initiated a penalty check after the ball appeared to strike Denzel Dumfries' arm in the box. Much to Barcelona's anger, a spot-kick was not awarded.
The host side held on to claim three points, making Xavi the first Barca manager to lose his first three Champions League away games in charge of the club.
Yet all of his frustration was saved for the officials, with Xavi telling reporters: "I am outraged.
"First they explain to us that Ansu Fati handled but another team-mate scored, then with the other incident, it is not clear what happened.
"It is my opinion, I would have liked to speak to the referee, because he did not blow the whistle. At the moment, I am outraged, it is an injustice and it makes no sense.
"Now we still have three finals left, we have already lost in Munich and we start again. But there is indignation.
"In general, it was a great injustice. I can't hide and say I'm not outraged, it's a great injustice. The referee should give explanations, instead he goes away and nothing happens. He has to come here and explain."
Despite what he perceived as poor refereeing from Slavko Vincic, Xavi acknowledged that officiating alone was not to blame for the defeat.
"We struggled in rhythm, in the circulation of the ball and in the last half hour we played better, we found good areas on the wing," he said. "We tried, we want to attack and we paid for the first half, we lacked a bit of rhythm. We need to be self-critical, beyond referee decisions.
"The first half was not up to the Champions League. We talked about it at half-time, we needed more rhythm in the exchange of the ball and I think the second half was positive."