Arsenal took the lead in bizarre circumstances at Old Trafford, Emile Smith Rowe volleying home from 20 yards while David de Gea laid on the ground injured after a collision with Fred.
Referee Martin Atkinson did not blow his whistle until the ball crossed the line and following a VAR review the goal was allowed to stand.
However, Bruno Fernandes drew United level in his 100th appearance for the club and Cristiano Ronaldo put the Red Devils in front with his 800th career goal.
Martin Odegaard restored parity three minutes later but his foul on Fred gave away a penalty that Ronaldo thundered home, giving United a much-needed win and denying Arsenal a chance to move into the Premier League's top four.
"It is what it is. When you concede three goals at Old Trafford you make it difficult to get a result," Arteta said.
"I'm disappointed with how we conceded some of the goals. We had some really good periods, getting back in the game and scoring the second and then we throw it away. After we needed two goals to win it and we weren't able to do it.
"I think we've come a long way since the game at Anfield [a 4-0 loss to Liverpool] because we were able to maintain a high level for longer periods. We had a period after scoring where we just kept giving the ball away in dangerous areas and you'll get punished against this opponent.
"We had very good structure. We were a threat. They couldn't run because we had a good structure in our counter-press, but when it was 2-2 and we had to build our momentum to go and win the game we conceded a penalty.
"There were a lot of things I liked in the game. I really liked the behaviour of the team, individually some of the performances were really good. We had some really good moments in the game and when you score two at Old Trafford you expect to get something out of the game. That's a regret but it's our own fault.
"If you look at the game, you would say we deserved much more than what we got. But you get punished against this level of opponent, this level of players. You cannot do certain things that we did."
The defeat was Arsenal's 25th in the Premier League against United, which is more than they have suffered against any other opponent in the history of the competition.