Ralf Rangnick's side crashed out after Anthony Elanga blazed his spot-kick over the bar in a nerve-jangling shoot-out.
Swedish teenager Elanga was the only player from either team to miss in the shoot-out, leaving the forward holding his head in disbelief as Middlesbrough celebrated their stunning victory.
Jadon Sancho had put United ahead in the first half, but Matt Crooks grabbed a controversial equaliser after the interval to set the stage for Middlesbrough's memorable upset.
Crooks' goal was allowed to stand despite Duncan Watmore controlling the ball with his hand before providing the assist.
Cristiano Ronaldo had missed a first half spot-kick for United, who have now lost seven of their last eight penalty shoot-outs, including last season's Europa League final against Villarreal.
United are without a major trophy in five years and their hopes of ending that drought are dwindling given their Premier League struggles and a tricky Champions League last 16 tie against Atletico Madrid.
The FA Cup embarrassment is the latest hammer blow for United at the end of a painful week that saw their England forward Mason Greenwood arrested over the alleged rape and assault of a young woman.
While still in custody, the 20-year-old was further arrested on Tuesday on suspicion of sexual assault and threats to kill.
England international Greenwood, who was released on bail on Wednesday, has been suspended from all football activities at United since his arrest.