The Patriots had glided through the first eight games of their schedule as they remained unbeaten through the opening half of the campaign, but they came unstuck at M&T Bank Stadium in a 37-20 defeat.
New England's loss drops them to 8-1 and leaves the San Francisco 49ers (8-0) as the NFL's last remaining unbeaten team.
Brady and the Patriots offense endured a particularly difficult evening in Baltimore as New England turned the ball over twice and committed uncharacteristic penalties in a performance that will give hope to their AFC rivals in the race to the Super Bowl.
"Losses always find a way to recalibrate how you see yourself. And we obviously have a lot of work to do," said Brady.
"So just wasn't obviously good enough. Get beat by 17 points, that's not what we're all about. I give the Ravens credit. I think they played really well, took advantage of their opportunities. And that's what it takes."
Head coach Belichick agreed with his quarterback's assessment.
"We've all got to do a better job. That includes everybody -- period," Belichick said. "We competed out there, but it just wasn't good enough. We all have to do a better job, starting with me.
"We didn't do anything well enough to deserve to win."