Osaka, 20, triumphed 6-2, 6-4 in the match marred by Williams's second set outburst, the American enraged by umpire Carlos Ramos's warning for receiving coaching from her box.
She tearfully accused him of being a "thief" and angrily demanded an apology from the official.
When a second code violation for racquet abuse was handed out to her -- along with a point penalty -- Williams exploded.
"You're attacking my character," she said. "You will never, ever be on another court of mine. You are the liar," she fumed and Ramos handed her a game penalty that put Osaka one game from victory at 5-3 in the second set.
Williams won the next game, and continued her tearful remonstrations with a supervisor on the changeover but Osaka -- who displayed not only a stellar game but remarkable poise throughout -- held serve to seal a historic win for her country.
Williams, seeking a first Grand Slam title since the birth of her daughter Olympia on September 1 2017, was denied a 24th Grand Slam title that would have matched Margaret Court's all-time record.
As the pro-Williams crowed booed the announcers at the trophy presentation, Osaka was tearing up herself, but Williams urged the spectators to show the young champion respect.
"She played well," Williams said, pausing to compose herself. "This is her first Grand Slam. "I know you guys were here rooting, but let's make this the best moment we can. Let's give everyone the credit where credit is due. Let's not boo any more. Congratulations Naomi."
When it was Osaka's turn she seemed at a loss.
"I know everyone was cheering for her and I'm sorry it had to end like this," she said.
"It was always my dream to play Serena in the US Open finals," she added, turning to Williams herself. "I'm really grateful I was able to play with you, thank you."