Rybakina, representing Kazakhstan, had not previously been past the second round at Roland Garros but she is enjoying a strong run this fortnight and showed no nerves against her illustrious opponent, hitting a series of impressive winners that the 39-year-old Williams could not match.
After breaking back and then moving 5-4 ahead in the second set, it seemed 23-time grand slam champion Williams might take the match to a decider, but Rybakina reeled off the next three games to seal the biggest win of her career and set up a quarter-final clash against Anastasia Pavlyuchenkova.
Rybakina started in a confident mood, winning her first two service games without dropping a point.
Williams came under intense pressure on her second service game, with Rybakina breaking and opening up a 3-1 lead when the American sent a backhand long.
Williams broke back in the seventh game of the match, but she squandered the opportunity to pull level when Rybakina broke again in the next game.
A nervy Rybakina needed four set points to win the opener 6-3, sealing it in 36 minutes when Williams flashed a cross-court backhand long.
Williams found herself in more trouble at the start of the second set, Rybakina comfortably breaking to move 1-0 up, but the three-time Roland Garros champion restored parity in the next game with four consecutive winners.
Rybakina missed a golden opportunity to set up a break point in the fifth match of the second set, sending a backhand long with Williams out of position, the American surviving to serve out the game and move 3-2 up.
The 21-year-old won the next game with an ace, setting the stage for her to break Williams and move 4-3 ahead.
Williams roared back, though, winning the next two games to leave Rybakina serving to stay in the set.
Not only did she do that, but she won the next two games as well to put the seal on a memorable victory that took one hour and 19 minutes.