The Norwegian will play defending champion Daniil Medvedev in the semi-finals after a gruelling 2-6 7-5 7-6 (7-5) victory over Rublev in Turin on Saturday (AEDT) that lasted two hours and 26 minutes.
Ruud had lost all four of his previous meetings with Rublev, who promptly moved ahead by comfortably winning his ninth straight set in this head-to-head.
Rublev then looked poised for victory when he struck first in the second set but Ruud, who hit 39 winners, fought back with two breaks of his own to force the decider in a high-quality contest.
The 22-year-old again bounced back from being broken in the third set and ultimately prevailed in a tense tie-break, producing a big serve when it mattered most with his second match point.
Ruud therefore finishes second to Novak Djokovic in the Green Group, with Rublev exiting at the round-robin stage for a second straight year after narrowly losing the winner-takes-all clash.
With Alexander Zverev also through, this is the first time since 1994 that three of the four semi-finalists are former champions, with Ruud the exception.