Djokovic, chasing a third Roland Garros championship and record-setting 23rd men's Grand Slam title, eased past his 94th-ranked Peruvian opponent, 6-3, 6-2, 6-2.
The 36-year-old Serb, champion in Paris in 2016 and 2021, will face 11th seed Karen Khachanov for a place in the semi-finals.
Djokovic holds a commanding 8-1 career lead over the Russian including their only previous meeting at the French Open in 2020.
"I had never played my opponent before. I knew he was a clay court specialist and that I had to earn the victory. It was the best I played this week," said Djokovic after ensuring a 14th successive last-eight spot at the tournament.
On his record 17th quarter-final in Paris, he added: "I am very proud of this record. I have put a lot of effort into my game and I am very motivated to continue."
On Sunday, Djokovic fought off a break point in the opening game of the first set before stretching out to a convincing 4-0 lead.
Two more breaks then helped him to a comfortable 5-1 advantage in the second set. It took Djokovic just 79 minutes to open a two-set lead.
That was in stark contrast to his gruelling third round win over Spain's Alejandro Davidovich Fokina.
The first two sets then needed two hours and 53 minutes to negotiate and his eventual straight-sets win proved to be the longest three-setter he had ever played, clocked at three hours and 36 minutes.
Varillas, who had never won a Grand Slam match before this year's French Open, continued to wilt, slipping 2-1 down and then 4-1 down in the third before Djokovic put him out of his misery after a shade under two hours on court.