Aulas has been unhappy with the decision to cancel the rest of the Ligue 1 season due to the coronavirus pandemic as Lyon finished seventh in the table.
The Lyon president feels officials called off the remainder of the campaign too soon, with the Bundesliga resuming and Premier League, LaLiga and Serie A still trying to restart.
But Le Graet believes Aulas' criticism has gone too far, telling Canal+: "We have known each other for so long. He sees me coming, I see him coming.
"On the other hand, he defends the interest of his club with too much vigour sometimes. Lately, he has gone a little too far.
"I told him either on the phone or through the press, but he is caught up in the desire to play this Champions League."
Lyon is still in the Champions League last 16 and Coupe de la Ligue final.