The Italian was fired by the Bundesliga giant in September 2017 after a 3-0 UEFA Champions League defeat at the hands of Paris Saint-Germain.
Ancelotti led Bayern to the Bundesliga title in his only full season in charge, but poor performances in Europe saw him depart.
Rummenigge was tasked with informing the former Chelsea and Real Madrid head coach of Bayern's decision to move him on, with the experience proving emotional for Bayern's chairman.
"First of all, I think Ancelotti is a good manager," former Germany striker Rummenigge said on Thursday. "Unfortunately, Bayern were not going well after a year and a half and Ancelotti did not get the results we were expecting in the second season.
"But Ancelotti is a good man, a real friend for me. He is always incredibly calm. I have worked with a lot of managers, each with different skills and abilities. But only Ancelotti has the ability to always stay calm, even during difficult moments.
"I remember I had the task to sack him. I was about to cry. Ancelotti understood the situation, he gave me a hug and said: 'It's fine. You are not my boss anymore, but we are still friends.'
"So, I cried because I was not expecting those words from him in that moment. He was not upset. I realised he was a really good man."
Ancelotti is now in charge of Napoli, who are fourth in the Serie A table having taken one win from their past four games in all competitions.
The 60-year-old is one of only three coaches to win the Champions League or European Cup three times, along with Bob Paisley and Zinedine Zidane.