The Colombia international has spent the past two seasons on loan at the Allianz Arena from Real Madrid, winning two Bundesliga titles and a DFB-Pokal.
Bayern reportedly had the option to sign James for €42million and Rummenigge claimed as recently as March that the club were hopeful of making his transfer permanent.
However, it has now been revealed that will not be the case, with James having told Bayern he did not wish to stay.
Rummenigge told Sport 1: "In principle, the decision was made because he [James] came to me before the end of the season and asked me not to activate the option.
"His agent [Jorge Mendes] also called me again last week. Activating the option now against his will makes no sense.
"You should not hire a player for €42million with an appropriate salary if you cannot offer him a firm position."
James has been linked with a switch to Napoli and a reunion with his former Madrid boss Carlo Ancelotti, the man who brought him to Munich.
He scored 14 Bundesliga goals in 43 appearances for Bayern and Rummenigge said the club's decision is not a reflection on his quality.
"I've always said that I'm a fan of his game culture. His left foot is exceptional, his scoring also," he added.
"He was a high-quality player [for Bayern]."