The change of manager was made on the orders of the country's president, Paul Biya, on the back of last month's Africa Cup of Nations disappointment.
The Indomitable Lions had been targeting continental glory on home soil, only to suffer a penalty shoot-out defeat to Egypt in the semi-finals en route to finishing third.
Conceicao was appointed in 2019 and also guided Cameroon to the 2022 World Cup qualifying play-offs, with a two-legged tie against Algeria coming up later this month.
However, the 60 year-old will not be in charge for that double-header, with the Cameroonian Football Federation confirming his departure.
Cameroon announced shortly after that Song, the nation's most capped player with 137 appearances, will succeed Conceicao with immediate effect.
A statement signed by Minister of Sports Narcisse Mouelle Kombi read: "On very high instructions from the President of the Republic, the coach of the men's national football team, Mr Antonio Conceicao, has been replaced by Rigobert Song.
"The Cameroonian Football Federation is invited to take the necessary measures for a rapid and harmonious implementation of these very high directives."
Song had spells with Liverpool and West Ham during his playing career and helped Cameroon to two Africa Cup of Nations crowns.
The 45 year-old has more recently spent time coaching Cameroon's Under-23s side and had a short stint as caretaker boss of the senior side in 2018.
He will be assisted in the role by former Equatorial Guinea and Kenya boss Sebastien Migne.