The 33 year-old's contract with the Ligue 1 side was terminated two weeks ago for gross misconduct, after he reportedly skipped training to take part in a reality TV show.
Rami, who spent two years at Marseille after spells with Lille, Valencia, AC Milan and Sevilla, has been handed a one-year deal by Fenerbahce with the option of a further 12 months.
"I'm very happy to be here," he told his new club's official website. "I've watched the derbies before - there is a real passion here.
"I'm going to start training now with my new team-mates. I'm going to try to find myself slowly, then I will prepare for the matches.
"I'm so proud to be here. I will do my best as a part of the Fenerbahce family."
Rami was part of France's World Cup-winning squad last year but did not play a single minute in Russia.