Leading 2-1 from the first leg after a successful trip to the south of France on Thursday, Nantes were beaten 1-0 at their Stade de la Beaujoire home by the side that finished third in Ligue 2.
Vakoun Bayo headed in a cross from Amine Adli to give Toulouse a 62nd-minute lead, but Nantes resisted further pressure from their visitors.
Toulouse were reduced to 10 players in the 87th minute when Nathan N'Goumou was sent off for a foul on Jean-Charles Castelletto, hitting their hopes of a late second goal.
Nantes won four consecutive games before losing to Montpellier on the final day of the regular season, jumping above Nimes in the top-flight table in the process to finish in 18th place.
Having at one stage looked destined for automatic relegation to the second tier, they have survived under the stewardship of Antoine Kombouare, the experienced head coach who was appointed in February as successor to Raymond Domenech and charged with leading the rescue mission.