Having been thrashed 4-1 by Mexico in their opening game of Group A, France hit back to keep their chances of progression alive, though they had to do so the hard way in Saitama.
Indeed, South Africa led on three occasions during a remarkable second half, yet it was Les Bleus who took the points.
Liberato Cacace's stunner was not enough to propel New Zealand to victory against Honduras in Group B, while reigning champions Brazil could only draw 0-0 with the Ivory Coast and Spain overcame Australia thanks to Mikel Oyarzabal's late winner.
GIGNAC AND SAVANIER SETTLE THRILLER
France were on the wrong end of conceding four last time out, and though they still let three in this time around, Teji Savanier's stoppage-time goal saw them clinch victory.
It was 35-year-old Gignac who had previously dragged France level on three occasions – a close-range finish followed by a thumping header before the Tigres striker, the leading scorer in the competition, converted his second penalty of the tournament to cancel out Teboho Mokoena's glorious 81st-minute strike.
Luther Singh had missed a first-half penalty for South Africa, with Kobamelo Kodisang and Evidence Makgopa also getting on the scoresheet.
But Gignac's efforts set the stage for France to nose themselves ahead when Montpellier's experienced Savanier lashed home, sending South Africa out in the process.
France sit third in Group A, three points behind leaders Japan, who survived a late scare to beat Mexico 2-1.
ALL WHITES LET LEAD SLIP
Cacace's exceptional strike put New Zealand ahead after just 10 minutes in Kashima – the defender picking out the top-right corner with a first-time effort from 30 yards out.
Luis Palma restored parity in first-half stoppage time, yet Chris Wood carried on his scoring form to put New Zealand back in front just after the restart.
Yet substitutes proved the difference for Honduras. Juan Obregon netted with 12 minutes remaining, paving the way for Rigoberto Rivas to clinch the comeback victory.
Matters were much more simple for South Korea, as they cruised to a 4-0 triumph against Romania, meaning all four Group B teams are on four points heading into the final matchday.
OYARZABAL RECORDS LONG-AWAITED SPAIN WIN
Spain secured their first win at an Olympic Games since 2000 thanks to Oyarzabal's header against Australia, which came with nine minutes of normal time remaining.
The 24-year-old Real Sociedad winger – who played a pivotal role in Spain's run to the semi-finals at Euro 2020 – has put La Roja top of Group C heading into Wednesday's encounter with Argentina, who beat Egypt.
Spain, who drew with Egypt in their opener, won gold in 1992 and silver in 2000, but had only qualified once since then, but they failed to score across their three appearances in the 2012 London Games.
CHAMPIONS HELD AS GERMANY BOUNCE BACK
Brazil sparkled in their 4-2 win over Germany on matchday one, though they could not find a way past the Ivory Coast, who held them to a 0-0 draw in Group D.
Douglas Luiz was sent off early on for Brazil, who missed a last-gasp chance to seal qualification through substitute Malcom after the Ivory Coast had also been reduced to 10 men.
Meanwhile, Germany recovered from their defeat on matchday one to beat Saudi Arabia 3-2 – Felix Uduokhai netting the decisive goal in the 75th minute.