Making only a third Test start, the unflappable Smith made the decisive contribution in a match that looked like getting away from the White Rose after an enterprising performance was mixed with ill-discipline.
England outscored South Africa three tries to one but a penalty count of 18 allowed the Springboks to stay in the game and the visitors were leading until the last minute.
However, the win marks a measure of revenge for Eddie Jones' side, which was beaten in the 2019 Rugby World Cup final the last time these two sides went head-to-head.
England was dominant in the first half and Manu Tuilagi crossed after fine work from Smith opened the play for Henry Slade to make the long pass.
The nerveless Handre Pollard put South Africa on the board prior to Freddie Steward showing great power to bundle over from close range and extend England's lead.
But England's lack of composure at the breakdown meant the lead was only five points at the break, with Pollard nailing three more kicks to Smith's one, and more poor discipline saw South Africa inch ahead thanks to Pollard and Elton Jantjies kicks.
A fine breakaway score from Joe Marchant was answered by an unconverted score from Makazole Mapimpi after Will Stuart was sent to the sin bin, and Frans Steyn kicked another penalty for South Africa.
But Springboks skipper Siya Kolisi saw yellow himself, before another penalty in front of the posts in the final minute allowed Smith to have the final say.