England was leading 12-11 at Twickenham earlier this month when, with the clock having ticked beyond 80 minutes, Farrell put in a strong, shoulder-led challenge on the Springbok.
On-field official Angus Gardner consulted the TMO but decided there was no foul play, allowing Farrell and England to celebrate a narrow triumph to kick off its November series.
Farrell's challenge caused widespread debate and, with the benefit of hindsight, Gardner believes he should have awarded the visitors the penalty that would have offered a chance to kick for the posts and snatch victory.
"I think in hindsight now, having discussed it with some other referees...I think the general consensus would be that a penalty was probably the outcome there that should have been given," the Australian, crowned Referee of the Year at the World Ruby Awards, told the Will Greenwood podcast.
"I think we need to see a wrap with both arms, and I think in hindsight – although he got pinned – there wasn't a big enough wrap from both arms, really. There was a wrap with one arm, but there wasn't a wrap with the other arm."
When asked why that had not been his opinion at the time, Gardner said: "The angles that I saw with the TMO, which were the head-on angles, showed a clear wrap of the front arm, but it was the back arm which got pinned.
"Of the angles that I was showed in the stadium at the time, that seemed to me to be enough of a wrap for me to constitute a legal tackle.
"It was never high and so all we were looking at was the tackle technique. The collision itself also kind of swayed my decision because it was a big rugby collision and we see these hits in the game.
"We don't always get it right and we understand that there are going to be decisions that are going to heavily influence the game. At this level the expectation is that we do get it right – and that's what we're striving to achieve – but we don't always.
"I suppose that's the best way, just to be honest about it. If I made a mistake, I've got to put my hand up and say I was wrong, and hopefully if I see that again then I'll know where I'm heading."