The club captain scored just four minutes after David Luiz had equalised as the Gunners fought back to win the Group B clash after a Bernd Leno mistake saw them fall behind early in the second half.
Thomas Partey made his first start for the club in a match from which Rapid will feel it should have taken something, with goalscorer Taxiarchis Fountas unable to capitalise on further Leno errors.
Rapid had 12 shots to Arsenal's 11 but could not avenge a 6-2 aggregate defeat in their only previous meeting back in the 1991-1992 European Cup.
Arsenal managed more yellow cards (three) than shots on target (one) in the first half, with Thomas going into the book for a stray elbow on Fountas.
The best chances fell to David Luiz, who saw a header well saved by Richard Strebinger, and Marcel Ritzmaier, whose clever backheel almost deflected beyond Leno.
It was Leno at fault for the opening goal, though, the Gunners keeping failing to clear under pressure, allowing Fountas to drill a good finish into the bottom-right corner after the ball fell his way.
Another loose kick allowed Fountas to drive towards the penalty area, but this time Leno was on hand to block his low strike.
The Gunners drew level with 20 minutes remaining, Nicolas Pepe's deep free-kick glanced in by David Luiz, who rose unchallenged in the heart of the penalty area.
Just after another Leno error had given Fountas the chance to restore Rapid's lead, Arsenal struck again. Mohamed Elneny slid a ball through to Hector Bellerin and Aubameyang was on hand to tap in his cut-back.