With head coach Mauricio Pochettino self-isolating after testing positive for COVID-19, assistant coach Jesus Perez took charge of the champion at Stade Raymond Kopa.
He looked set to oversee a frustrating draw before Kurzawa's first goal of the season 20 minutes from time secured back-to-back league wins after last weekend's victory over Brest.
The result moved PSG two points clear of Lyon at the Ligue 1 summit, although Rudi Garcia's side can reclaim top spot if it overcomes Metz on Monday (AEDT).
The hosts started in a confident mood and Pierrick Capelle forced Keylor Navas into a smart save after just two minutes.
PSG belatedly woke from its slumber, with Kylian Mbappe clipping over after racing onto Neymar's pass before the Brazil international saw a close-range effort deflected into the side netting.
Moise Kean then glanced a header wide as PSG ended the first half having failed to register a single shot on target.
Mbappe flashed a strike into the side netting shortly after the interval, while Navas superbly kept out Lois Diony's diving header at the other end.
Paul Bernardoni diverted Mbappe's effort wide after the France star had been played clean through by Neymar, but the Angers goalkeeper could do little about Kurzawa's winner.
Alessandro Florenzi's right-wing cross was deflected into the left-back's path and he powered home a fine volley from eight yards to secure PSG's 10th win in 11 games against Angers.