Peru stuttered to a goalless draw against Venezuela in their Group A opener and Ricardo Gareca's men fell behind to Marcelo Martins' 18th international goal in Rio de Janeiro.
Martins converted a 28th-minute penalty after a VAR check to give Bolivia – which lost 3-0 to host Brazil – an unlikely lead before star forward and captain Guerrero equalised on the stroke of half-time.
Guerrero then provided the assist for Jefferson Farfan's 55th-minute header and substitute Edison Flores added a third in the sixth minute of stoppage time.
Peru controlled proceedings throughout the first half but struggled to create any clear-cut chances and the 2018 World Cup participant was left stunned when Bolivia broke the deadlock against the run of play.
There was confusion when the referee appeared to point to the spot for a handball against Peru's Carlos Zambrano in the 24th minute, but he then signalled for offside.
After a lengthy VAR check, a penalty was eventually awarded and Martins made no mistake as he coolly converted past Pedro Gallese four minutes later to move within two goals of Bolivia's all-time scoring record.
With time almost up in the opening period, Peru drew level. Christian Cueva dispossessed Bolivia on the halfway line and played a defence-splitting pass down the middle for Guerrero, who touched the ball around goalkeeper Carlos Lampe and finished into an empty net.
Farfan put Peru ahead 10 minutes into the second half following a towering header after Guerrero turned provider to tee up the Lokomotiv Moscow veteran.
Bolivia had a golden opportunity to restore parity but Erwin Saavedra blazed over from close range after Raul Castro's shot was parried into his path, with Flores adding some gloss to the scoreline in the 96th minute following a swift counter-attack.