Substitute Edwin Cardona broke Bolivia's hearts as his stoppage-time goal earned Colombia a dramatic and potentially crucial 3-2 win in Friday's (AEDT) FIFA World Cup qualifier.

Bolivia had fought back valiantly from a two-goal half-time deficit in the punishing La Paz altitude, but Cardona kept his cool in the closing stages to slot past Romel Quinonez in the Bolivia goal.

Colombia was in cruise control at half-time as James Rodriguez set aside criticism of his form at Real Madrid to score the opener from Carlos Bacca's assist, before returning the favour for the AC Milan striker to make it 2-0.

However, Bolivia was rewarded for a fast start in the second half when Juan Arce halved the deficit from the penalty spot and Alejandro Chumacero levelled with a superb strike from the edge of the box just before the hour.

The visitors weathered a second-half storm, though, and Cardona's late intervention helped them move on to seven points from five matches in the CONMEBOL qualifying group for Russia 2018.

Bolivia, which remains on three points, was lively in the opening exchanges and Rudy Cardozo's ambitious 35-yard effort had enough dip on it to trouble David Ospina in the Colombia goal.

It was Colombia that struck first, though. Bacca held the ball up well on the left flank and picked out James, who took one touch before brilliantly drilling across goal into the bottom right-hand corner.

Undeterred, Bolivia saw plenty of the ball with Cardozo a constant nuisance, but they created little and Juan Cuadrado's long-range strike almost caught them out with another sucker punch.

The hosts were dealt a blow when Leonel Morales was accidentally kicked on the back of the head by Cuadrado and was replaced by Marvin Bejarano.

Bolivia's situation worsened four minutes before the break, James the creator on this occasion with a dinked ball that Bacca controlled well in the area before completing the job with a neat finish.

Bolivia hauled itself back into the match five minutes after the break when Jeison Murillo was judged to have handled Rodrigo Ramallo's header in the area and Arce clinically dispatched the resulting penalty to the right of Ospina.

Buoyed by that goal, the hosts pushed for a second and it duly arrived in the 62nd minute.

Ramallo outmuscled Murillo on the left flank before teeing up Chumacero, who provided a brilliant first-time finish that flashed past Ospina to the goalkeeper's right.

Colombia appeared shellshocked and Ospina — who stayed on despite taking a heavy knock — had to be on hand to make a fine save from Ramallo's thumping point-blank header.

However, the visitors finished the stronger and, although Luis Muriel wasted one golden chance, Quinonez was beaten in time added on as Cardona collected Marlos Moreno's throughball and struck a huge winner for Jose Pekerman's side.