Missed the match? Catch Up via beIN SPORTS CONNECT
Carlos Queiroz's side earned all three points at the Arena Fonte Nova in Salvador, despite the coach making 10 changes to the team that beat Qatar last time out.
Gustavo Cuellar's first-half strike, which went in off the right foot of Paraguay goalkeeper Roberto Fernandez, made the difference.
Colombia had a goal disallowed in the second half as well, when substitute James Rodriguez flicked a clever pass with the outside out of his boot to Luis Diaz, who controlled and lashed a shot past Fernandez from 10 yards. After a long wait, a VAR review showed the ball had brushed Diaz's right arm and the goal was ruled out.
Paraguay, which has two points, now faces a nervy wait to find out if they have done enough to progress out of Group C with one of the two best third-placed records.
