Nguyen Quang Hai fired home the first goal of the game with a 30-yard free-kick, helping Vietnam to a crucial three points that could see it finish as one the best four third-placed sides.
Quang Hai's effort was a rare moment of quality in a scrappy encounter between two sides who had lost both of their opening matches in the United Arab Emirates.
A penalty from Que Ngoc Hai wrapped things up for the Golden Dragons, leaving debutants Yemen rooted to the bottom of the group and out of the competition.
With a draw suiting neither side, attacking play should have been the order of the day but both goals were not threatened in the opening half an hour.
The game's first moment of real quality came eight minutes later, although Yemen were aggrieved at the awarding of a free-kick for a foul by Mudir Abdurabu.
Quang Hai rubbed salt into their wounds with a stunning finish from the set piece, the forward curling his effort over the wall and beyond Salem Al-Haresh.
Yemen tried to up the tempo after the break to keep their dreams alive, however it ultimately cost them as Vietnam were awarded a penalty in the 64th minute.
Phan Van Duc's piercing run was halted by a clumsy Abdulaziz Al-Gumaei tackle in the area, and captain Ngoc Hai made no mistake from 12 yards, sending Al-Haresh the wrong way.
Vietnam must now wait and see if it will progress to the last 16, with the final four group stage matches taking place.