Villa took the lead in the 12th minute, El Ghazi letting fly with a drive from 20 yards that took a decisive deflection off defender Ahmed Hegazi.
West Brom soon levelled though, Dwight Gayle providing the finishing touch with a controlled low volley after Harvey Barnes outmuscled and outpaced Alan Hutton on the break before delivering a fine cross.
John McGinn rattled the post with a 20-yard-drive but, after West Brom regained control of the contest, Villa retook the lead against the run of play in the 58th minute.
El Ghazi had already scored once from outside the box but he was given space to shoot again and picked out the top-right corner with a sensational strike.
Villa seemed set to close out a fifth win in six league games to move into the Championship's play-off places, but they were denied in the 91st minute in stunning circumstances.
A scramble in the box saw Rodriguez throw himself at the ball, the striker seeming to inadvertently handle the ball twice to force the ball over the line from close range.
Rodriguez did not even celebrate as Villa protested but the goal was given, ensuring West Brom moved within three points of the Championship's automatic promotion places.