Giovanni van Bronckhorst's side had little chance of securing a Europa League spot heading into the final match of its Group A campaign, and any slim hope was dashed when it went 2-0 down within 29 minutes at Ibrox.
Steven Berghuis opened the scoring for the visitor in the fourth minute with Mohammed Kudus doubling Ajax's advantage.
Kudus was on the scoresheet again after the break, though the effort was disallowed for offside by VAR.
Rangers did get a late consolation goal from the penalty spot, tucked away by captain James Tavernier, but Francisco Conceicao rounded things off for Ajax, who finished third and will attempt to qualify for the last 16 of the Europa League in the new year.
It leaves Rangers having conceded 22 goals in their six group games, scoring only twice.
That is only the second time a Scottish side has shipped more than 20 goals in a single campaign in Europe's elite club competition, with Van Bronckhorst's team joining the 1959-1960 Rangers side in that regard.