A contest dominated by early VAR decisions saw goals from James Tavernier, Alfredo Morelos and Leon Balogun secure a big win for Giovanni van Bronckhorst's men.
The visitos conceded twice, had two goals ruled out for offside, and saw a penalty saved by Allan McGregor in a frenetic opening 25 minutes in Glasgow.
Dejan Stankovic's side will still believe they can turn things around at the Rajko Mitic Stadium next week, where it has won three of its four Europa League home games this season (L1), but only once has it won by more than one goal (4-0 v CFR Cluj).
Red Star made a bright start and Aleksandar Katai had an early goal disallowed for offside, but it was Rangers who took the lead on 10 minutes when a VAR review judged that Ryan Kent had been tripped in the box by Slavoljub Srnic, with Tavernier dispatching the penalty into the top-right corner of the net, despite slipping over.
Everything was going wrong for the visitors as they had a second goal ruled out for offside, before a cross from Ryan Jack off a short corner dropped to Morelos, who fired past Milan Borjan to double the lead for the Scottish champions.
A remarkable opening period continued as Red Star was awarded a penalty on 23 minutes after Jack fouled Guelor Kanga in the box, but McGregor got a strong right hand to Katai's spot kick to keep it out.
Glen Kamara almost made it three before half-time after good work from Calvin Bassey and Kent down the left, but the Finland midfielder lifted his effort just over the bar, though it was three six minutes after the break as Balogun rose highest to plant a header past Borjan from an out-swinging Tavernier corner.
Kanga crashed a shot against the crossbar and substitute El Fardou Ben Nabouhane had a third goal ruled out for offside as Red Star looked for a way back in, while Connor Goldson missed a glorious chance at a fourth at the back post off a Tavernier free-kick, but Rangers will be happy with another impressive win in Europe.