The visitor had won on its last three visits to Selhurst Park but struggled on this occasion to open up the determined Eagles, managed by former Liverpool and England boss Roy Hodgson.
However, Milner crucially converted from the penalty spot after a foul on Mohamed Salah at the end of the first half, with Mane's breakaway goal wrapping up the points deep into stoppage time.
Palace opened its campaign by winning at Fulham but never looked likely to equalise after falling behind, especially once the otherwise outstanding Aaron Wan-Bissaka saw red for a clumsy foul on Salah, with Jurgen Klopp's men looking on track for a title challenge.
Liverpool thought it had scored in the 21st minute as Wayne Hennessey inadvertently threw the ball in his own net, but referee Michael Oliver gave a foul for Mane's challenge on the Wales goalkeeper, who also collided with team-mate Wan-Bissaka in mid-air.
Naby Keita sent Salah racing clear but while last season's Golden Boot winner collected the ball neatly, his lobbed finish easily cleared the crossbar. Andros Townsend was closer at the other end, though, seeing a 30-yard curler crash back off the woodwork.
But Liverpool took the lead on the stroke of half-time, Milner confidently tucking home a penalty when former Liverpool defender Mamadou Sakho was adjudged to have tripped Salah in the box.
Luka Milivojevic's free-kick forced a flying save from Alisson early in the second half, but hopes of a home fightback were extinguished with 15 minutes to go when Wan-Bissaka walked for bumping over Salah as the Egypt international raced through on goal.
Palace was duly caught out in the 93rd minute, Salah sending Mane through as Liverpool countered from defending a corner, the Senegal forward holding off Patrick van Aanholt before rounding Hennessey to tuck in his sixth goal in eight games against the Eagles.
Next up Liverpool returns home for a fixture against Brighton and Hove Albion, which beat Manchester United last time out, while Palace makes the short trip to Watford - Javi Gracia's men have two wins from two - the following day.