WATCH Liverpool TV on beIN SPORTS | via Foxtel, Kayo, Fetch & beIN SPORTS CONNECT
Milner was forced off in the 38th minute of Liverpool's 3-2 Carabo Cup defeat to Manchester City at the Etihad Stadium after sustaining a hamstring injury against his former club.
He will now be unavailable for the clash with another of his former teams, Villa, on December 27 (AEDT) and the December 31 (AEDT) home match against Leicester.
Firmino missed the loss to City with a calf injury sustained in training, while Alex Oxlade-Chamberlain has been dealing with a virus that kept Trent Alexander-Arnold out of the last-16 defeat.
Oxlade-Chamberlain came on as a substitute against City.
"Milly [Milner] will be out for a couple of games," Klopp said.
"Same for Bobby [Firmino], even though Bobby is already on the better side of it, it's not a major thing and we will go day by day with him.
"Ox is fine. No other stuff. We have to be careful [with the viruses going around] like everyone else.
"Trent hopefully will be better. He hasn't taken part in a full session, let's see how that develops until tomorrow."
Liverpool resumes the league season after the World Cup break sitting 15 points behind leader Arsenal and seven behind fourth-placed Tottenham.
The Gunners' renaissance and Newcastle United's emergence as a Champions League contender is complicating Liverpool's pursuit of a place in the top four but Klopp remains positive.
"We are really excited [about the Premier League season restarting]. All the players who are back from the World Cup look really on it," Klopp said.
"Hendo [Jordan Henderson], for example, had a good World Cup and that was helpful.
"We have left a gap between us and much more exciting positions in the table. We have to chase.
"There is a lot to come, we don't think about the break any more.
"It's a start for something new and building on the things we've done so far and knowing and expecting that we can do better – starting on Boxing Day."
