Abdallah El Said scored in the 89th minute to clinch all three points for Egypt and leave Uganda unable to qualify for the knockout stage, having lost its first two games.
Uganda missed a number of opportunities to take the lead in the first half and Sredojevic pointed to Egypt's Serie A winger Mohamed Salah - who set up the decisive goal - as a factor that gave it the edge.
"We didn't deserve to lose the game. If someone beats you comprehensively, you accept it," Sredojevic said. "But if small details decide the game, it is hard to accept.
"At the time when we were supposed to close out the game, we have allowed a counter-attack.
"The experience of Mohamed Salah set up the game. It was one moment that prevailed, a moment of experience. This level of competition demands total focus."
"We want to go home with element of pride and win the game against Mali."
Egypt needs at least a draw in its final group game against Mali to be certain of qualifying for the quarter-finals, and coach Hector Cuper praised his players' determination.
"I'm satisfied with my players. They have fought till the end on a difficult pitch and in humid conditions," he said.