Barcelona coach Koeman said on Monday he could not understand the failure to award a penalty to Eibar late in their 3-1 defeat to Madrid.
No spot-kick was given and the on-field decision was not reviewed, a call Koeman claimed nine out of 10 people would disagree with.
These comments were relayed to Zidane on Tuesday, to which he replied: "It has been annoying because I never get involved with the referees.
"I think the referees are in charge of the game and they do a good job. Like everyone here, they just try to do their job. I never get involved with them. It's a very difficult job.
"What Koeman said I'm not going to get into that. We have an important game tomorrow [against Granada], a very demanding game, and that's the only thing we're thinking about."
In fact, Zidane believes decisions go against Madrid more often than not.
"It's true that in every game we are unlucky with penalties, but I'm not too focused on that really," Zidane told the media.
"The only thing we're interested in is tomorrow's game. That's what interests us. It's very important to us. We have to be concentrated.
"The most important thing for us is just to keep doing what we're doing now. We're in very good form and [will] try to win all three points tomorrow."
Zidane was also asked about the news of Iker Casillas returning to the club as assistant to the general director at the Real Madrid Foundation.
The Frenchman said: "We know the person Iker is, we know what he has done, we know what he means for being a Madridista. I think it's good news for everyone.
"It's his club, the club that's in his heart, and he's going to work and going to continue to do what he knows. I'm very happy for him and for the club. I think it's very good news indeed."