The Rossoneri were denied a win in Gattuso's first game as head coach thanks to Brignoli's incredible header, which snatched a 2-2 draw for the home side.
The result ended a run of 14 consecutive defeats for Benevento in Serie A and put a further dent in Milan's diminishing UEFA Champions League hopes, with the seven-time European Cup winner seventh on the table.
Gattuso was left stunned at the manner in which his side conceded but suggested it had been coming after 15 minutes of pressure after Alessio Romagnoli's red card.
"It certainly burns, it hurts. It would've been better to be stabbed," he told Sky Sport Italia. "It hurts to concede goals like that. We suffered in the last 15 minutes but we conceded a strange goal. I would never expect to concede from a goalkeeper in the 95th minute."
Gattuso was seen confronting the officials at full-time but he denied that referee Maurizio Mariani or his players were to blame for the extraordinary end to the match.
"The referee has nothing to do with it. I have to thank the boys, who gave it their all," Gattuso said. "We conceded a strange goal and that's just how our luck is at the moment.
"Today, we lick our wounds, we have a duty to improve our physical condition and our mentality, but I can't reproach them for anything.
"We could have developed our play a lot better when moving forward on some occasions. Benevento have lost their last games without deserving to. They're a lively team and we knew it would be difficult. We paid dearly for the first mistake and we go forward.
"We have a long way to go and a lot to improve."