The Hawks moved level at 2-2 in the Eastern Conference finals with a 110-88 win without Trae Young at State Farm Arena on Tuesday.
Young was ruled out after spraining his ankle when he stepped on a referee's ankle in Game 3.
Atlanta showed they can cope without 2020 All-Star Young on a painful evening for the Bucks, who lost Giannis Antetokounmpo when he suffered a hyperextended left knee.
Williams was charged with the task of replacing Young and top scored for the Hawks with 21 points and eight assists, while Bogdan Bogdanovic finished with 20 points.
Asked about being named as a starter, shooting guard Williams said: "It's not a big adjustment. You just know your minutes are going to go up, the time of the games is going to be different.
"Other than that, you just get ready to play a basketball game. We've got to be pros. I know it's cliche to hear, but one guy goes down, another guy got to step up."
Williams revealed there was no rousing speech from Hawks coach Nate McMillan when he was informed he would be starting.
"Honestly, I was on the training table, Nate walked up, said, 'Trae is going to be out, so I'm going to start you.' I said okay, and he walked off.
"That was the conversation. It's not like a 'Remember the Titans' thing that happens in the locker room. I promise you it don't. That was it. That was the extent of our conversation, and we got ready for the game."
McMillan is unsure if Young will return in Game 5 on Thursday.
"I don't know," McMillan said. "The report was just tonight he's out. He didn't feel comfortable enough to put pressure on his foot tonight. I'll get a report tomorrow, and I'm sure it'll be a game-time decision."