The two sides face off at Ahmad bin Ali Stadium on Wednesday (AEDT) for the right to return to Qatar later this year and join Germany, Japan and Spain in Group E.
A total of 31 countries have qualified for the global showpiece after Australia's penalty shoot-out win against Peru, leaving one place up for grabs.
New Zealand enters the intercontinental play-off clash as the underdog - 101st in the latest FIFA rankings, 70 places below its Central American opponen.
The All Whites cruised through the extended qualifying process, racking up 5-0 and 7-1 wins along the way, but now face undoubtedly their toughest test yet in Costa Rica.
Whereas New Zealand has only ever qualified for two World Cups, and none since 2010, Costa Rica have qualified five times and is seeking a third straight finals appearance.
Hay's side is in a relaxed mood, though, with the 47 year-old glad that all the pressure is on the opposition.
"The world's media have basically written us off, given us no chance," Hay said at his pre-match news conference. "They're 31 in the world playing against 101 in the world.
"I'm sleeping pretty well. I'm not sure their coach will be, though.
"We've got a great group. They're injury-free, they’re in a good space mentally, that’s why we need to seize the moment now. This really is our moment, this is our time."
New Zealand has been boosted by the return to fitness of captain Winston Reid, who had been struggling with a groin injury.
It warmed up for the match with a 1-0 defeat to Peru and goalless draw against Oman, both of those games being friendlies.
Costa Rica has lost just one of its past nine matches, meanwhile, but coach Luis Fernandez Suarez is not taking New Zealand for granted.
"We'll make sure that the group is strong mentally," he said. "New Zealand are a team that like to make things dangerous.
"They are a team that from the back with their goalkeeper almost always playing inside, from the beginning to the last third, with one of their advantages being aerial football.
"But for every situation or attack New Zealand present, we have prepared something to respond to it."