The host scored two tries through the lively Louis Rees-Zammit on his first Test start and scrum-half Rhys Webb, with Callum Sheedy scoring the rest of the points with the boot.
The laboured manner of the victory will not have eased pressure on head coach Wayne Pivac whose side faces a far tougher task in the final pool match against unbeaten England next weekend.
Pivac had given three players their Test debuts in the starting line-up – a fourth when Lloyd came on in the second-half – and said he was pleased with the performance.
"I am very happy with the team who played in pretty poor conditions," he said.
"I was very, very happy with the set piece as it had not gone well in the past couple of games but tonight, while not perfect, it was a step in the right direction."
The New Zealander, whose only other Test win since taking over from Warren Gatland came over Six Nations wooden spoon holder Italy, said a win over England would be some consolation.
"It would certainly make up for a lot of the performances this year," he said.
Rees-Zammit settled Welsh nerves in the first-half as the 19-year-old winger scored his first Test try when he was set up by a lovely pass by Sheedy.
Sheedy, who had out the hosts ahead with a penalty, converted brilliantly from the touchline for 10-0 approaching the half-hour mark.
Georgia came more into the match in the final 10 minutes of the half but spurned a chance to reduce the deficit to a converted try.
Five-eighth Tedo Abzhandadze sent his penalty kick wide of the posts right on half-time, missing an opportunity to bring up his century of Test points.
Sheedy, making his first Test start, added another penalty to make it 13-0 early in the second-half.
Rees-Zammit showed a flash of brilliance to light up a dour encounter with a scintillating solo run carving open the Georgia defence.
The visitors were reduced to 14 men when referee Luke Pearce sin-binned Georgia flanker Beka Saginadze for a swinging forearm connecting with Justin Tipuric's head as the Wales captain was tackled.
A dazed Tipuric was wheeled off casting doubts on whether he will be fit for the England game next weekend.
"That was a cheap shot," Pivac said.
Wales rarely threatened the Georgia try-line in the second-half, save when new cap James Botham was set free down the right wing.
However, the grandson of England cricket legend Ian Botham was brought down yards from scoring a try on his debut.
Wales did finally add a second try when Sheedy and Rees-Zammit combined nicely, the winger looking back inside to send Webb over.