The 23-time grand slam champion had not played since her victory in the 2017 Australian Open, which she won while pregnant, having given birth to her first child in September.
The USA had already won the tie 3-0 by the time Serena and Venus took the court to a rapturous reception from the home crowd in Asheville, North Carolina.
But there was to be no icing on the cake after clinching qualification for the semi-finals, as the Dutch pair of Lesley Kerkhove and Demi Schuurs spoiled the party with a 6-2 6-3 win.
Serena made a rusty start to proceedings as Kerkhove and Schuurs broke in the first game and then consolidated with a hold as the 36 year-old ballooned a simple overhead smash wide.
She continued to look like a player out of match practice as Netherlands then moved into a 3-0 lead and, though that advantage was reduced to 3-2, Kerkhove and Schuurs rattled off the next three games to take the set.
Despite improving as the match progressed, specifically with her reactions at the net, Serena could not prevent Netherlands from breaking again to start the second set, which it controlled to wrap up a win that was sealed by a Venus double-fault.
Speaking after the match, which took place with daughter Alexis Olympia Ohanian Jr. courtside, Serena said: "It was really special to be out here with Venus and we're so happy that team USA won.
"It's her first match so I'm glad she got to see it. It's been really, really fun."