The victory meant Gregg Berhalter clinched a second win from as many games as coach.
Berhalter's reign began with a comfortable 3-0 win over Panama, and although it looked be heading for a frustrating tie, United States, whose squad is made up entirely of players from clubs in North America, finally made the breakthrough towards the end.
A dull first half had few highlights and looked to be evidence of the squad's lack of familiarity, but United States really stepped things up after half-time.
Opportunities became a regular occurrence and United States even hit the post twice, as Costa Rica struggled to hold on, and the visitors' luck finally ran out 10 minutes from time as second-half substitute Lletget opened the scoring, and Arriola wrapped things up.
The closest either team went before the break saw Costa Rica's Marvin Loria blast just over from 25 yards in the early exchanges.
United States looked much more switched on at the start of second half, piling the pressure on almost immediately.
But Cristian Roldan was guilty of an inexplicable miss in the 51st minute, shooting wide from 10 yards despite Gyasi Zardes's header teeing him up perfectly.
Nick Lima then sent a fierce drive against the left-hand post from inside the box, before Djordje Mihailovic shot over from a similar distance.
Costa Rica continued to ride its luck and United States hit the left-hand post once again 22 minutes from time, Roldan's sweetly struck 20-yard attempt hitting the upright.
United States eventually got the goal its second-half display deserved with 80 minutes on the clock, as Lletget met the cross from fellow substitute Jonathan Lewis at the back post, nodding past Esteban Alvarado.
Arriola sealed the win right at the end, clipping a neat finish over the approaching Alvarado.