The biennial showdown of CONCACAF region (North American) nations saw a rare start outside the United States for Costa Rica, playing in the national stadium at San Jose.
In another Group B match played there Sunday, Haiti edged Bermuda 2-1.
Oviedo opened the scoring for Costa Rica in the seventh minute and Borges followed in the 19th as the Ticos seized command early.
Elias Aguilar, a midfielder for South Korean side Jeju United, scored on a free kick in the first minute of first half stoppage time to boost Costa Rica's edge to 3-0 at half-time.
Allan Cruz added the final tally in the 75th minute.
Costa Rica, which made a semi-final run in the 2017 Gold Cup, last reached the final in 2002 and last won the trophy in 1989.
Bermuda took a 1-0 half-time lead over Haiti on captain Dante Leverock's header in the second minute of first half stoppage time. Leverock is a centre back for Irish side Sligo Rovers.
Frantzdy Pierrot, a striker for Belgian side Royal Excel Mouscron, scored in the 54th and 66th minutes in response to bring the Haitians victory.
The next Group B matches will be Thursday with Nicaragua against Haiti and Bermuda facing Costa Rica at Frisco, Texas.
Monday's Gold Cup matches will be played at the Jamaican national stadium in Kingston, where the Reggae Boyz will entertain Honduras and Curacao will meet El Salvador in Group C encounters.