John Cooney was the match-winner for the Irish side as he kicked all of its points with penalties, including an 80th-minute effort that denied the visitors victory at the last.
Scarlets scored the only try early on through Rhys Patchell, but the game developed into a direct battle between the boots of Cooney and Dan Jones in the second half.
It appeared as though Jones and the Welsh outfit would come out on top, even after David Bulbring's yellow card, only for Ulster to have the last laugh.
Elsewhere, Glasgow Warriors edged a similarly tight encounter 27-26 at Connacht.
Jack Carty's kicking had the hosts in control despite a first half in which they were outscored three tries to two, but Adam Ashe crossed and Stuart Hogg dispatched a late drop goal to secure victory, with the Warriors surviving a scare as Craig Ronaldson fired against the post.
Six-try Munster thrashed the Cheetahs 38-0 to begin the campaign in style, while Benetton scraped past the Dragons 21-17.