Ulster hit the front early on and managed to hang on for a rousing Ravenhill success, leaving the Ospreys to reflect on a losing start to their Magners League title defence.
Richard Fussell marked his debut for the Ospreys with a try and out-half Dan Biggar supplied the rest of their points for a handsome 21-point haul, but dogged Ulster came out on top in this titanic tournament opener.
A fumble by the Ospreys' stand-in captain Jerry Collins was punished by flanker Stephen Ferris for Ulster's opening try, and centre Darren Cave added a second for an 18-16 interval lead.
Biggar had sandwiched Fussell's 29th-minute effort with four successful place kicks and he converted his own try, early in the second half, to give the Welsh region the advantage.
Biggar and his opposite number, Niall O'Connor, traded penalties coming up to the hour mark before the Ulster number 10 landed the match-winning penalty in the 71st minute, handing Brian McLaughlin's men a morale-boosting opening win.
Owing to injuries and international commitments, the Ospreys only had five of their grand final-winning side involved from the start – Biggar, Collins, Marty Holah, Jonathan Thomas and Andrew Bishop.
Mike Phillips, Ryan Jones and ex-Ulster winger Tommy Bowe were among the visitors' absentees, but Collins' quicker than expected recovery from knee and shoulder surgery did give the defending league champions a timely boost.
Ulster also had team captain Rory Best and Ferris making welcome returns to competitive action, while new signings Adam D'Arcy, Johann Muller, Pedrie Wannenburg and young wingers Tommy Seymour and David McIlwaine made their competitive debuts.
The home side were guilty of some sloppy play early on, and Biggar was able to slot a fourth minute penalty for the first points. O'Connor replied with an 11th penalty as Ulster began to exert some pressure.
The opening try of the evening arrived midway through the half. Ulster had a lineout inside the Ospreys' 22, Best's throw missed Muller and came off the hand of Collins, allowing Ferris to gather and gleefully race clear for the try-line.
O'Connor landed the resulting conversion, before his 22nd minute penalty pushed Ulster into quite a comfortable position – 13-3.
The Ospreys came to Belfast having had four league points deducted for failing to field a side for a re-arranged date in last season's corresponding fixture, and they soon had the bit between their teeth.
Biggar punished an Ulster offside with his second penalty and even after winger Nikki Walker was sin-binned for taking out McIlwaine without the ball, the Welshmen responded with a converted try.
New man Fussell was the scorer, taking a final pass from Sonny Parker before finishing off a six-man move in the left corner.
Biggar swung over the difficult conversion but as the game continued to ebb and flow, Ulster hit back soon after. Seven minutes before half-time, replacement Paddy Wallace did well to link with his centre partner Cave and he beat the tackle of Holah to cross for an unconverted try.
A third penalty from Biggar reduced the deficit to two for the break, and although O'Connor cancelled that strike out on the resumption, Ulster leaked a second try just a minute later.
O'Connor's loose pass in midfield went to ground, Bishop hacked it on and Biggar won the race for the touchdown, adding the conversion for good measure.
O'Connor redeemed himself by nailing the second of two penalty attempts, Biggar was not to be outdone and he pumped a long range penalty over, as the game entered its final quarter.
26-24 ahead, the Ospreys brought on fresh legs, including veteran prop Paul James who took his caps record for the region to 150.
The game was tantalisingly balanced, with errors on both sides but a huge amount of endeavour. Chasing just their second win over the Ospreys in eight meetings, Ulster snapped back into the lead when referee Peter Allan spotted the Ospreys chasers in front of the kicker.
O'Connor delivered what proved to be the match-winning score, and almost added a drop goal following eleven phases of possession from the province's hard grafting pack.
The Ospreys could not find the same control when going in search of a late leveller. A knock on from replacement Lee Byrne signalled the end for Sean Holley's charges.
Speaking after the game, Ospreys coach Holley admitted: "We probably did enough to get a win from that game. Our discipline was poor, it was very disappointing – there's no escaping that.
"There were some strange interpretations out there at times but we can't hide behind that, we gave away far too many penalties.
"There were some positives that we'll take away from the game though. I thought our set piece was good, while Dan Biggar orchestrated the game very well."
Pleased with how his side battled right until the end, Ulster boss Brian McLaughlin said: "It was a brilliant second half, it was backs to the wall but we got stuck in and we came away with what we wanted, a victory we so badly wanted.
"It was very close in the end. All credit to the guys, they stuck in there in the second, up in their 22, and we came away with the points that we needed.
"Our defence was excellent – all night it was excellent. We could have done better with the ball, at times, but certainly our defence was brilliant."
He added: "Big performances (by the players), you can't single anyone out tonight. Man-of-the-match doesn't matter. It was the whole squad, 23 boys, a superb effort – well done by them all."