But the Welsh region were forced to come back from 13-0 down in the first half and 19-14 down at the start of the second to secure the spoils.
Ulster started well opening up a 6-0 lead after 11 minutes through two penalties from fly-half Ian Humphreys. It was a lacklustre start from the Ospreys who then conceded a penalty try on 15 minutes.
Scottish referee Neil Paterson awarded the score when winger Tom Prydie tackled Humphreys without the ball as he chased a ball across the line.
But after a difficult first quarter the Ospreys launched a comeback. Biggar put over a 20th-minute penalty and that was followed six minutes later when flanker Marty Holah was sent over in the left corner from a long pass from centre Ashley Beck.
With the Ulster pack getting penalised for a second time at a set scrum Biggar kicked a penalty to close the gap to just two points.
And Ulster's cause was not helped when Humphreys was forced off with a shoulder injury on 32 minutes to be replaced by Paul Marshall.
There was worse to follow for the Ulstermen just before half-time when Tim Barker was sin-binned for throwing a punch on the floor with Biggar converting the subsequent penalty from five metres inside his own half. It was the last kick of the first period as the Ospreys led 14-13.
Despite being down a man Ulster were kicked back into the lead with a Ruan Pienaar penalty three minutes after the break following a line-out offence, and a further six minutes later the South African was on target again, this time from just outside the 22.
But back to their full complement Ulster gave away three penalties with Biggar kicking penalties on 57, 64 and 67 minutes to give the Ospreys a 23-19 lead.
With seven minutes left on the clock Ulster earned a penalty goal in front of the Ospreys posts, which Pienaar kicked to make the score 23-22 going into the business end of the contest.
Biggar missed a penalty goal deep in to injury time but it mattered little as the Ospreys held on for victory.
|Ospreys Score Card|
|Ulster Rugby Score Card|