With Leinster victorious against the Dragons, Mark Anscombe's side now hold a two point lead over their Irish rivals, with Glasgow also in action on Friday, but will feel that they should have wrapped this game up before Kiwi flanker Budd's late heroics.
Treviso started well with full-back Brendan Williams looking dangerous and Di Bernardo notched a penalty after four minutes to put the visitors ahead.
Ulster upped the tempo after that scare and began to spread the ball wide, being rewarded with a penalty but Paddy Jackson continued his kicking woes by missing the target.
Ulster were first to cross the whitewash from first phase lineout ball. Andrew Trimble received a pass from his scrum-half Ruan Pienaar and stepped around the Treviso defence for his eighth try of the season. With Jackson's conversion the league leaders were 7-3 up after 20 minutes.
Jackson extended the Ulstermen's lead with 13 minutes of the half left, slotting a confidence-boosting kick to take the score to 10-3.
Treviso replacement Tommaso Iannone was then sin-binned with eight minutes of the half left for pulling Trimble's shirt and 19-year-old centre Stuart Olding took full advantage of the extra man.
The youngster scored a superb individual try by side-stepping his way past the first line of Treviso's defence and then stepped Fabio Semenzato too for the finish.
Jackson again converted to give the hosts a comfortable 17-3 lead. But then Treviso came on the attack with Williams once again the danger man.
Ulster's Iain Henderson was sin-binned after he kicked the ball out of scrum-half Semenzato's hands as Treviso recycled the ball quickly and the visitors began to batter the Ulster line.
Despite heroic defence from the hosts, flanker Budd got a try before half-time for the Italians which Di Bernardo converted to take the sides in at 17-10.
At the start of the second half, try scorer Budd broke his bind on a scrum and from the resulting penalty Jackson made it 20-10.
Ulster's Lewis Stevenson was yellow-carded for a second high tackle and Robert Barbieri immediately punished Mark Anscombe's side, peeling off the back of a maul for a simple score.
At 20-17, Treviso were within touching distance but Ulster kept it tight and earned a penalty which Jackson converted to take the gap back to six points.
Disaster then struck for Ulster as Ricky Andrew's clearance kick was charged down by Budd.
Paddy Jackson was beaten to the ball by the flanker and he touched down for his second try. With the conversion an upset was on the cards at 23-24.
Jackson edged his team in front with two penalties in the final ten minutes but Ulster lost possession from a kick-off three minutes from time and Budd crashed over for his third try to tie the game in dramatic fashion.
|Ulster Rugby Score Card|
|Benetton Treviso Score Card|
|Alberto di Bernardo||3||1||9|