It was only the third league defeat of the season for Mark Anscombe's team and it trims their advantage at the summit to just three points.
Ulster's Springbok scrum-half Ruan Pienaar gave the visitors an early lead with a sixth-minute penalty but it wasn't long before Glasgow lock Tim Swinson continued his scoring run.
The 26-year old bagged a double in their last outing against Newport Dragons and capitalised on an overlap to give the hosts a 15th-minute advantage.
Pienaar threaded the posts again as the game stayed tight before a sublime bit of skill turned the match back in Glasgow's favour.
Scrum half Niko Matawalu, who won man of the match for the fourth straight game, was at the centre of all the home side's creative endeavour and he provided the killer pass which Tommy Seymour grasped to crash over the line, establishing a lead they then never lost.
Glasgow repelled a lengthy spell of Ulster pressure, Pienaar missing a penalty before full-back Peter Murchie increased the hosts' advantage just seconds after half-time following a charge down.
But for the third time Horne missed the conversion, a statistic some feared might be telling after Pienaar kept the visitors within a converted score with his third well-taken penalty.
Glasgow lost Canadian winger DTH van der Merwe to a head injury early in the half as Ulster worked hard to impose their authority on proceedings.
Robbie Diack finally broke the hosts' resolve after a solid spell of domination, only for the referee to call play back for a forward pass.
But Ulster were now starting to sense their rivals were feeling the tiring effects of their increasingly desperate defending.
And, in the 67th minute, the pressure finally told when centre Stuart Olding crashed over in the corner for his first senior try.
But Pienaar missed the conversion from a tight angle to maintain Glasgow's lead at a solitary point with just 13 minutes remaining.
Ulster were now in the ascendancy but another moment of brilliance involving Matawalu established the Scottish side's decisive advantage and secured an all-important bonus fourth try point.
He finished off a move that started with some clever work by Horne inside his own half but the score went unconverted to keep things close and edgy heading into the closing minutes.
|Glasgow Warriors Score Card|
|Ulster Rugby Score Card|