Edinburgh, who were unbeaten before their trip to Belfast were in control of much of the game but came out worse as the leade changed hands several times in the second period.
A crisp drop goal from Chris Paterson got the Gunners off the mark after two minutes and the opening quarter was all about Edinburgh with their pack dominating up front.
Duncan Hodge saw a drop goal come off the upright and Scott Murray charged over only to be adjudged offside. The score was disallowed but Paterson landed a penalty after 12 minutes to stretch Edinburgh’s lead to 6-0.
Ulster hit back though on 20 minutes when Kieran Campbell broke out from his 22 and cut his way through Gunners’ ranks. The resulting penalty was slotted by Humphreys, making up for his miss in the sixth minute when his first attempt rebounded off an upright.
This spurred Ulster to up the ante but, just before the half hour, quick thinking from Mike Blair saw Edinburgh run a penalty out of their own 22.
Simon Webster made ground and the ball was spun right, and Francisco Leonelli hacked on to score. Paterson converted and the visitors were 13-3 up. An injury-time Humphreys penalty narrowed it to 13-6 at the interval.
Five minutes after the restart, it was Ulster whose quick reactions were dramatically rewarded. Tommy Bowe intercepted a pass just outside his 22 and ran the length of the field to score. Humphreys’ conversion drew the sides level.
Paterson’s second penalty put the Gunners back in front, two minutes later but then Ulster came again and Andrew Trimble smashed over after a flat pass from Humphreys.
The Ulster winger’s 50th-minute try went unconverted but Humphreys made amends five minutes later with his third penalty to give Ulster a 21-16 lead.
The lead changed hands again, two minutes before the hour, when Rob Dewey scored under the posts. Paterson converted but the Gunners’ 23-21 lead was cancelled out by Humphreys’ fourth penalty, giving the home side a one-point lead.
This lead was stretched to four points when a fifth penalty from Humphreys put Ulster ahead 27-23. With one minute of normal time remaining another Humphreys penalty completed the scoring.