The Scottish side, who had been hit by injury this week, collected three tries in a six minute burst in the second half to secure a winning bonus point.
After lee Thomas and Ally Warnock had exchanged penalties the visitors seized the initiative with two well worked tries. First, flanker Robin Sowden-Taylor surged over to give Thomas a simple conversion - and four minutes later, scrum-half Ryan Powell increased the lead with Thomas again converting.
John Weston narrow the gap for the Borders as he burrowed over from close range just before the break with Warnock converting
Penalties were again traded early in the second half and after losing Warnock to the sin bin Craig Morgan went over for the third Blues touchdown - only for Borders to carve out two of their own in quick fire order through Calum MacRae and Warnock, who slotted both conversions.
The thrilling revival continued as Brown forced his way across for the clincher - and Warnock finished of with a last-gasp penalty.