Hereford took the lead after 30 minutes when the linesman flagged to say the ball had crossed the line after Brezovan had dropped the ball after a shot from Done.
On the stroke of half time Simon Cox equalised when he scored from close range, making the score 1-1 going into half time.
Swindon had a goal ruled out in the second half when it looked to have crossed the line, a bit like Hereford's first half goal, but this time it wasn't given and Hereford were instead given a free kick.
The game looked to be heading towards a penalty shoot out before on 84 minutes Lee Peacock scored the winner with a shot from just inside the box to make it 2-1.
A good win for Swindon which moves them a step closer to Wembley. Swindon now return to league action on Saturday with a home game against Huddersfield.