The game got off to a bad start for Swindon with them going down 1-0 after only 4 minutes with Adam Boyd opening the scoring from close range.
Things then got worse for Swindon when Hartlepool scored again on 12 minutes. This time it was Sweeney with a lob from the edge of the area over David Lucas, who was back in goal for Swindon after Phil Smith played in the last two games.
Hartlepool came close to adding a third before half time when they hit the bar.
The second half Swindon looked much better, but had to wait until 72 minutes to score when Thomas Dossevi made it 2-1.
Two minutes later Charlie Austin equalised with a close range goal to make it 2-2, and give Swindon their first point of the new season.