New signings Federico Bessone, Darren Ward, Adam Rooney and Giles Coke all started with Swindon, along with Foderingham in goal for Swindon after being taken off in Sunday's game.
Swindon had plenty of chances but just couldn't score. The game looked to be heading towards a penalty shoot out before in the 88th minute a mix up in the Swindon defence allowed Potter to score, and send Swindon out of the trophy, after reaching the final at Wembley last season.
Swindon return to league action on Saturday with a home game against Leyton Orient.