Shefki Kuqi put Palace in front after eight minutes and Jobi McAnuff headed a second on 70 minutes. Jerel Ifil's late goal was not enough for Paul Sturrock's plucky Swindon side.
The first half was an even affair despite Swindon going behind in the early stages. Indeed the Robins gave Palace a run for their money and Christian Roberts whipped over a couple of dangerous crosses which had the hosts defending desperately.
Stuart Green had had a shot saved by Phil Smith before McAnuff tested the Swindon custodian again, this time a corner resulting from it. Green's flag kick was perfect and Kuqi towered above everyone to nod firmly into the net.
If only Jamie Vincent, a former Palace man, had enjoyed better luck at the other end. Michael Timlin, eligible to play after Fulham agreed to extend his loan period, placed his corner perfectly into the goalmouth but Vincent, with the goal at his mercy, headed agonisingly wide of the post.
The incessant rain refused to ease up and neither did Swindon. Ricky Shakes raced clear but hesitated as he allowed Leon Cort to catch him up and make a crucial tackle.
Swindon were enjoying their best spell of the match as Michael Pook watched his effort cleared off the line by Cort.
After the break the Robins continued to press. In terms of possession they were well ahead but credit to the hosts who defended well with keeper Scott Flinders not really having a serious save to make.
Although Roberts recklessly shot over the bar when well placed, ironically it was Palace that created the better chances.
Green latched on to a Dougie Freedman pass and chipped over Smith but his effort hit the crossbar and rebounded to safety.
Then Kuqi had a powerful header saved by Smith before McAnuff brought relief to Palace. A free-kick from the halfway line was launched into the box and the ball fell nicely to McAnuff, who needed two touches before unleashing a tremendous right-foot shot into the top corner of the net.
Swindon never gave up and deservedly closed the gap when Aaron Brown's corner was sweetly headed past Flinders by Ifil.
That ensured Palace were forced to endure an anxious finish as Shakes drove over the bar while Darren Ward and Cort had to make desperate tackles as panicky Palace kept them out to earn a place into the next round.