Swindon had more of the chances throughout the game with Flint coming close in the first half with his header hitting the bar.
Ritchie also came close when he rounded the 'keeper, but then hit the ball just wide as the angle was too tight.
Loan signing Ronan Murray came on mid way through the first half to replace Kerrouche.
Swindon did well during the second half, although Colchester looked better as well, but it was Swindon who took the lead with a 25 yard shot from Ritchie finding the top corner of the net.
Colchester came close to getting an equaliser including chances in the final few minutes, but it was Swindon who were able to hold on to book their place in Round 3.