Andy Gurney fouled ex Swindon striker Tony Thorpe in the box on 15 minutes and the referee awarded Colchester with a penalty which Chris Iwelumo scored from after sending Rhys Evans the wrong way.
Tony Thorpe came close to getting a goal, but the ball was forced out for a corner, and then away to safety.
Both sides have chances to score before the half time break with Rhys Evans making a number of good saves to keep the score down. Andy Nicholas also came close for Swindon.
Andy Gurney was lucky not to get himself sent off when he fouled Tony Thorpe just outside the box, but was only given a yellow card.
In the second half Evans had a make a number of good saves again, and at one point the Colchester fans thought they had scored again, but the ball was cleared away off the line.
A couple of times Evans went down injured, but he was able to carry on which was good as for some reason Swindon didn't name a sub 'keeper.
Swindon made a number of subs with Trevor Benjamin replacing Lee Peacock, and later Charlie Comyn-Platt replacing Andy Nicholas, and finally Christian Roberts replacing Gareth Whalley.
Overall a fairly average game with both sides having chances, so a draw would have been a fair result. The defeat still leaves Swindon only 2 points away from safety, so there is still time for them to stay up even though they are now in 23rd place just above Milton Keynes Dons.