Colchester started off the better side with Griemink forced to make a string of saves to keep Swindon in the game.
Just before half time as Izzet was brought down in the area by Matt Hewlett, and Keith scored from the penalty spot.
On the hour mark Sabin was fouled in the area by Scott Fitzgerald, and Swindon's captain for the game scored from the penalty spot.
Minutes later put Swindon in the lead from close range. Andy Gurney made sure of all 3 points on 74 minutes when his 40 yard free kick went straight in leaving the keeper standing.
The win moves Swindon up to 14th.
After the game Andy King said: Well, after a disappointing run in which where we had drawn too many games, the lads did exactly what was asked of them against Colchester.
“Since my return it has been a frustrating period in terms of results. If you cast your minds back to the Bournemouth game at home, we had a lot of chances to win the game and ended up settling for a point. That seems to have been the trend so far and it was nice to come away from Layer Road with all three points in the bag. It was thoroughly deserved too.”