Aljofree bundled the ball home from close range after Gillingham failed to clear Jon-Paul McGovern's corner to leave the home side stunned, after they had dominated for long periods.
The early kick-off had given Gillingham a perfect opportunity to put pressure on their relegation rivals, and they made a dream start, taking the lead inside the opening two minutes.
Nicky Southall's swirling free-kick was met with a firm header by Garry Richards at the far post and the ball found its way into the net via Petr Brezovan's left-hand post.
Swindon had a great chance of a swift reply when Lee Peacock beat the Gillingham offside trap but blasted just wide of Derek Stillie's right-hand post.
Brezovan remained the busier keeper however, and suffered a heart-stopping moment on 21 minutes when he rushed out to intercept a pass with his head only to present Simeon Jackson with a chance that the striker squandered.
The game continued to simmer, and Gillingham felt they should have been awarded a penalty minutes before the interval when Gary Mulligan tumbled under Jack Smith's challenge in the area, but the referee, to the dismay of the home fans, chose to award a free-kick on the edge of the box.
The interval failed to break the pattern of play, with the home side continuing to push forward, and Jackson almost doubled their lead when he fired just wide after latching on to Mulligan's cheeky back-heel.
Swindon threw on Billy Paynter in response and began to look a greater threat but it seemed Gillingham would hang on for a vital win.
But when substitute Sean Morrison headed McGovern's corner back across goal, skipper Aljofree was in the perfect position to tap home and leave Gillingham's survival hopes in tatters.