Lambert scored first for Bristol Rovers from the edge of the area on 26 minutes making it 1-0.
On 39 minutes Swindon were once again level with a close range goal from Kanyuka.
Barry Corr headed Swindon into the lead from close range on 53 minutes.
Bristol Rovers then came close a number of times, and hit the post, but Swindon looked to have held on until 3 minutes and 59 seconds into the 4 minutes of time added on when Lines scored.
He was then sent off for a second yellow card, but it didn't matter as within seconds of the kick off the final whistle blew and the game ended 2-2.