Two penalties inside the opening quarter of an hour set the tone for a disappointing first home Southern Counties East Football League home defeat on Breast Cancer Charity Day at Greatness Park.
Rochester's Tom Carr converted both, before Byron Walker pulled one back before half-time. Jerson Dos Santos hit the post two minutes after the break, before the visitors added two more to claim the points.
Referee Alasdair King - a new name on the Southern Counties East Football League panel this season - pointed to the spot first after nine minutes - penalising Jon Lord's challenge in the box - Carr stepping up to score.
It got worse five minutes later when Paul Gross' cross struck Jack Miles at close range, Mr King awarding a second spot-kick, with Carr again doing the damage from 12-yards.
Mikey Dalton fired over the bar following up after Chris Edwards' shot was blocked. Walker got Town back in the game, after 25 minutes, finishing with a low strike, after being set-up by Kieron McCann.
Rochester goalkeeper Michael Elder then needed treatment after colliding with Walker as the pair challenged for the ball. Town nearly went 3-1 down just before half time, Dan Gonzalez-Bello seeing his effort go inches wide of the far post.
Dos Santos went even closer in the first home attack after the break - seeing his effort bounce of the same woodwork. Greg Benbow's low cross three minutes later then needed only a touch for Town's second.
United's Alex Kendall almost punished a home error soon afterwards, before Ricki Holmes got the visitors' third as Town struggled to clear a free-kick.
Dalton had a shot saved, but Town were again caught out at the back with 13 minutes left allowing Kendall to run unchallenged from around the halfway line, before shooting past Brown.
Town appealed in vain for a penalty of their own eight minutes from time when McCann went down in the box, before new signing Billy Bennett on from the bench, fired straight at Elder with five minutes left.