A councillor says his town is being “left behind” after a centre for Britain’s fastest-growing sport was refused.