Owners of a Fenland restaurant face having to remove signage after councillors decided it was out of place in the town's conservation area.