Councillors leaving the Liberal Democrats