Currently, according to the law, a City Rabbi officially appointed continues to serve until the age of 75, with an almost automatic extension given for an additional five years, until the age of 80. Until the rav reaches 75 years of age, there is no way to get rid of him - even if he just lays in bed all day or is corrupt or does nothing productive in his position. It is an appointment until he reaches 75, and there is no way around it.

The Ministry of Religious Affairs has pushed a law proposal that will allow a rav to be removed from his post if he is not fulfilling his duties and responsibilities. The authority to do so will be given to the Director of the Chief Rabbinate who will appoint a committee to investigate any such claims, in consultation with the Chief Rabbi or in consultation with the Rabbinic head of the region in which he serves.
source: Kipa

I think it is good to have such an option available for extreme cases (as I am sure most city rabbis work diligently in their positions). My one concern would be that such a law not be used as a way of removing rabbis someone in the Rabbanut disagrees with or disapproves of.