The city councils, being met in convention upon the day appointed by law or ordinance, shall by joint ballot elect a city clerk, who shall hold his office for the term fixed by law or ordinance, and until another is chosen and qualified, removable, however, at the pleasure of the city councils.