It shall be the duty of the Mayor to appoint the directors of all departments and to make all other appointments provided for by law or ordinance unless otherwise provided herein. All appointees of the Mayor shall serve until removed by him or until their respective successors are appointed.