All department heads and board and commission members shall enter upon the duties of their several offices upon taking and filing the oath of office and filing the requisite surety, if any be required of them. All appointments shall expire on their expiration, unless sooner removed.