All buildings or portions thereof within the town or in the buffer zone which are determined, after an inspection by the Building Inspector, to be unsafe, as defined in § 154.03, are hereby declared a public nuisance and shall be abated by repair, rehabilitation, demolition or removal in accordance with the procedures specified in this chapter.