In any Residence District, any building or structure, all or substantially all of which is designed or intended for a use permitted only in a Business or Industrial District, and any building or structure which is located in a Business District and which is designed or intended for a use permitted only in an Industrial District, shall be removed or shall be altered, remodeled or converted for a permitted use within six (6) months after the amortization period of such building or structure.