A building, structure or portion thereof, all or substantially all of which is designed or intended for a use which is not permitted in the district in which it is located and which is vacant on the effective date hereof or thereafter becomes vacant or is not used for a continuous period of one hundred eighty (180) days, shall not, thereafter, be occupied or used except by a use which conforms to the regulations of the district in which it is located. (1965 Code title 6 § 501.8)