Within the districts established by this chapter there exist:
1. lots,
2. structures,
3. uses of land and structures, and
4. characteristics of use,
which were lawful before these regulations were passed or amended, but which are prohibited, regulated, or restricted under the terms of this chapter or future amendments. It is the intent of this chapter to permit these nonconformities to continue until they are removed, but not to encourage their survival. Further, nonconformities shall not be enlarged upon, expanded or extended, except in conformity with the following rules. Nonconformities shall not be used as grounds for adding other structures or uses prohibited elsewhere in the same district. To avoid undue hardship, nothing in this chapter shall be deemed to require a change in the plans, construction or designated use of any building on which actual construction was lawfully begun prior to the effective date of adoption or amendment of the Zoning Ordinance and upon which actual building construction has been carried on diligently. Actual construction is hereby defined to include the placing of construction materials in permanent position and fastened in a permanent manner. Where excavation or demolition or removal of an existing building has been substantially begun preparatory to rebuilding, such excavation or demolition or removal shall be deemed to be actual construction, provided that work shall be carried on diligently.