Nothing in this ordinance shall be deemed to require any change in the plans, construction, or design use of any building upon which actual construction was lawfully begun prior to the adoption of this ordinance, and upon which building actual construction has been diligently carried on, and provided further, that the building shall be completed within 2 years from the date of passage of this ordinance.