The owner, agent or person in charge of any dwelling structure shall have the right to appeal from any notice issued by the Building and Zoning Inspector, within the time specified in such notice for compliance therewith, and to appear before the Board of Zoning Appeals at a specified time and place to show cause why he should not comply with such notice. Failure to file an appeal within the time prescribed shall constitute a waiver of the right to appeal. Filing of an appeal from any notice of the Inspector shall suspend action on enforcement of such notice until the appeal is acted upon by the Board.