No person in any manner shall interfere with the affixing of any number to any building or remove, destroy, alter or deface any number of any building assigned and put up thereon. No person, after having received the notice provided for in Section 913.06, shall retain an improper number or substitute any other number on his building other than the numbers assigned in the manner provided for in this chapter. (1964 Code §519.81)