In case of damage to any street by reason of the moving of any building or structure or section or portion thereof, the Board shall do such work as may be necessary to restore the street to as good a condition as the same was in prior to such damage and shall charge the cost thereof to the house mover to whom the permit was issued for the moving of such building, or structure, or section, or portion thereof.