All permits issued by departments or offices as outlined in Section 1 of this Article that require the payment of a permit or license fee shall be issued by the departments designated, as a permit for the conduct of the specified business, enterprise or activity at the specified location, and such permit shall be delivered to the applicant by the Tax Collector on the payment to the Tax Collector of the fee required by law or ordinance.