Each inspector of buildings, fire marshal or other person authorized to issue building permits shall enforce the provisions of this part and report all violations thereof to the proper prosecuting officer. When no other officer is authorized to issue building permits, the director of health of each town shall be charged with the duties assigned by this part to building inspectors.