Section 33. The board of health of a city shall, and the board of health or the selectmen acting as such board of a town may, appoint one or more inspectors of milk and one or more collectors of samples of milk, who shall have the powers, and perform the duties, conferred and imposed by law upon such inspectors and collectors, who shall be under the control of, and whose compensation shall be fixed by, the board appointing him or them; but no person whose business, in whole or in part, is the buying or selling of milk, or who is an officer, agent or employee of any person engaged in the sale thereof, shall be appointed as such inspector. Each person so appointed shall be sworn before entering upon his official duties and each such inspector shall publish a notice of his appointment for two successive weeks in a newspaper published in his town, if any; otherwise he shall post such notice in two or more public places therein. Such inspectors may be removed in cities in the manner provided in sections forty-one to forty-five, inclusive, of chapter thirty-one, and in towns at any time, by the appointing board.