The health officers may withdraw the permission by notice in writing whenever in their opinion such use or occupancy of a building or place becomes a nuisance. In such case they shall order the abatement of the nuisance and the discontinuance of the use or occupancy within a time limited. If the owner or occupant neglects to comply with the order such owner or occupant shall be guilty of a violation for each day the owner or occupant so neglects or refuses after the expiration of the time limited, for the use of the town.