The selectmen may remove from office any collector of taxes, town clerk, or any treasurer, who, in their judgment, has become insane or otherwise incapacitated to discharge the duties of the office. They may proceed without notice in any case arising under this section.