Whenever any municipality conducts a referendum on a day other than a state or local election, the polls shall be open between twelve noon and eight p.m., but any municipality may, any provision of any special act to the contrary notwithstanding, by vote of its legislative body provide that the polls at any such referendum shall open at an earlier hour but not earlier than six a.m.