Section 96. All ballots transmitted under any provision of sections eighty-six to one hundred and three, inclusive, shall be subject to challenge when and as cast for non-compliance with any provision of sections eighty-six to one hundred and three, inclusive, or for any other reason allowed by law, including that the voter was not unable by reason of physical disability to cast his vote in person at the polling place on the day of the election, and if challenged shall be disposed of in accordance with section eighty-five, except that so much of said section as involves the administering of an oath shall not apply thereto, and the writing of the name and address of the voter on the ballot shall be performed by the officer charged with depositing the ballot in the ballot box.