The board shall require one of the clerks to keep a list of the names of all persons challenged, the grounds of the challenge and the determination of the board upon the challenge. Copies of the list shall be kept in the office of the county recorder or other officer in charge of early ballot processing as a public record. Affidavits of challenged voters, decisions of election officials and challenge lists shall be a part of the official returns and shall be delivered to the board of supervisors.