It is hereby made the duty of the tax collector and public administrator of every county of this state to deposit any and all sums of money coming into his hands by virtue of his office in a depository designated by the supervising board under the provisions of this chapter, and any such moneys so deposited shall be a part of the public moneys as defined in this chapter, but shall remain subject to withdrawal by such tax collector or public administrator so depositing the same. Such sums while so on deposit in said depository, shall be held in separate accounts respectively designated as "Tax Collector’s Account" and "Public Administrator’s Account."