Each judge of probate may from time to time appoint a deputy to act in his stead during his temporary absence, and in evidence of such appointment shall issue an order which shall be filed and recorded as herein provided. Each deputy so appointed shall have power, during the temporary absence of the judge of probate, to perform all the duties of his office; and all such acts, judgments, decrees, orders and licenses shall be done and issued in the name of the judge of probate by his deputy and when so done and issued shall have the same force and effect in law as if done and issued by the judge of probate. The judge of probate shall be accountable and responsible for all acts of his deputy within the scope of his duties, and may, at his pleasure, by order, remove any such deputy. All orders appointing or removing such deputy shall be recorded and indexed in the office of the judge of probate in a book to be kept for that purpose, available for public inspection.