Whenever the term of office for which any sheriff shall have been elected has expired, or he shall have resigned or removed without the county, or be removed from office, it shall be his duty to deliver over all writs of execution not executed to such person as may have been elected or appointed and qualified to discharge the duties of sheriff; and such new sheriff shall receive all such writs, and proceed to execute the same, in the same manner as if such writs had been originally directed to him; and for any failure or neglect to perform the duties therein imposed upon the former sheriff, such former sheriff and his sureties shall be subject to the like penalties, proceedings and judgments as if he still continued in office.