Each application made under this act for a marriage license shall be in the usual form and shall be accompanied by a fee of $3.00, $2.00 of which the judge of probate shall keep for services rendered, and $1.00 of which the judge of probate shall forward to the state registrar for deposit in the state general fund. The judge of probate, upon the filing of an application under this act, shall perform the marriage ceremony. If the applicant or either of the parties to the marriage desires to have the marriage ceremony performed by some person competent to perform the marriage ceremony other than the judge of probate, the judge of probate shall issue a written permit to the person designated by the applicant or contracting party directing that person to perform the marriage ceremony. The party so designated, if competent to perform the marriage ceremony under the laws of this state, may perform the marriage ceremony, but a record shall not be made of the marriage, except the record made by the judge of probate under this act. Upon the performance of the marriage ceremony, the party performing it shall return the marriage certificate to the judge of probate, who shall attach the license and certificate to the application. The papers described in this section shall be executed in duplicate, and the person performing the marriage ceremony shall deliver a certificate of the marriage to the parties.