Lands may be added to or withdrawn from such district in the manner provided for its formation, but no withdrawal may be allowed if the result thereof would be to reduce the remaining territory included in the district below the minimum area provided in section 23-2904.