Nothing in this subdivision shall be construed to prevent ordinary maintenance or repair of any structure or place within any historic district nor to prevent the construction, alteration, repair, moving or demolition of any structure under a permit issued by the building inspector or other duly delegated authority prior to the establishment of any historic district.