 Ayres was born in San Antonio to Atlee B. Ayres and Olive Moss Ayres.His parents sent him to the private military preparatory San Antonio Academy, and later to the college preparatory Haverford School.Upon graduation from Haverford, he studied architecture with Paul Philippe Cret at the University of Pennsylvania.