 Fox was born in New York City, the son of Mollie and Walter Fox, a Jewish immigrant from Szydlowiec, Poland.While still a student at the High School of Music & Art, Fox studied jazz piano with Lennie Tristano.He then continued his musical education with Nadia Boulanger, first at Fontainebleau and then privately in Paris.