 Simon was born to a Jewish family in Williamsburg, Brooklyn and grew up in the Bronx, the son of Max and Mae Simon.His father was a tailor who had emigrated from Central Europe.Simon graduated from the Bronx High School of Science and earned a degree in accounting from the City College of New York in 1949.