 He grew up in the Crotona Park section of The Bronx, where he played baseball with future Hall of Famer Hank Greenberg.He briefly attended DeWitt Clinton High School, but dropped out after failing to make the varsity baseball team.At age 14, Jaffee took up skating at the Gay Blades of Iceland rink (which later became the Roseland Ballroom).