 He had a sister Lucy, who later married Charles Lewis Mathews.His cousin, Jubal Early, became the grandfather of Jubal Anderson Early (1816–1894), later a prominent Confederate general.Peter Early graduated from the Lexington Academy (current-day Washington and Lee University).