 His father, John Wharton, served as coroner of Chester County.His paternal grandfather, Thomas Wharton, a native of Westmorland, England, came to Pennsylvania around 1683; he served on the Philadelphia Common Council (1713–1718).In 1762, Wharton married Susannah Lloyd, the daughter of Thomas Lloyd and great-granddaughter of Thomas Lloyd, an early governor of Pennsylvania and a colleague of William Penn.