 Pyne was born on May 5, 1857 in Manhattan, New York City, the son of Percy Rivington Pyne I (1820-1895) and Albertina Shelton (née Taylor) Pyne (1833–1900).His maternal grandfather was Moses Taylor, founder of the First National City Bank of New York and a stockholder in the Delaware, Lackawanna and Western Railroad.His older brother, Moses Taylor Pyne, inherited much of the family fortune and was a major benefactor of Princeton University.