 On 10 January 1859 he married Elizabeth King, the daughter of John James King, granddaughter of the third Earl of Egremont, and the widow of George Frederick Dawson.They settled in Hillhouse, Cawthorne, and had one daughter, Mary, in 1860.That same year, Spencer-Stanhope’s house Sandroyd (now part of Reed's School), near Cobham in Surrey, was commissioned from the architect Philip Webb.