 The original house was probably built by Edward Seymour, 1st Earl of Hertford in about 1575, when it was known as Totnam Lodge.In 1675 the estate passed to Lady Elizabeth Seymour, who married Thomas Bruce, 2nd Earl of Ailesbury, passing the house to the Bruce family.In 1721, Seymour's son and heir, Charles Bruce, 3rd Earl of Ailesbury, rebuilt Totnam Lodge to the design of his brother-in-law the pioneering Palladian architect Lord Burlington, and parts of the grounds, including the kitchen garden, were laid out by Capability Brown from 1764 to c 1770.