 His uncles included Sir Thomas Temple, 1st Baronet, of Stowe and William Fiennes, 1st Viscount Saye and Sele.His sister, Susan Temple, Lady Lister, was the mother of Martin Lister and the grandmother of Sarah Churchill, Duchess of Marlborough.In 1607, following his mother's death, he moved to Longhouse Place (now known as Chadwell Place) in Chadwell St Mary, Essex.