 Thomas Fane was the second son of Henry Fane of Brympton d'Evercy in Somerset and Anne Scrope, sister and coheir of John Scrope.Anne and John were the grandchildren of Colonel Adrian Scrope, a regicide of Charles I. Thomas Fane inherited John Scrope's fortune and mansion in Bristol, and Colonel Adrian Scrope's property in Oxfordshire and Buckinghamshire, which included Wormsley Park.In 1757 he succeeded his unmarried elder brother Francis to their father's Brympton estate and in 1762 inherited the title of Earl of Westmoreland from John Fane, 7th Earl of Westmorland, his father's childless second-cousin.