 The castle passed to Lady Elizabeth Chaworth, and on her death in 1501 the Crown gave it to the Harts of Lullingstone Castle.The Harts-Dykes branch of the family later acquired the lordship of the castle and during the 18th century they used the ruined site for their hunting kennels and stables.In 1835, the castle ceased to be used as kennels and the architect Edward Cresy was employed to remove the more recent modifications.