 A small settlement, called Nova Scotia or Novia Scotia, was built on the site of the castle's harbour, possibly by Scottish immigrants.Several engravings were published of the castle in the 18th century, including a somewhat inaccurate depiction by Samuel and Nathaniel Buck in 1720, and by Francis Grose and William Hutchinson in 1773 and 1776 respectively.Mary's descendants, the Earls of Tankerville, owned the property until the heavily indebted Charles Bennet, the 6th earl, sold it for £155,000 in 1869 to the trustees of the estate of the late Samuel Eyres.