 According to John Knox, James V of Scotland visited the castle just before his death in 1542, to visit the earl's daughter who was "one of his whores".It was unoccupied in the 17th century, and was used for religious meetings by an outlawed Episcopal congregation.It was later used for agricultural purposes.