 In 1824 the castle was sold to James Balfour of Whittingehame, father of James Maitland Balfour, and grandfather of Arthur Balfour, who served as British Prime Minister from 1902-1905.He was unable to arrest the decay which was advancing, and in the mid nineteenth century the roofs were removed to avoid paying tax on the property.Much vandalism occurred in the 1960s, and it was not until 1971 that restoration of the castle, then owned by David Maxwell, began.