 It passed to the Milbournes and in 1846 onto William Lowther, the 2nd Earl of Lonsdale.It was reported in 1892 that the castle had fallen into disrepair.In the early 19th century the castle was converted into a mansion, faced with ashlar cut stone in a classical style, with an adjacent wing for offices.