 Ashley Castle itself became a residence of the Bishops of Winchester until the middle of the 15th century.It was eventually abandoned as a residence in about the 17th century.All retrievable stone was removed and re-used elsewhere.