 1557 saw the death of the 4th Lord Hay.His son, also John, abandoned the castle and moved into a new tower house on the site of the present day mansion of Yester House.In 1646 the 8th Lord was created Marquess of Tweeddale.