 By the early 12th century, Grosmont was owned by the Anglo-Norman nobleman Pain fitzJohn.In 1135, a major Welsh revolt took place, and in response King Stephen restructured the landholdings along this section of the Marches, bringing Grosmont Castle and its sister fortifications of Skenfrith and White Castle back under the control of the Crown to form a lordship known as the "Three Castles".Conflict with the Welsh continued, and following a period of détente under Henry II in the 1160s, the de Braose and de Mortimer Marcher families attacked their Welsh rivals during the 1170s, leading to a Welsh assault on nearby Abergavenny Castle in 1182.