 Walter's son Roger de Lacy began the construction of Ludlow Castle on the crest of the hill about 1075, forming what is now the inner bailey.Between about 1090 and 1120, the Chapel of St. Mary Magdalene was built inside the walls, and by 1130 the Great Tower was added to form the gatehouse.About 1170 the larger outer bailey was added to the castle.