 A fragment of the castle remains and is a Grade II listed building.It was named after Roger d'Oilly who was granted it by William the Conqueror and whose brother built Oxford Castle.It is thought that the castle was built around 1129 and it was demolished soon after 1175.