 Although it is generally accepted that he built the keep, it is thought that some form of fortification already existed.Some form of wooden fortress may have existed on the site before the Norman conquest.A reference to the ""castellatu Rogerii pictaviensis"" in the Domesday Book entry for nearby Barnoldswick, has been used to argue that it was first built before 1086 by Roger the Poitevin.