 His ancestors included Captain William Locker, Edward Hawke Locker, Benjamin Stillingfleet and Jonathan Boucher.Oliver was educated at Cheam School, Eton and Trinity College, Cambridge where he gained an Honours Tripos Degree in History and Modern Languages.While at Cambridge, he was co-editor of "Granta" with Edwin Montagu and President of the Amateur Dramatic Club.