 He was the son of Charles Cecil Byers (1888–1957), a Lloyd's underwriter, who was Liberal candidate for Westbury at the 1935 general election.He moved with the family to Potters Bar and was educated at Westminster School, followed by Christ Church, Oxford, where he won a Blue for athletics.At Oxford he was president of the Union of Liberal Students and president of the University Liberal Club.