Ough was born in Leytonstone, London. His father, Arthur Ough (1863–1946), was an architect, surveyor and civil engineer. At the age of two Ough accompanied his parents to Hong Kong, where his father was employed as an architect for the University of Hong Kong and the Kowloon-Canton Railway, remaining there for four years. He was educated at Highfield School, Liphook, Hampshire and Bootham School in York.