 Drummond was born in London, the son of a Scottish master mariner in the British India line and an Australian "lieder" singer.He spent much of his childhood in Bournemouth, spending hours in the public library absorbing all he could on creative arts, and also attending concerts by the Bournemouth Symphony Orchestra.He was educated at Canford School and, after his National Service in the Navy (where he studied Russian), read History at Trinity College, Cambridge from 1955 to 1958.