 Brown was born of Scottish parents in Izatnagar, Uttar Pradesh, India, where his father was a railway engineer.In the early 1930s, the family relocated to Edinburgh, Scotland, where Brown was later educated at the Royal High School.He taught himself clarinet from the age of twelve.