 Graham-Brown was born at Thetford in Norfolk, the son of Lewis Graham-Brown and his wife Elizabeth Blaxland.He attended Sevenoaks School in Kent, playing in the First XI for several years and as captain in 1970, when he scored 403 runs at an average of 40.30 and took 45 wickets at 8.60.He read English Literature at the University of Kent, obtaining a first-class honours degree, and then went on to Bristol University to complete a master's degree in Philosophy.