 Davidson's father was naval captain Alastair Gordon Davidson, of a gentry family of Tillychetly and Dess, formerly of Inchmarlo, all in Aberdeenshire.Captain Davidson's father, Henry Oliver Duncan Davidson (1854-1915) was 4th of Tillychetly; his eldest son Duncan Gordon, 5th of Tillychetly, inherited Dess from a cousin, becoming 7th of that place.James Davidson succeeded his uncle Duncan in 1954, becoming 6th of Tillychetly.