 Baillie-Hamilton was born in Greenwich, Kent, the son of Admiral William Alexander Baillie-Hamilton (1803–1881) and Lady Harriet Hamilton (1812–1884).His father was Second Secretary to the Admiralty from 1845 to 1855; his mother was the daughter of James Hamilton, Viscount Hamilton M.P.