 Walker was born on Leith Walk in Edinburgh to grazier John William Walker and his wife Elizabeth, "née" Waterston.The family migrated to New South Wales in 1844 and settled on Castlereads Station near Boorowa.In 1849, John Walker sold the property to Hamilton Hume and returned to Scotland.