 Born in the inner Sydney suburb of Zetland, Ferguson was educated at Granville Convent and Marist Brothers College, Parramatta, both Catholic schools.After leaving school he was variously a farmhand, textile worker, builder's labourer and bricklayer and was an organiser for the Building Workers' Industrial Union.From 1942–46 he served in the Second Australian Imperial Force.