 Fairfax was born in 1724 on the island of New Providence in the Bahamas.He was the son of Sarah (née Walker), and her husband Sir William Fairfax, a British colonel who had served as an English Customs agent in Barbados, as well as a justice and Governor of the Bahamas.At his son's birth, William was working as the Customs Collector in Marblehead.