 Martin was born in Kent in 1731, and was the grandson of Thomas Fairfax, 5th Lord Fairfax of Cameron (1657–1710) through his mother, Frances Fairfax Martin.Raised in humble surroundings in England, Martin relocated to Virginia in 1751 to assist his uncle, Thomas Fairfax, 6th Lord Fairfax of Cameron (known as Lord Fairfax), in administering the Northern Neck Proprietary, which encompassed up to .Martin resided with his uncle on their frontier Greenway Court estate in present-day Clarke County, Virginia.