 Lee was born at Stratford Hall Plantation, Westmoreland County, Virginia to Thomas Lee and his wife Hannah Harrison Ludwell.He was the couple's third son and the second one to survive into adulthood.Much about Lee's early life is unknown, but it is assumed that he studied in England along with his brothers.