 His grandfather John G. Jackson had served in the U.S. House of Representatives as well as a U.S. District Judge, and earlier as a Brigadier General in the Virginia militia.His father was also (Virginia militia) General John Jay Jackson.His brothers became Federal Judge John Jay Jackson, Jr. and Circuit Judge and West Virginia Governor Jacob B. Jackson.