 Terrill was born in Covington, Virginia, and grew up at Warm Springs, in Bath County, Virginia.His father, Colonel William Henry Terrill, was a Virginia lawyer and member of the Virginia Legislature prior to the war.He became a cadet at U.S. Military Academy in 1849 and graduated in 1853.