 His wife Fannie is an attorney, and they have four children

Gilliam attended Brewer High School in Greenwood, South Carolina, then played college football at South Carolina State University, from which he graduated with a degree in biology.While in college he also lettered in track and was clocked in the 100 yard dash in 9.5 seconds, four tenths of a second off the world record at the time.He was drafted in the second round by the expansion New Orleans Saints.