 He was the second child and son of William Marvin Williams and Lula Taylor Williams.He had an older brother, Manning Holland Williams, and a younger sister, Lenora Ellen Williams.The family farm where Williams grew up had been granted to his great-great-grandfather in recognition of his service during the American Revolutionary War.