 Tolliver was born one of 12 children to sharecroppers Ike and Laney Tolliver in the Pike Road community, near Montgomery, Alabama.His exact year of birth is unknown, though it is known he was born on the Fourth of July.He attended school only until the third grade due to a self-described lack of interest in education.