 Holley was born in Pensacola, Florida, and graduated from Tate High School in 1973.While a student there he was a basketball standout and active in student government.He graduated in 1975 from Faulkner State Junior College in Bay Minette, Alabama, with an Associate of Arts degree in general studies, and from Alabama State University in Montgomery, Alabama in 1977 with a Bachelor of Arts in management.