White was born in Tuskegee, Alabama. His mother, Susie Mae Fort White, worked at John Andrew Hospital and his father, Herman Brenner White Senior, served in the military. He studied at the Tuskegee Institute High School, where he became interested in nuclear engineering. He was raised in a segregated community. He saw his role in the civil rights movement as being an exceptional student who could prove that black people deserved equal access to education. He studied nuclear physics at Earlham College, before joining Michigan State University, earning a bachelor's degree in physics in 1970. White was awarded an Alfred P. Sloan Foundation Fellowship to study at CERN, and also worked as a research associate at Argonne National Laboratory. He completed his master's degree in 1974. He joined Fermilab in 1974, where he was the first African-American physicist to be appointed.