Richardson was born to Issac E. and Susan M. B. Richardson about 1854 in Hartford, Maine, where he grew up. He attended the nearby Colby College in Waterville, Maine where he graduated in 1883; whereupon he took the job of organizing the newly created Madison Normal School in South Dakota. He was president of the school from 1883 to 1887. In 1884 he married Edith May Adams. Richardson was succeeded as president by William F. Gorrie of Watertown, South Dakota, who assumed office in September 1887.