 In 1859, he met, and in 1860, he married Sallie Ann Johnson, with whom he had six children (Mattie, Willie, Ellen, Elizabeth, Thomas, and Jefferson Davis) before her death in 1872.In 1861, Hardy raised Company H of the 16th Mississippi Infantry Regiment, of which he was elected Captain.He served with this unit until October 1862, when illness forced his resignation.