 Sherman was the eldest of thirteen children born to Charles Robert Sherman and his wife, Mary (Hoyt) Sherman.His family emigrated from England to Massachusetts in 1634.His great-grandfather and grandfather both served on the state courts of Connecticut.