 Willard was born on November 1, 1809 in Brattleboro, Vermont.He grew up in Rockingham, Vermont, and attended American School for the Deaf in Hartford, Connecticut.During his schooling, he was a student of the famous Laurent Clerc, who was and is considered the "Father of Deaf Education" in America.