 Fletcher was born in Cavendish, Vermont on December 11, 1839, the son of Ryland Fletcher and Mary (May) Fletcher.Fletcher was the son and grandson of prominent Vermont politicians; his father was both the Lieutenant Governor (1854–1856) and the Governor (1856–1858) of Vermont.His grandfather, Asaph Fletcher, was a member of the convention which applied to Congress for the admission of Vermont into the Union, and served for several sessions in the Vermont Legislature, was a county judge and presidential elector.