 Kelley was of New England stock, his ancestors having come to America from England before 1650.He himself was born in Sparta, Wisconsin.His mother was from a musical family, and herself was skilled in music; she became his first teacher.