 Willis was born in Oxford, Maryland, attended the district schools there and studied under private tutors.Willis married twice.To Annie Barratt Townsend (1843 - 1885), they had one daughter Elizabeth Townsend Willis (1864 - 1934).