 In 1877, Ellis, by then an artist with maturing skills, returned to Rochester where he became one of the founders of the Rochester Art Club.In 1879 while simultaneously functioning as an artist, art teacher and active club member, he and his brother Charles established the architectural firm of H. and C. S. Ellis.Charles was adept at soliciting business, some of it through family connections, while Harvey did the designing.