 In 1940 Brody met artist Alice Neel, beginning a complicated, sometimes controversial, on and off affair that would end in 1955.They had one son, Hartley Neel, in 1941.Brody remained friends with Neel until her death in 1984.