 In the early 1950s, while living in Harlem, New York City, Brown married Rhoda Palestine, whose family was Russian-Jewish.Through this relationship Brown later converted to Reform Judaism.They married at a time when interracial relationships and marriages were considered by many as taboo, and had one son, Drew Brown III (born January 20, 1955, in Harlem).