Ellison had come under fire for his history of making racially inflammatory statements, as well as his past ties to radical figures like notorious anti-Semite Louis Farrakhan, who heads up the black nationalist group Nation of Islam.