 He studied the violin in Harlem with composer and violinist John Thomas Douglass, the son of a freed slave, and later in Berlin with Karel Halíř.Mannes was a violinist in the New York Symphony Orchestra from 1891 and its concertmaster from 1898 to 1912.In 1912 he helped found the Colored Music Settlement School and in 1916, with his wife Clara Mannes (the daughter of Leopold Damrosch and sister of Walter Damrosch), the Mannes Music School, both in New York City.