 Lytle recorded more than 30 albums for various jazz labels including Jazzland, Pacific Jazz, Solid State, Milestone and Muse.Throughout his career he performed and recorded with jazz greats including Louis Armstrong, Lionel Hampton, Miles Davis, Nancy Wilson, Bobby Timmons and Roy Ayers.Lytle was such an admirer of the music of Miles Davis that he wrote "Selim" (Miles spelled backwards) in honor of Davis.