 While attending Emory University, Bush was the lead singer and guitarist in the rock band Storyteller, with fellow Emory students Chris "Tex" Nolter (bass, guitar) and Jon Slatkin (drums).Storyteller covered the Replacement's "I Can't Wait," Rod Stewart's "Maggie May," The English Beat's "Mirror in the Bathroom," and Jane's Addiction's "Jane Says," among other songs, and they performed at numerous bars, campus events and parties in Atlanta.Bush started his career in the early 1990s forming folk rock duo Billy Pilgrim along with Andrew Hyra.