 Born in Darlington, South Carolina, Johnson took piano lessons as a child, and classical music remained one of his passions.In 1938 he moved to New York City, and the following year toured Europe with the Cotton Club Revue, being expelled from Nazi Germany.Later in 1939 he first recorded for Decca Records with his band, soon afterwards being joined by his sister Ella as vocalist.