 Gilbert was born in Milwaukee, Wisconsin to Samuel Gilbert and Bertha Shevinsky, both born in Russia.At the age of nine, he began studying the violin in Shorewood in Milwaukee County.By the time he was 15, he had formed his own dance band.