 Rupp was born in Port Fairy, Victoria to Charles Ludwig Hermann Rupp, a Prussian-born Anglican clergyman and Marie Ann Catherine Rupp, a Tasmanian who died two weeks after the birth of Montague.Montague Rupp was educated at Geelong Grammar School as a boarder, where an uncle John Bracebridge Wilson, the naturalist, was headmaster.Charles's parents died on the voyage to Australia or shortly before, and the boy was raised by William Frederic Augustus Rucker (1807 - 1882), another Prussian émigré.