Rumer was born in Moscow into a Jewish merchant family. His elder brothers Osip and Isidor were well-known translators and philosophers. After graduating from non-classical secondary school in 1917, in 1918 Rumer entered the Physics and Mathematics Faculty of Moscow State University and graduated in 1924. In 1927 he married Lyudmila Zalkind, his girlfriend of nine years, and emigrated with her to Oldenburg, Germany, where he enrolled to study construction engineering. The same year he abandoned this boring for him topic in favor of theoretical physics, and moved to Göttingen. During an internship at the University of Göttingen he worked as an assistant of Max Born. He collaborated with Walter Heitler and published several theoretical works on the structure of molecules.