Dyson married his first wife, the Swiss mathematician Verena Huber, on 11 August 1950. They had two children, Esther and George, before divorcing in 1958. In November 1958 he married Imme Jung (born 1936) and they had four more children: Dorothy, Mia, Rebecca, and Emily Dyson.