 Berg was born the eldest son of a family with three siblings, including his younger brother Gustav "Astaf" Gotthard Karl von Berg, the owner of the .His other brother Alexander was a diplomat and consul in Naples and London.Count von Berg married late in his life.