 Born Henry Gough, he had assumed the additional surname of Calthorpe upon inheriting the Elvetham and Norfolk estates of his maternal uncle, Sir Henry Calthorpe, in 1788.The Baronetcy, of Edgbaston in the County of Warwick, had been created in the Baronetage of Great Britain on 6 April 1728 for Lord Calthorpe's father Henry Gough, who represented Totnes and Bramber in the House of Commons.He was the husband of Barbara, daughter of Reynolds Calthorpe.