 He was the son of Simon Luttrell, 1st Earl of Carhampton and brother-in-law of Prince Henry, Duke of Cumberland and Strathearn.Henry Luttrell was the scion of an Anglo-Irish landed family, descendants of Sir Geoffrey de Luterel, who established Luttrellstown Castle, County Dublin, in the early 13th century.Educated at Westminster School and Christ Church, Oxford, Luttrell was commissioned into the 48th Regiment of Foot in 1757.