 Among his elder brothers were William, Lord Bagot, the Rev.Walter Bagot of Pype Hayes Hall (who married Anne Swinnerton and, later, Mary Ward), and Richard Bagot (who married a daughter of Viscount Andover).He was educated at Westminster School and at Christ Church, Oxford.