 Gilbert married Margaret Sherard (sister of Philip Sherard, 2nd Earl of Harborough and daughter of Bennet Sherard, of Whissendine and Dorothy, daughter of Henry Fairfax, 5th Lord Fairfax of Cameron), who predeceased him.His only child Emma was married on 6 August 1761 to George Edgcumbe, 1st Earl of Mount Edgcumbe, at her father's house at Twickenham, three days before his death.Gilbert's only publications were occasional sermons.