 He was raised by his widowed mother, who remarried Sir Charles Shelley, 2nd Baronet; Shelley died in 1681.George came from an openly recusant family on both sides.During the Popish Plot, given the long imprisonment of the "Five Catholic Lords" on fabricated charges of treason, and the fact that George was closely related to the Vaux and Brooksby families, who had been deeply implicated in the Gunpowder Plot,  his mother understandably became concerned about his safety, and in 1678 she took him to live abroad for a time.