 He was baptised the day of his birth at Harting.His family seat was Chillingham Castle in Northumberland, which he inherited on his father's death in 1675.In 1682 Grey achieved notoriety for being found guilty of seducing his wife's sister, Lady Henrietta Berkeley for which he was arrested, tried and ultimately freed.