 Like his younger brother Richard, Fitzpatrick was a friend of Charles James Fox, whilst their half-sister Mary FitzPatrick had married Charles James Fox's older brother Stephen Fox, 2nd Baron Holland.As such Fitzpatrick became step-father to Henry Vassall-Fox, 3rd Baron Holland on the death of his parents.He was also a noted friend to Horace Walpole; a large volume of correspondence between Walpole and Fitzpatrick's wife, the Countess of Upper Ossory survives.