 John Fitzpatrick died in February 1818, aged 72, when his titles became extinct.His natural son, John, succeeded to parts of his estates and was given the title of Baron Castletown in 1869; however on his death his estate was inherited by his step-son Henry Vassall-Fox, 3rd Baron Holland - who also adopted Ampthill Park as his seat.Fermyn Woods Hall, inherited from his father, passed to his daughters.