 Ferrers' first wife was Elizabeth, the widow of his friend Humphrey Bourchier (d.1540), whom he married by 10 December 1541.On 29 July 1548 he obtained the reversion of her right to the lease of Markyate Priory.His second wife was Jane, the daughter of John Southcote of St Albans, whom he married by licence dated 5 March 1546, and with whom he had a son, Julius.