 On 1 March 1678, he married firstly Margaret Lowther, daughter of Sir John Lowther, 1st Baronet in St James's in London.By 1691, he married secondly Mary Jephson, daughter of William Lewis and widow of William Jephson.Aubrey died at Boarstall in Buckinghamshire after a fall from his horse and was buried at the old family home Llantrithyd in Glamorgan.