 Elwes was married in Kensington on 11 May 1889 to Lady Winefride Mary Elizabeth Feilding (died 24 February 1959), National President of the Catholic Women's League, daughter of Rudolph William Basil Feilding, 8th Earl of Denbigh and Mary Berkeley.They lived at The Manor House, Brigg, Lincolnshire, and at Billing Hall, Northamptonshire, and had six sons and two daughters together