 Hylda, his wife died in 1935 and Calverley remarried in 1943.His second wife was Hannah Baxter (née Taylor) widow of Fane Fleming Baxter.In 1953 Calverley sold Close House and High Close House and moved to Shawdon Hall where he continued his racehorse interests.