 Rhys-Williams was the son of Judge Gwilym Williams and Emma Eleanor Williams JP.His wife Juliet Rhys-Williams (née Glyn) was a writer and prominent Liberal politician who, like her husband, later joined the Conservative Party via the Liberal Nationals.They met in 1919 when Juliet Glyn began working for Rhys-Williams as his private secretary during his period of office as parliamentary secretary at the Ministry of Transport.