 He had a long affair with the commoner Mary Milner, the beautiful daughter of a gardener; according to some versions (notably offered by Augustus Hare) he went through a false ceremony of marriage with her.They had one son