 They first went to Caen where they were accommodated for some time by Elizabeth Preston, the Marchioness of Ormond.He seems then to have been employed at Charles II's wandering exile court in some ways, whereas his mother went to Paris where she lived in the convent of the Feuillantines, together with her sister Eleanor Butler, Lady Muskerry.James returned with his parents and siblings from France to London in 1660 with the advent of the English Restoration.