 His time in the conseil d'État ended in February 1797 when, due to courtesans' intrigues, he fell into disgrace with the pretender, who his father had nicknamed "the disingenous".The count of Saint-Priest replaced him.He then stayed for a time in Hamburg before returning to Spain, where he stayed until 1805.