 Vauguyon was descended from an old aristocratic family.His father, Antoine de Quélen de Stuer de Caussade (1706–1772) was the duc de La Vauguyon (1759), prince de Carency, pair de France, Menin to the Dauphin, lieutenant général of the royal armies, governor, first gentleman of the chamber and grand master of the garde-robe to the duke of Burgundy, to the Dauphin and to the counts of Provence and Artois, knight of the ordre du Saint-Esprit and the order of Saint-Louis.His mother, Marie Françoise de Béthune (1712–1799) was the daughter of the duke of Charost, himself governor to the king's person to Louis XV.