 He had three other brothers, at least one of whom, Yuan Baoyue (), was older, and born of Yang Aofei.Yuan Yu favored Yang Aofei but not his wife, Princess Yu, a sister to Emperor Xuanwu's wife Empress Yu.Consequently, Empress Yu once summoned Yang Aofei to the palace, beat her severely, and then forced her to become a Buddhist nun.