 Over time, Lyttelton earned the respect of the Queen and the Prince Consort and, in April 1843, she was appointed governess to the royal children, who continued to call her "Laddle", even once they were grown.In 1850, Lavinia died following the birth of her fourth child, and Lady Lyttelton requested the Queen's permission to resign so that she and her remaining daughter, Caroline, could return to Hagley to care for the motherless children.This was very reluctantly granted, Victoria accepting that her reasons were unanswerable.