 In 1769, the Paynes had returned to Virginia and young Dolley grew up at her parents' plantation in rural eastern Virginia and became deeply attached to her mother's family.Eventually, she had three sisters (Lucy, Anna, and Mary) and four brothers (Walter, William Temple, Isaac, and John).In 1783, following the American Revolutionary War, John Payne emancipated his slaves, as did numerous slaveholders in the Upper South.