 As Congress began discussions over the construction of a new capital, Dolley and James moved into The Octagon House.Dolley would establish the Washington City Female Orphan Asylum.On April 6, 1817, a month after his retirement from the presidency, Dolley and James Madison returned to the Montpelier plantation in Orange County, Virginia.