 In 1811, Chastain moved to Milledgeville, Georgia.Elected to represent Georgia's 5th congressional district in the United States House of Representatives as a Unionist during the 32nd United States Congress, Chastain won reelection as a Democrat to an additional term in the 33rd Congress and served in Congress from March 4, 1851 to March 3, 1855.Chastain was delegate to the Georgia secession convention in Milledgeville in 1861 which passed the Ordinance of Secession.