 Patton was born in Fredericksburg, Virginia to Robert Patton (1760–1851), a Scottish immigrant, and Ann Gordon Mercer (1762–1857), daughter of General Hugh Mercer.Patton attended Princeton University and graduated from the medical department at the University of Pennsylvania in 1818.He went on to study law and was admitted to the bar, commencing practice in Fredericksburg.