 He was a descendant of Nathaniel Colburn, a selectman and signer of the Dedham Covenant.Colburn attended Harvard Law School from 1848 to 1849 studied law in the office of Ira Cleveland, and was admitted to the Massachusetts Bar on May 3, 1850.In 1856 Colburn was elected to serve in the Massachusetts House of Representatives.