 He was admitted to the bar about 1812 and began practice in Penn Yan.Oliver was First Judge of the Yates County Court from 1823 to 1828, and from 1838 to 1844.He was a member of the New York State Senate (7th D.) from 1827 to 1830, sitting in the 50th, 51st, 52nd and 53rd New York State Legislatures.