 From 1847, he served as chaplain to Queen Victoria.Pelham retired as bishop in 1893 and spent the next year in Thorpe St Andrew.On 6 November 1845, Pelham married Henrietta Tatton, second daughter of Thomas William Tatton, and had by her four sons and a daughter.