 He then taught classics and mathematics at Eton for the rest of World War I.In later life, when fully retired, he taught mathematics to children in Bridport as a volunteer.His father had started as a Liberal, but Colfox followed his mother's family — which included Joseph Chamberlain (Founder of the National Liberal Federation, Liberal Unionist Association and National Radical Union), his sons Austen and Neville Chamberlain — first to become Liberal or Radical Unionists then Unionists and later Conservative Unionists.