 Harvey held his seat at the next two general elections, until his defeat at the 1966 election by Labour's William Robinson.After losing his seat in 1966 he left politics to concentrate on his business career.He was a director and deputy chairman of Burmah Castrol Europe until his retirement.