 Downes moved up to light heavyweight in 1963, winning his first three fights at the weight before facing Willie Pastrano for the world title in Manchester on 30 November 1964.Downes was knocked down twice in the 11th round, while reportedly well ahead on points, and Pastrano retained his title when referee Andrew Smyth controversially waved it off – it was to be Downes' last fight.Downes was famous for a number of quips.