 Prior to joining Peterborough, Rayner, a native of Cornsay, County Durham, had begun his football career as a striker at Grimsby Town, and later had short spells at Bury, Hartlepool United, Barrow and non-league Grantham Town before joining Peterborough in 1958.He spent five seasons at Peterborough before rejoining Grantham as player-coach in 1963.Moving back to the striker position, Rayner scored an incredible 71 goals in 45 matches for Grantham in 1963–64, leading the Gingerbreads to their first Midland League title.