 Walker was loaned out to Northampton Town and Oxford United in the 2003–04 season, before joining Bristol Rovers in the summer of 2004.A natural goalscorer, he formed a formidable partnership with Junior Agogo before the Ghanaian was sold to Nottingham Forest.Walker scored two goals in the 2007 League Two Play-off Final against Shrewsbury at Wembley, as the Gasheads secured a 3–1 victory to send them up to League One.