 Born in Darfield, West Riding of Yorkshire, Storey worked at a W. H. Smith in Barnsley, West Riding of Yorkshire at the age of 14 before becoming a miner at Houghton Main Colliery.Meanwhile, he played non-League football for local colliery sides Ardsley Welfare and Grimethorpe Athletic before signing for Huddersfield Town in September 1943.He was released by the club soon after and returned to play non-League football for Wombwell Athletic, where his father worked as a trainer.