About two dozen homes were evacuated Monday on Herring Road near U.S. Highway 13 South in Goldsboro after the discovery of two cases of dynamite in a storage building belonging to feed wholesaler Herring Milling Co.