Take the 2-minute tour ×  I get this warning when running scrapy
:   [scrapy] WARNING: Disabled HttpCacheMiddleware: Unable to
find scrapy.cfg file to infer project data dir share|improve
this question add comment  1 Answer  Where is your scrapy.cfg
file? Are you running from the command line? If so, you should
be running from the same directory as scrapy.cfg.  What does
your project directory look like. It shoud look something like:
├── dirbot │   ├── __init__.py │   ├── items.py │   ├──
pipelines.py │   ├── settings.py │   └── spiders │       ├──
dmoz.py │       └── __init__.py ├── README.rst └── scrapy.cfg
If you are using cron, try not to (meaning don't) use the root
user. You can add the PATH info right into the crontab: How to
get CRON to call in the correct paths  share|improve this answer
Project directory is the same as the one you described, the
spider is called from a scheduler.php file in the same dir as
scrapy.cfg –  Vanddel Apr 2 '13 at 12:20 How are you calling
scrapy? Popen()? who is callin