peregrinogris/scrapy.md

## scrapy.md

      
    Raw
  

              scrapy.md
            
          
    Installing Scrapy

sudo apt-get install libxml2-dev libxslt-dev
pip install scrapy
pip install python-dateutil

Running Scrapy

When running scrapy, this bash script is handy:
#!/bin/bash
export DB_PATH=/vagrant/server/db
export PYTHONPATH=/vagrant/server
scrapy crawl ${1:-zvents}

save it as 'run' inside the crawlers and run it as ./run  Here, bear in mind that:

DB_PATH: Absolute Path to the database directory
PYTHONPATH: Absolute Path to the project