yum install python-pip -y
yum install python-devel -y
yum install gcc gcc-devel -y
yum install libxml2 libxml2-devel -y
yum install libxslt libxslt-devel -y
yum install openssl openssl-devel -y
yum install libffi libffi-devel -y
CFLAGS="-O0" pip install lxml
pip install Scrapy==0.16.0
scrapy --version
-
-
Save clasense4/22007c4cc7ba2b625717 to your computer and use it in GitHub Desktop.
yum install python-pip -y | |
yum install python-devel -y | |
yum install gcc gcc-devel -y | |
yum install libxml2 libxml2-devel -y | |
yum install libxslt libxslt-devel -y | |
yum install openssl openssl-devel -y | |
yum install libffi libffi-devel -y | |
CFLAGS="-O0" pip install lxml | |
pip install Scrapy==0.16.0 | |
scrapy --version |
Everythin installed, But I am getting the follwoing output when the version is checked:
Traceback (most recent call last): File "/usr/local/bin/scrapy", line 3, in <module> from scrapy.cmdline import execute File "/usr/local/lib/python2.7/site-packages/scrapy/cmdline.py", line 7, in <module> from scrapy.crawler import CrawlerProcess File "/usr/local/lib/python2.7/site-packages/scrapy/crawler.py", line 5, in <module> from scrapy.core.engine import ExecutionEngine File "/usr/local/lib/python2.7/site-packages/scrapy/core/engine.py", line 14, in <module> from scrapy.core.downloader import Downloader File "/usr/local/lib/python2.7/site-packages/scrapy/core/downloader/__init__.py", line 16, in <module> from .middleware import DownloaderMiddlewareManager File "/usr/local/lib/python2.7/site-packages/scrapy/core/downloader/middleware.py", line 7, in <module> from scrapy.http import Request, Response File "/usr/local/lib/python2.7/site-packages/scrapy/http/__init__.py", line 11, in <module> from scrapy.http.request.form import FormRequest File "/usr/local/lib/python2.7/site-packages/scrapy/http/request/form.py", line 9, in <module> import lxml.html File "/usr/local/lib/python2.7/site-packages/lxml-3.4.4-py2.7-linux-x86_64.egg/lxml/html/__init__.py", line 42, in <module> from lxml import etree File "lxml.etree.pyx", line 161, in init lxml.etree (src/lxml/lxml.etree.c:198847) TypeError: encode() argument 1 must be string without null bytes, not unicode
Great stuff. I couldn't get around that lxml dependency. Thanks for the save.