Skip to content

Instantly share code, notes, and snippets.

@peketamin
Last active August 14, 2017 00:38
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save peketamin/177db2400733736e6775dd1ceb837739 to your computer and use it in GitHub Desktop.
Save peketamin/177db2400733736e6775dd1ceb837739 to your computer and use it in GitHub Desktop.
Splash - A javascript rendering service を CentOS に入れた時のログ

Amazon 2015.09 に Splash 2.3 を入れる

https://github.com/scrapinghub/splash/blob/master/dockerfiles/splash/provision.sh を見て、それっぽい感じに再現できるように頑張る。

python など

yum install -y \
python34 \
python34-devel \
python34-setuptools \
lua-devel \
libjpeg-turbo-devel  \
re2-devel \
libffi-devel \
xorg-x11-server-Xvfb

venv を使う。pip を入れる (--without-pip なしで pip も入れるとエラーが出るので)。

pyvenv3.4 venv --without-pip
source venv/bin/activate

curl https://bootstrap.pypa.io/get-pip.py | python
pip install zope.interface

yum

rpm -e --nodeps libpng
yum install -y http://mirror.centos.org/centos/7/os/x86_64/Packages/atk-2.14.0-1.el7.x86_64.rpm
yum install -y http://mirror.centos.org/centos/7/os/x86_64/Packages/hicolor-icon-theme-0.12-7.el7.noarch.rpm
yum install -y http://mirror.centos.org/centos/7/os/x86_64/Packages/glib2-2.46.2-4.el7.x86_64.rpm
yum install -y http://mirror.centos.org/centos/7/os/x86_64/Packages/libpng12-1.2.50-10.el7.x86_64.rpm
yum install -y http://mirror.centos.org/centos/7/os/x86_64/Packages/libpng-1.5.13-7.el7_2.x86_64.rpm
yum install -y http://mirror.centos.org/centos/7/os/x86_64/Packages/gdk-pixbuf2-2.31.6-3.el7.x86_64.rpm
yum install -y http://mirror.centos.org/centos/7/os/x86_64/Packages/gtk2-2.24.28-8.el7.x86_64.rpm
yum install -y \
    qt5-qtbase-devel \
    qt5-qtwebkit-devel \
    qt5-qtsvg-devel

sip

https://riverbankcomputing.com/software/sip/download
wget https://sourceforge.net/projects/pyqt/files/sip/sip-4.18.1/sip-4.18.1.tar.gz
tar zxfv sip-4.18.1.tar.gz 
cd sip-4.18.1
python configure.py
make -j 4
make install

pyqt5

https://www.riverbankcomputing.com/software/pyqt/download5
wget https://sourceforge.net/projects/pyqt/files/PyQt5/PyQt-5.7/PyQt5_gpl-5.7.tar.gz
tar zxfv PyQt5_gpl-5.7.tar.gz
cd PyQt5_gpl-5.7
python configure.py \
    --confirm-license \
    --verbose \
    --destdir=../venv/lib/python3.4/site-packages \
    --sip=../sip-4.18.1/sipgen/sip \
    --sip-incdir=../sip-4.18.1/siplib \
    --target-py-version=3.4 \
    --qmake "/usr/bin/qmake-qt5" \
    --no-designer-plugin \
    -e QtCore \
    -e QtGui \
    -e QtWidgets \
    -e QtNetwork \
    -e QtWebKit \
    -e QtWebKitWidgets \
    -e QtSvg \
    -e QtPrintSupport 

make -j 4
make install 

splash

git clone https://github.com/scrapinghub/splash/
cd splash
pip install -r requirements.txt
pip install -I lupa==1.3 --install-option --no-luajit

python -m splash.server --disable-ui --disable-lua

@peketamin
Copy link
Author

peketamin commented Dec 14, 2016

Scrapy + Selenium + PhantomJS を組み合わせた方が構築早そう。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment