Skip to content

Instantly share code, notes, and snippets.

@aaronsilber
Last active February 25, 2020 14:48
Show Gist options
  • Save aaronsilber/47371fec0022d1530dce to your computer and use it in GitHub Desktop.
Save aaronsilber/47371fec0022d1530dce to your computer and use it in GitHub Desktop.
CentOS Scrapyd web crawler init script (w/ chkconfig support)
#!/bin/bash
#
# scrapyd This shell script enables scrapyd server on boot
#
# Author: Aaron Silber <aaron@brandthropology.com>
#
# chkconfig: - 50 01
#
# description: Autostart scrapyd web scraper framework daemon
# processname: scrapyd
#
# source function library
. /etc/rc.d/init.d/functions
RETVAL=0
PID='/var/run/scrapyd.pid'
VIRTUALENV='/usr/local/python27/bin/activate'
OPTIONS='--pidfile='$PID
start() {
source $VIRTUALENV
if [ -f $PID ]; then
echo -n $"Scrapy already running."
RETVAL=0
else
echo -n $"Attempting to start scrapyd service... "
scrapyd $OPTIONS &
RETVAL=$?
fi
echo
}
stop() {
source $VIRTUALENV
if [ -f $PID ]; then
echo -n $"Attempting to stop scrapyd service... "
kill `cat $PID`
RETVAL=$?
else
echo -n $"Scrapyd not running"
RETVAL=0
fi
echo
}
restart() {
stop
start
}
case "$1" in
start)
start
;;
stop)
stop
;;
restart|force-reload)
restart
;;
reload)
;;
status)
if [ -f $PID ]; then
echo $"Scrapyd is running."
RETVAL=0
else
echo $"Scrapyd is not running."
RETVAL=3
fi
;;
*)
echo $"Usage: $0 {start|stop|status|restart|reload|force-reload}"
exit 1
esac
exit $RETVAL
@JanWe92
Copy link

JanWe92 commented May 23, 2017

Hey,
thanks for the script! Did you use it to run scrapyd on centos?
I am struggeling with running scrapyd on centos. I could install and start the scrapyd server, but it is not working to add a scrapy crawler to it. I always get an "scrapy - no active projekt" error. Do you have any experience with running scrapyd on centos?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment