Skip to content

Instantly share code, notes, and snippets.

@andygimma

andygimma/MGS_scraper_uptime_README.md

Created September 24, 2021 21:42
Show Gist options
  • Save andygimma/45cbaa0ddd9500fa5a140f5c318e8c4a to your computer and use it in GitHub Desktop.
Save andygimma/45cbaa0ddd9500fa5a140f5c318e8c4a to your computer and use it in GitHub Desktop.
Download ZIP
Raw
MGS_scraper_uptime_README.md

Improve Python Scraper Uptime

Start MGS_scraper.py on boot

  1. $ crontab -e
  2. Add @reboot sh /path/to/find_or_start_scraper.sh
  3. Be sure to replace /path/to/ with the correct path.

Restart the scraper every X minutes, if it has crashed

  1. $ crontab -e
  2. Add 10 * * * * /path/to/find_or_start_scraper.sh
    1. Be sure to replace /path/to/ with the correct path.
  4. The first number represents minutes. The example is set to run every 10 minutes. Setting it to '0' would run the script hourly. Seeting it to '*' would run the script every minute.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment