Skip to content

Instantly share code, notes, and snippets.

Maxim Lianov limushi

  • Linux Office
  • Moscow, Russia
View GitHub Profile
@limushi
limushi / curl-crawler.sh
Created Jan 11, 2019 — forked from oliveratgithub/curl-crawler.sh
Unix Shell-Script to crawl a list of website URLs using curl
View curl-crawler.sh
#!/bin/sh
timezone="Europe/Zurich"
# List of valid timezones: wikipedia.org/wiki/List_of_tz_database_time_zones
script="${0##*/}"
rootdir=$(cd `dirname "${BASH_SOURCE[0]}"` && pwd)
logfile="$script.log"
log="$rootdir/$logfile"
now=$(TZ=":$timezone" date)
# Uncomment 'mailto=' (remove #) to enable emailing the log upon completion
#mailto="your@email.com"
You can’t perform that action at this time.