Create a gist now

Instantly share code, notes, and snippets.

@bardo /README.md
Last active Apr 8, 2018

Embed
What would you like to do?
Watch a web page for edits with IFTTT and urlwatch

This setup allows to trigger an IFTTT event whenever a web page changes. It uses the urlwatch utility and the IFTTT Maker Channel.

Instructions:

  • Sign up to IFTTT.
  • Connect IFTTT's Maker Channel, and take note of your Maker key.
  • Create a new Recipe that takes the Maker Channel as input and outputs wherever you want. I used IF Notifications with the IF Android app, so that I get a notification on my phone whenever the trigger runs. The script passes the updated URL to IFTTT as value1, and my notification contains URL updated: {{Value1}}, so that I know which of the web pages has been updated.
  • On the machine that will check the website for changes install urlwatch and python2-requests.
  • Echo the URLs to check to ~/.urlwatch/urls.txt, one per line.
  • Then, copy over my hooks.py to ~/.urlwatch/libs/hooks.py adding the URLs to the ifttt_urls tuple and substituting YOUR_TRIGGER_NAME and YOUR_MAKER_KEY.
  • Finally launch crontab -e and set up a cron job to run urlwatch at the frequency you prefer. For example, to have it running every 30 minutes, add a line containing: */30 * * * * urlwatch.
#! /usr/bin/env python2
import hashlib
import os
from requests import post
from urlwatch import html2txt
from urlwatch.handler import JobBase
def filter(url, data):
# URLs that will trigger an IFTTT notification
ifttt_urls = (
'FIRST_URL',
'SECOND_URL',
'THIRD_URL',
# ...and so on
)
if url in ifttt_urls:
# Strip html tags (optional) and compute the cache file name
data = html2txt.html2text(data, method='re')
cache_file = os.path.join(os.path.expanduser('~'), '.urlwatch/cache', JobBase(url).get_guid())
# Don't try comparison if there is no cache file yet
if os.path.isfile(cache_file):
# Hash the data and the cache file
data_hash = hashlib.md5(data).hexdigest()
cache_hash = hashlib.md5(open(cache_file, 'rb').read()).hexdigest()
# If hashes differ, trigger a notification on IFTTT
if data_hash != cache_hash:
post('https://maker.ifttt.com/trigger/YOUR_TRIGGER_NAME/with/key/YOUR_MAKER_KEY', data={'value1':url})
return data
# Catch all
return data
@byorgey

This comment has been minimized.

Show comment
Hide comment
@byorgey

byorgey Nov 29, 2017

Thanks, this works great! One minor typo report that took me a few minutes of head-scratching to figure out: ~/.urlwatch/libs/hooks.py should instead be ~/.urlwatch/lib/hooks.py (lib not libs). At least that's how it works for my version of urlwatch.

byorgey commented Nov 29, 2017

Thanks, this works great! One minor typo report that took me a few minutes of head-scratching to figure out: ~/.urlwatch/libs/hooks.py should instead be ~/.urlwatch/lib/hooks.py (lib not libs). At least that's how it works for my version of urlwatch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment