Instantly share code, notes, and snippets.

@bardo /README.md
Last active Oct 17, 2018

Embed
What would you like to do?
Watch a web page for edits with IFTTT and urlwatch

This setup allows to trigger an IFTTT event whenever a web page changes. It uses the urlwatch utility and the IFTTT Maker Channel.

Instructions:

  • Sign up to IFTTT.
  • Connect IFTTT's Maker Channel, and take note of your Maker key.
  • Create a new Recipe that takes the Maker Channel as input and outputs wherever you want. I used IF Notifications with the IF Android app, so that I get a notification on my phone whenever the trigger runs. The script passes the updated URL to IFTTT as value1, and my notification contains URL updated: {{Value1}}, so that I know which of the web pages has been updated.
  • On the machine that will check the website for changes install urlwatch and python2-requests.
  • Echo the URLs to check to ~/.urlwatch/urls.txt, one per line.
  • Then, copy over my hooks.py to ~/.urlwatch/libs/hooks.py adding the URLs to the ifttt_urls tuple and substituting YOUR_TRIGGER_NAME and YOUR_MAKER_KEY.
  • Finally launch crontab -e and set up a cron job to run urlwatch at the frequency you prefer. For example, to have it running every 30 minutes, add a line containing: */30 * * * * urlwatch.
#! /usr/bin/env python2
import hashlib
import os
from requests import post
from urlwatch import html2txt
from urlwatch.handler import JobBase
def filter(url, data):
# URLs that will trigger an IFTTT notification
ifttt_urls = (
'FIRST_URL',
'SECOND_URL',
'THIRD_URL',
# ...and so on
)
if url in ifttt_urls:
# Strip html tags (optional) and compute the cache file name
data = html2txt.html2text(data, method='re')
cache_file = os.path.join(os.path.expanduser('~'), '.urlwatch/cache', JobBase(url).get_guid())
# Don't try comparison if there is no cache file yet
if os.path.isfile(cache_file):
# Hash the data and the cache file
data_hash = hashlib.md5(data).hexdigest()
cache_hash = hashlib.md5(open(cache_file, 'rb').read()).hexdigest()
# If hashes differ, trigger a notification on IFTTT
if data_hash != cache_hash:
post('https://maker.ifttt.com/trigger/YOUR_TRIGGER_NAME/with/key/YOUR_MAKER_KEY', data={'value1':url})
return data
# Catch all
return data
@byorgey

This comment has been minimized.

Show comment
Hide comment
@byorgey

byorgey Nov 29, 2017

Thanks, this works great! One minor typo report that took me a few minutes of head-scratching to figure out: ~/.urlwatch/libs/hooks.py should instead be ~/.urlwatch/lib/hooks.py (lib not libs). At least that's how it works for my version of urlwatch.

byorgey commented Nov 29, 2017

Thanks, this works great! One minor typo report that took me a few minutes of head-scratching to figure out: ~/.urlwatch/libs/hooks.py should instead be ~/.urlwatch/lib/hooks.py (lib not libs). At least that's how it works for my version of urlwatch.

@silviuvulcan

This comment has been minimized.

Show comment
Hide comment
@silviuvulcan

silviuvulcan Oct 2, 2018

I guess it doesn't work with urlwatch2 ? Seeing as it's python3 based...

silviuvulcan commented Oct 2, 2018

I guess it doesn't work with urlwatch2 ? Seeing as it's python3 based...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment