Roy Keyes
22 July 2020 - This is a post on my blog. Grab the RSS feed here 😉
Recently one of my relatives really wanted a dog. Not just any dog, but a specific puppy at an animal shelter in town. The problem was that the dog was not yet available for adoption, as the shelter was still waiting some amount of time in the event that the dog's owner would claim it. That basically meant we could call the shelter to check the status or keep checking the shelter's website.
Of course checking the web and all is doable, but why do that when we have perfectly good computers designed to do exactly that kind of thing. So naturally I decided to write a script to scrape the webpage with the dog's status and alert me when the status changed to "available for adoption".
Scraping the webpage was straight-forward, I used just a few throwaway lines of Python with requests to fetch the page and Beautiful Soup to extract the specific piece of text I wanted.
import requests
from bs4 import BeautifulSoup as bs
url = <the url>
r = requests.get(url)
soup = bs(r.content, 'lxml')
status = soup.find(id="lblStage").text
if status == 'Not Available - Pending':
s = status
else:
s = status + ' 🔍🔍🔍🐕🐕🐕🔍🔍🔍'
print("Puppy status: {}".format(s))Easy enough!
I could then run this periodically via cron. But the question I didn't quite know the answer to, was how I could easily create alerts (in particular on my phone) when the dog's adoption status changed.
This is where Keybase comes in. For those of you not familiar with Keybase, it's a company that provides services centered on end-to-end encryption, specifically Slack-style chat, Dropbox-style file storage, and encrypted git repos, while also serving as an identity / encryption key directory.
(Keybase is not without controversy, in particular its publicity scheme around giving away free cryptocurrency[1] and later being purchased by Zoom[2].)
Keybase is particularly interesting if you like command line interfaces, as it is designed to be very unix-y. I wanted to see if it would be possible to easily create a script that would send me an alert via keybase chat to let me know when the status of the dog had changed.
Keybase has a pretty decent desktop GUI chat client and corresponding mobile apps, but you can also use the command line API to do pretty much everything you can do in the apps (plus a lot more). To send a chat message to another Keybase user, you simply do something like this:
keybase chat send <another users username> "Hello!"And because this is unix-y, you can of course use this in conjunction with other command line tools. For example, you could pipe the value of a shell variable to a chat via:
echo $greeting | keybase chat send <another user>Try keybase --help and keybase <command> --help to see all of the commands and options.
Using the above commands was getting me 90% of the way there, but there was one problem: since I was sending messages from my own user to myself, Keybase would not create notifications, under the logical assumption that I had probably already read what I myself wrote. The solution was to create a Keybase bot to send me messages.
Like several other chat services, Keybase allows users to build and integrate bots. A bot on Keybase is essentially the same as any other user, except that it's managed by another user. To create a bot I used the following commands:
keybase bot token create
keybase bot signup -u <my bots name> -t <the token I generated above>I then set the bot up with its own home directory, where its Keybase config files, etc, would live:
mkdir /path/to/mybotshomedir
keybase --home=/path/to/mybotshomedir loginThis will prompt you to provision a (paper key) device for your bot. You are now logged in as the bot, but since the bot has a different home directory than your regular user, running commands like you normally would will be done as your normal user. To run commands as the bot, you can pass the global option --home to explicitly specify a home directory:
keybase --home=/path/to/mybotshomedir chat send roycoding "Hello!"Now, since a different user (i.e. my bot) is messaging me, I will get a notification on my desktop and phone, depending on my client notification preferences 🚨!!
N.b. help on global options can be found via keybase help advanced.
Initially I had specified the bot's directory via a relative path like ~/mybotshomedir, but this resulted in buggy behavior. The simplest workaround I found was to create the bot's directory with the full path and always use that 🤷
I'll update this post if I find and fix the root issue.
I now had notifications working, which is great, but on the other hand, I didn't want a notification for non-interesting events (while still wanting to know that the checks were occurring). To solve this I put logic into my script to send the current "unavailable" status messages from my normal user and any other status messages as my bot user, thus triggering a notification.
Finally I set the script to run every 15 minutes via cron.
#! /bin/bash
export XDG_RUNTIME_DIR=/run/user/1000
HOME=/home/roycoding
source $HOME/venv-dog/bin/activate
timestamp=`date +%F-%T-%Z`
status=`python $HOME/dog/status_check.py`
if [ "$status" == "Puppy status: Not Available - Pending" ];
then
echo $timestamp: $status | keybase chat send mybot
else
echo $status | keybase --home=/path/to/mybotshomedir chat send roycoding
fi
deactivateSending messages as my normal user initially didn't work running this script via cron. I ended up with a hacky workaround by setting the XDG_RUNTIME_DIR environmental variable explicitly. In the future I plan to do what I'm told is the "correct" way to do this and run the script via systemd, but that's for another post.
It worked!
After a few days I suddenly heard a ping on my phone and saw a message from my bot that the dog's adoption status was now "Available". A quick trip down to the shelter and my relative now has their new dog! I suspect the shelter was not expecting that someone was creating a bot to let them "snipe" the dog before someone else adopted it...
I'm pretty happy with how easily this came together. As with any project there were a few hiccups, but I was able to overcome them at the level needed for this one-off project. I'm hoping to recycle and improve upon this for some other future projects, where mobile event notifications would be nice.
- Keybase has a nice document called the Keybase Book, which provides a decent walk-through of there many services.
- I want to thank the members of the Keybase chat group
mkbot, who provided me with help figuring this out and troubleshooting.
[1] Checkout Vicki's newsletter, The Vicki Boykis Experience, for coverage of Keybase and their cryptocurrency "event".
[2] Unfortunately the acquisistion of Keybase by Zoom makes the continued availability of Keybase's services much less clear 😦