General event notification

event-notifications.txt

In http://blog.ndrix.com/2009/02/generalizing-twitter.html I wrote
some thoughts about a generalized event notification system.  This
gist is just some random notes about things I might want to be
notified about.

    * a new bid is placed on an eBay item
    * a UPS/FedEx/USPS package is delivered
    * a UPS/FedEx/USPS package changes location
    * someone commits to an open source project I care about
    * a server I monitor goes down
    * an important bug receives a comment
    * when someone answers my surveymonkey.com survey
    * when the road conditions change (open->closed, closed->open, etc)
    * when the weather changes ("it's raining", i better roll up windows)
    * when a sports score changes ("Broncos scored a touch down")
    * when a Facebook friend changes his status
    * when a Facebook friend adds a new friend
    * when a Facebook friend produces a news item (includes the previous 2)
    * a message is posted in an IRC channel
    * a message is posted in an XMPP chat channel
    * when a wiki page is modified
    * email matching a pattern arrives ("a reply to a specific email I sent")
    * when someone replies to a specific forum thread
    * when a new RFC is published
    * when someone notable dies (Michael Jackson, Les Paul, Ted Kennedy, etc)

I imagine that my typical view on all this would be sort of like an
IRC chat room.  My IRC client would be in that room and various events
would post to the room as they happened.  I'd get Growl notifications,
etc.

Somewhere between the events happening out in the cloud and getting
growl notifications is some infrastructure for tying it all together.

It occurs to me that email maps to this space somewhat nicely.  An
event is a collection of headers and a body.  The body can be in
various formats.  For instance, an event might want to have a Subject
which could be used as the human-readable content to post to an IRC
chat room.  It could also have an application/json body part to be
processed by a machine.

There are tons of sites out there that notify customers about events
using email.  If they could be persuaded to add application/json parts
to those emails, I'd get what I want.

I'd set up an email address for receiving these notifications.  Then
I'd use IMAP IDLE to monitor that address.  As emails came in, I'd
process them and post summaries to my "chat room".

Modeling events as MIME messages would make it easy for others to
publish their events into this network.  One could set up email
addresses to which others could publish properly formatted messages.
Bots would monitor these addresses and send the appropriate
notifications.

Perhaps the best public interface for this system would be a simple
search box.  One describes what one wants to be notified about
(UPS tracking number, sports team, etc) and it sets your account
to receive notifications about that (according to your notification
preferences).  Of course if multiple event streams match your search,
you'd have to choose which one you really care about.

A rough architecture without much thought might be:

  * one bot for each class of events.  these bots sit on servers
    somewhere. each bot monitors the cloud for events it knows about.
    when an event happens, it creates a message representing the
    event in various formats (text/html, text/plain, text/x-sms,
    application/josn, etc).  It sends this message off to queue E

  * dispatch bots monitor queue E.  when a message comes in a dispatch
    bot generates a list of notifications and puts these into their
    respective queues.  for instance, a message about the weather
    changing in Hanna, WY might queue an email notification using the
    text/plain and text/html representations, an IM notification to
    foo@gmail.com and an SMS notification to +1-123-456-7890

  * there are bots for each type of notification.  they monitor their
    respective queues and generate the external notifications requested.

This architecture provides a reasonable separation of concerns.  Adding new
events and new notification mechanisms is trivial.  The dispatch bots are
trickier.  Some event types, like weather(82327), can be reduced to a hash
lookup mapping the event identifier to a list of notifications.  Others require
slightly more computation, like "price of IBM drops below $100".  Ideally, one
should be able to perform arbitrary calculations on an event to decide if it's
interesting and if so what sort of notice to provide.  For instance, "if IBM
drops below $100, send an email.  If it drops below $80, send an IM.  If it
drops below $60, send a text message"

Here's a neat little diagram showing some distractions and one person's view
of how they inter-relate: http://www.informationisbeautiful.net/visualizations/the-hierarchy-of-digital-distractions/