Skip to content

Instantly share code, notes, and snippets.

@beporter
Last active August 30, 2019 20:03
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save beporter/1d0175508409b77ce5688693d64ba86f to your computer and use it in GitHub Desktop.
Save beporter/1d0175508409b77ce5688693d64ba86f to your computer and use it in GitHub Desktop.
Python script to fetch Guild Wars 2 trading post sell orders and send a push notification when they drop below a threshold.

Guild Wars 2 Trading Post alerting script

This script allows you to track the sell price for GW2 items and send an IFTTT notification to your phone when the price drops below your configured threshold. I built this so I can wait till items drop to an acceptable range and be notified to log in and purchase them. The script incorporates rate limiting so as not to overload the GW2 API, as well as a mechanism to avoid repeated alerts if an item stays below your threshold for a length of time.

Details and set up steps outlined below.

Requirements

  • Python 2.7
  • cron or some method of running the script periodically.
  • An ifttt.com account.

Optional

  • (You do not technically need a GW2 API token, but I've written the script to utilize one when present.)
  • bash shell (if you wish to use the gw2_tp_alerts.sh wrapper script for saving your GW2 API token and IFTTT key.)

Preparation

You will have to create an IFTTT.com webhook in order for the script to send notifications to your phone.

  • Create an IFTTT.com account if you don't already have one.

    • Log into your IFTTT account.
  • Create a new applet.

    • Click on +this.
    • Search for webhook.
    • Choose Webhooks.
    • Choose Receive a web request.
      • Enter gw2_tp_alert as the Event Name. ❗ This must be the event name.
    • Press the Create trigger button.
    • Click on +that.
    • Search for notifications.
    • Choose Notifications.
    • Choose Send a rich notification from the IFTTT app.
      • For Title, enter: GW2 TP Alert: {{Value1}}
      • For Message, enter: {{Value2}}
      • For Image URL (optional), enter {{Value3}}
      • Save the applet.
    • Make a note of your webhook key:
      • Go here: https://ifttt.com/maker_webhooks
      • Click on the Documentation link in the header.
      • Make a note of the "key" listed at the top of the page. You'll need this later.
  • Install the IFTTT mobile app on your handheld device.

    • Log into your IFTTT account in the mobile app.

Installation

  • Download gw2_tp_alerts.py and gw2_tp_alerts.sh to a folder on your system.
  • Make sure both files are executable.
  • Save your GW2 API token (if you have one) and IFTTT webhook key (required) in gw2_tp_alerts.sh.
    • Take this opportunity to set the script to limited read access: chmod u=rwx gw2_tp_alerts.sh
  • Find the names and item IDs in the wiki for the items you wish to track.
    • For example, the item Sunrise has ID = 30703.
    • The ID can be found by clicking on the "edit" tab in the wiki and examining the {{infobox}} element at the top. Here is the infobox from the edit page for Sunrise:
      {{Weapon infobox
      | uslot= Superior Sigil of Strength
      | type = Greatsword
      | level = 80
      | bound = account use
      | skin = Sunrise
      | rarity = Legendary
      | prefix = selectable
      | gallery2 = Sunrise wielded.jpg
      | gallery2-text = Wielded, with aura.
      | gallery3 = Sunrise footfalls.jpg
      | gallery3-text = Sunrise footfalls.
      | id = 30703
      }}
      
  • Run gw2_tp_alerts.sh add <id> <threshold> for each item you wish to track.
    • The threshold value supports g and s suffixes: 35g ➡️ 350000, 42s ➡️ 4200

cron

This script should be run regularly for maximum effect. Here's a sample crontab entry that will run the script every 5 minutes:

*/5 * * * *    cd ~/gw2tp/ >/dev/null; (date && ./gw2_tp_alerts.sh) >> gw2_tp_alerts.log 2>&1

Item maintenance

The script is designed to only send notifications when prices drop below the lowest previously seen. This is to prevent duplicate and repeating alerts as the script re-runs and items are still at the same (low) price.

To see the items you have configured:

gw2_tp_alerts.sh list

To add a new item, run:

gw2_tp_alerts.sh add <id> <threshold>

To remove an item:

gw2_tp_alerts.sh del <id>

To reset your preferred threshold for an item:

gw2_tp_alerts.sh threshold <id> <threshold_in_copper>

Gotchas

  • ⚠️ Some shells (and particularly file redirection such as ./gw2_tp_alerts.py > alerts.log) will have issues with unicode output. You can disable it in gw2_tp_alerts.sh by clearing the value for USE_UNICODE. (Set it to the empty string.)
#!/usr/bin/env python
# -*- coding: utf8 -*-
# Ref: https://wiki.guildwars2.com/wiki/API:2/commerce/prices
# Ref: https://forum-en.gw2archive.eu/forum/community/api/HEADS-UP-rate-limiting-is-coming
# Ref: https://wiki.guildwars2.com/wiki/API:2/items
import argparse
import json
import os
import re
import sqlite3
import sys
import time
import urllib
from urllib2 import Request, urlopen, URLError, HTTPError
# ===========================================================================
class DB:
stateVars = ['last_run', 'limit_until']
def __init__(self):
# Create the DB file if it doesn't exist.
self.__dict__['file'] = os.path.basename(__file__) + '.sqlite3'
if not os.path.exists(self.file):
with open(self.file, 'w') as fh:
fh.close()
self.__dict__['conn'] = sqlite3.connect(self.file)
self.conn.row_factory = sqlite3.Row
self.__migrate()
# Apply changes to the schema developed over the lifetime of this script.
def __migrate(self):
# Initialize the tables if they don't exist.
self.conn.execute('''
CREATE TABLE IF NOT EXISTS `items` (
`id` INTEGER NOT NULL UNIQUE,
`name` TEXT,
`threshold` INTEGER NOT NULL,
`lowest_seen` INTEGER DEFAULT 999999999,
PRIMARY KEY(`id`)
);
''')
self.conn.execute('CREATE TABLE IF NOT EXISTS state (last_run INT, limit_until INT);')
# Initialize state _values_ if they don't exist.
cursor = self.conn.cursor()
cursor.execute('SELECT COUNT(*) FROM state;')
if cursor.fetchone()[0] < 1:
self.conn.execute('INSERT INTO state (last_run, limit_until) VALUES (0, 0);')
self.conn.commit()
# Apply sequential migrations.
userVersion = self.conn.execute('PRAGMA user_version;').fetchone()[0]
if userVersion < 1:
self.conn.execute('ALTER TABLE `items` ADD COLUMN `icon_url` TEXT;')
self.conn.execute('PRAGMA user_version(1);')
# Access `state` table vars quickly.
def __getattr__(self, name):
if name not in self.stateVars:
raise AttributeError('state table has no column named {}'.format(name))
cursor = self.conn.cursor()
cursor.execute('SELECT * FROM state;')
return cursor.fetchone()[name]
# Set `state` table vars quickly.
def __setattr__(self, name, value):
if name not in self.stateVars:
raise AttributeError('state table has no column named {}'.format(name))
cursor = self.conn.cursor()
cursor.execute('UPDATE state SET {} = ?;'.format(name), (value,))
self.conn.commit()
# Fetch all configured items to monitor on the trading post.
def items(self):
cursor = self.conn.cursor()
cursor.execute('SELECT * FROM items;')
return { each['id']: each for each in cursor.fetchall() }
# Set the lowest_seen value for the given item.
def lowestSeen(self, itemdId, lowestSeen):
cursor = self.conn.cursor()
cursor.execute('UPDATE items SET lowest_seen = ? WHERE id = ?;', (lowestSeen, itemdId))
self.conn.commit()
return cursor.rowcount == 1
# Add a new item to the db.
def itemAdd(self, id, name, iconUrl, threshold):
cursor = self.conn.cursor()
cursor.execute('SELECT COUNT(*) FROM items WHERE id = ? OR name = ?;', (id, name))
if cursor.fetchone()[0] > 0:
raise ValueError('An item with ID={} or name={} already exists in the database.'.format(id, name))
self.conn.execute('INSERT INTO items (id, name, threshold, icon_url) VALUES (?, ?, ?, ?);', (id, name, threshold, iconUrl))
self.conn.commit()
# Remove an item from the db by id.
def itemDel(self, id):
cursor = self.conn.cursor()
cursor.execute('DELETE FROM items WHERE id = ?;', (id,))
self.conn.commit()
return cursor.rowcount == 1
# Reset the threshold for a given item id.
def itemThreshold(self, id, threshold):
cursor = self.conn.cursor()
cursor.execute('UPDATE items SET threshold = ?, lowest_seen = 999999999 WHERE id = ?;', (threshold, id))
self.conn.commit()
return cursor.rowcount == 1
# ===========================================================================
class RateLimitHitException(Exception):
pass
# ===========================================================================
class RateLimitTtlException(Exception):
pass
# ===========================================================================
class GW2API:
baseUrl = 'https://api.guildwars2.com'
itemsFragment = '/v2/items?ids='
pricesFragment = '/v2/commerce/prices?ids='
token = None
headers = {}
def __init__(self, token = None):
self.token = token
if token:
self.headers = {
'Authorization': 'Bearer ' + token,
}
# Query the Items API for item names and icon URLs.
def items(self, idList):
items = {}
for i in self.__fetch(self.itemsFragment + ','.join(idList)):
items[i['id']] = {
'name': i['name'],
'icon': i['icon'],
}
return items
# Query the API for the collected, comma-separated list of IDs.
def prices(self, idList):
prices = {}
for i in self.__fetch(self.pricesFragment + ','.join(idList)):
prices[i['id']] = i['sells']['unit_price']
return prices
# Internal helper to wrap up http request logistics.
def __fetch(self, urlFragment):
#self.__check_rate_limit()
targetUrl = self.baseUrl + urlFragment
request = Request(targetUrl, None, self.headers)
try:
response = urlopen(request)
except HTTPError as e:
if e.code == 429:
raise RateLimitHitException('Detected rate limit header: ' + e.reason)
raise e
return json.loads(response.read())
# ===========================================================================
class IFTTTNotify:
baseUrl = 'https://maker.ifttt.com/trigger/gw2_tp_alert/with/key/'
key = None
def __init__(self, key):
self.key = key
if self.key is None:
raise EnvironmentError('IFTTT_KEY must be defined in the environment.')
def send(self, itemName, message, link, icon):
targetUrl = self.baseUrl + self.key
payload = {
'value1': itemName,
'value2': message,
'value3': icon,
}
request = Request(targetUrl, urllib.urlencode(payload)) # `payload` implies POST
try:
response = urlopen(request)
except HTTPError as e:
raise e
else:
return response.read()
# ===========================================================================
def copper_to_gold(c):
if c < 100:
return(str(c) + 'c')
if c < 10000:
return(str(round(c / 100.0, 2)) + 's')
return(str(round(c / 10000.0, 2)) + 'g')
# ===========================================================================
# Convert a string like '25s' or '850g' to a numeric value in copper.
def money_to_copper(s):
# Strip everything but numbers and `g` & `s`.
s = re.sub(r"[^0-9gsc]", '', s)
# Replace an optional trailing char with the appropriate number of zeroes.
return re.sub(
r"(\d+)(g|s|c)?",
lambda m: m.group(1) + {'g': '0000', 's': '00', 'c': ''}.get(m.group(2), ''),
s
)
# ===========================================================================
# Helper to obey rate limite headers whenever encountered.
def check_rate_limit(db):
# Grab the last_run time and update the db.
unixNow = int(time.time())
lastRun = db.last_run
db.last_run = unixNow
# Exit early if we're inside a rate limit window.
limitUntil = db.limit_until
if limitUntil > unixNow:
remaining = (limitUntil - unixNow) / 60.0
sys.stderr.write('Rate limit in effect for {} more minutes.'.format(round(remaining, 1))),
exit(1)
# ===========================================================================
def actionRun(*dummy):
db = DB()
# Pull the list of items to query.
items = db.items()
if len(items) == 0:
sys.stderr.write('No items in sqlite database. Please use the `add` command.')
exit(2)
# Perform the trading post API fetch.
api = GW2API(os.environ.get('GW2_API_TOKEN', None)) # https://account.arena.net/applications
try:
prices = api.prices(map(str, items.keys()))
except RateLimitHitException as e:
limitUntil = max(abs(unixNow - lastRun) * 2, 60) # wait at least 60 secs
db.limit_until = unixNow + limitUntil
sys.stderr.write('Rate limit header recieved. Terminating.')
exit(3)
# Clear rate limiting on success.
db.limit_until = 0
# Loop over the stored items and compare to the json price results from the api.
ifttt = IFTTTNotify(os.environ.get('IFTTT_KEY')) # https://ifttt.com/maker_webhooks
for id, props in items.iteritems():
suffix = u'❌' if os.environ.get('USE_UNICODE', False) else 'X'
# Update the "lowest_seen" price for the item so we don't alert again till it's even lower.
db.lowestSeen(id, prices[id])
# If the sell price is below the threshold or lower than we've ever seen,
if prices[id] < min(props['lowest_seen'], props['threshold']):
suffix = u'✅' if os.environ.get('USE_UNICODE', False) else 'O'
# Send a notification via IFTTT.
ifttt.send(
props['name'],
'{} is down to {}'.format(props['name'], copper_to_gold(prices[id])),
'https://wiki.guildwars2.com/index.php?title=Special%3ASearch&search={}&go=Go'.format(urllib.quote(props['name'])),
props['icon_url']
)
print(u'' + props['name'] + ' = ' + copper_to_gold(prices[id]) + ' ' + suffix)
# ===========================================================================
def actionList(*dummy):
db = DB()
items = db.items()
print(' ID | Name | Threshold | Lowest Seen')
print('========|===============================|============|============')
fmt = u'{id:>7} | {name:30}| {threshold:>10} |{lowest_seen:>12}'
for id, props in items.iteritems():
print(fmt.format(
id=props['id'],
name=props['name'],
threshold=copper_to_gold(props['threshold']),
lowest_seen=copper_to_gold(props['lowest_seen'])
))
# ===========================================================================
def actionAdd(args):
db = DB()
api = GW2API(os.environ.get('GW2_API_TOKEN', None)) # https://account.arena.net/applications
try:
items = api.items([str(args.id)])
except RateLimitHitException as e:
limitUntil = max(abs(unixNow - lastRun) * 2, 60) # wait at least 60 secs
db.limit_until = unixNow + limitUntil
sys.stderr.write('Rate limit header recieved. Terminating.')
exit(3)
try:
for id, item in items.items():
db.itemAdd(id, item['name'], item['icon'], money_to_copper(args.threshold))
except ValueError as e:
sys.stderr.write('Item already exists. Not adding.')
exit(4)
print('Item added.')
print('')
actionList()
# ===========================================================================
def actionDel(args):
db = DB()
if not db.itemDel(args.id):
sys.stderr.write('Item could not be removed.')
exit(5)
print('Item deleted.')
print('')
actionList()
# ===========================================================================
def actionThreshold(args):
db = DB()
if not db.itemThreshold(args.id, args.threshold):
sys.stderr.write('Item could not be updated.')
exit(5)
print('Item updated.')
print('')
actionList()
# ===========================================================================
if __name__ == '__main__':
parser = argparse.ArgumentParser(description='Check the Guild Wars 2 Trading Post API for price drops on stored items.')
subparsers = parser.add_subparsers(help='commands', dest='cmd')
runParser = subparsers.add_parser('run', help='[DEFAULT] Run the script, querying the API for current prices and sending alerts for those below the thresholds stored in the SQLite database. Run any command with -h to see detailed help.')
runParser.set_defaults(func=actionRun)
listParser = subparsers.add_parser('list', help='List items stored in the local SQLite database.')
listParser.set_defaults(func=actionList)
addParser = subparsers.add_parser('add', help='Add an item to the local SQLite database.')
addParser.add_argument('id', type=int, help='The numeric API item ID.')
addParser.add_argument('threshold', type=str, help='Sell price at which you wish to be notified. Supports `g` and `s` suffixes. Examples: `25g`, `77s`.')
addParser.set_defaults(func=actionAdd)
delParser = subparsers.add_parser('del', help='Remove an item from the local SQLite database.')
delParser.add_argument('id', type=int, help='The numeric ID of the item to remove.')
delParser.set_defaults(func=actionDel)
thresholdParser = subparsers.add_parser('threshold', help='Reset the threshold for an item in the local SQLite database.')
thresholdParser.add_argument('id', type=int, help='The numeric ID of the item to update.')
thresholdParser.add_argument('threshold', type=int, help='New sell price threshold to save (in copper).')
thresholdParser.set_defaults(func=actionThreshold)
args = parser.parse_args(['run'] if len(sys.argv) == 1 else None)
args.func(args)
#!/usr/bin/env bash
# Wrapper script for embedding your GW2 API token and IFTTT webhook key.
#
# Get these from:
# - https://account.arena.net/applications
# - https://ifttt.com/maker_webhooks (click on "Documentation" in header.)
USE_UNICODE='Yes' \
GW2_API_TOKEN='' \
IFTTT_KEY='' \
./gw2_tp_alerts.py "$@"
@beporter
Copy link
Author

Example iOS message:

gw2_tp_ios_alert

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment