Wiki scraper to grab scroll lists for Mojang's Scrolls game.

README.md

Contains a list of each faction's scrolls, and a script to update them all.

The format used is a tab-separated table, columns are:

Card ID | Set | Name | Type | Subtype | Rarity | Cost | Attack | Countdown | Health | Trait(s) | Ability | Flavor

Run update in order to grab the latest (hopefully) scrolls data from the scrollsguide wiki.

You will need to have Python 2 as well as the python modules requests and beautiful soup installed.

This does not provide any sort of parsing ability, but if you have grep/awk you can search through the files pretty easily, without having to open a browser.

update

#!/usr/bin/python

from requests import get
from bs4 import BeautifulSoup

factions = {
    "decay" : "http://www.scrollsguide.com/wiki/Decay",
    "growth": "http://www.scrollsguide.com/wiki/Growth",
    "order" : "http://www.scrollsguide.com/wiki/Order",
    "energy": "http://www.scrollsguide.com/wiki/Energy"
}

def updatelist(faction):
    raw_page = get(factions[faction]).content
    soup = BeautifulSoup(raw_page)
    tsv_list = parselist(soup)
    outfile = open(faction, 'w')
    outfile.write(tsv_list)
    outfile.close()

def parselist(soup):
    # I cba to actually find out which of the tables we need, so let's just take
    # the biggest one. The scrolls list has 150+ lines, the other ones < 10.
    scrollstable = max(soup.find_all("table"), key = lambda t: len(t.contents))
    # Strip the first row, we don't need headers
    rows = scrollstable.find_all("tr")[1:]
    cleaned_rows = map(lambda row: [e.text.strip() for e in row.find_all("td")], rows)
    cleaned_rows = map(lambda row: map(lambda e: e if e else ".", row), cleaned_rows)
    return "\n".join(map(lambda row: "\t".join(row), cleaned_rows))

if __name__ == "__main__":
    for faction in factions:
        print "Updating %s ..." % faction
        updatelist(faction)