Skip to content

Instantly share code, notes, and snippets.

Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
import operator
import requests
from bs4 import BeautifulSoup
def get_data_for_category(soup, data_category_id):
output_data = {}
data_rows = soup.find_all("div", attrs={"data-category-id": data_category_id})[0]
for row in data_rows.find_all("div", "ProgressBar-textWrapper"):
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@bcbwilla
bcbwilla / amcsd_scraper.py
Created February 25, 2015 16:01
amcsd scraper
""" Extract the CIF files for each mineral in the
American Minerologist Crystal Structure Database
http://rruff.geo.arizona.edu/AMS/
Puts files in 'amcsd_data' folder.
"""
import requests
import string
import urllib
@bcbwilla
bcbwilla / mrl_scraper.py
Last active August 29, 2015 14:15
Scrapes data from mrl.uscb.edu thermoelectric materials datamine.
"""
Scrapes all dimensions of data from http://www.mrl.ucsb.edu:8080/datamine/thermoelectric.jsp.
RE: electrical resistivity
SE: seebeck coefficient
TH: thermal conductivity
AT: average atomic mass
SC: scarcity
HP: HHI (production)
HR: HHI (reserves)
@bcbwilla
bcbwilla / backup.sh
Created December 7, 2014 00:21
Super simple minecraft server world backup with zippy bits
#!/bin/bash
# Minecraft Server Backup
stamp=$(date +%d_%m_%Y_%H_%M_%S)
cd $HOME/servers/backups/mcprohost/
mkdir -p $stamp
wget -r -nH -P $stamp ftp://$MCP_USER:$MCP_PASS@$MCP_IP/world*
zip -rm $stamp $stamp
@bcbwilla
bcbwilla / analysis.ipynb
Last active August 29, 2015 14:05
Fun with Overcast Network Player Count Data
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@bcbwilla
bcbwilla / graph.py
Last active August 29, 2015 14:03
Overcast Network (oc.tc) server player count data collector.
import random
import math
import datetime
from pymongo import MongoClient
import matplotlib.pyplot as plt
import matplotlib.dates as mdates
from matplotlib.ticker import MultipleLocator
import numpy as np
@bcbwilla
bcbwilla / bstat.py
Last active December 19, 2015 16:49
This script is an improved version of the PBS 'qstat' command that prints out only USEFUL job information (like the FULL name) in a readable way. All of the output of the command 'qstat -f' is captured and organized, so it would be very easy to print out additional info by modifying the last loop of the script.
#! /usr/bin/python
#
# This script is an improved version of the 'qstat' command
# that prints out only USEFUL information (like the job's
# FULL name) in a readable way.
#
# All of the output of the command 'qstat -f' is captured
# and organized, so it would be very easy to print out
# additional info by modifying the last loop of the script.
#