Skip to content

Instantly share code, notes, and snippets.

View jiffyclub's full-sized avatar

Matt Davis jiffyclub

372 followers · 9 following
View GitHub Profile
@jiffyclub
jiffyclub / matplotlib_links.md
Created July 9, 2015 21:03
Links to matplotlib lessons
@jiffyclub
jiffyclub / precipout.py
Last active November 15, 2020 00:37
Example of using Orca to download and concatenate multiple datasets from the California Department of Water Resources. More on Orca at https://udst.github.io/orca/.
import sys
from datetime import datetime
import orca
from caldwr.parsers.precipout import load_precipout
# this will be replaced when calling orca.run, but it's good to
# have a placeholder here so things don't crash when evaluating
# stuff ala carte or with orca-server.
@jiffyclub
jiffyclub / hdf_to_parquet.py
Last active September 24, 2022 16:05
Do the same thing in Spark and Pandas
"""
Convert Pandas DFs in an HDFStore to parquet files for better compatibility
with Spark.
Run from the command line with:
spark-submit --driver-memory 4g --master 'local[*]' hdf5_to_parquet.py
"""
import pandas as pd
@jiffyclub
jiffyclub / svstatic
Last active January 20, 2022 06:00
Convert a SnakeViz HTML file into a self-contained static file that can be hosted anywhere. This script replaces instances of static files being loaded from the local server by having them come from the rawgit CDN.
#!/usr/bin/env python
"""
Prepare an HTML file from SnakeViz for use as a static page.
This makes it so all static files are loaded from a CDN instead
of from the local server.
To get the SnakeViz HTML file run the snakeviz CLI to load a profile
in your browser, than save that page as an HTML file to your computer.
Finally, run this script on that HTML file.
@jiffyclub
jiffyclub / loc_then_groupby.html
Created April 16, 2015 23:47
SnakeViz profile of running df.loc[df.price > 10000].groupby('cut').carat.mean() on a Pandas DataFrame containing the diamonds dataset. See rendered at https://cdn.rawgit.com/jiffyclub/317ae693c15a0aac6cc4/raw/621f23f778beba7b26537ada73e64eb356901bc1/loc_then_groupby.html.
<!doctype html>
<html>
<head>
<meta charset="utf-8" .>
<title>tmppou4ttv_</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<link href="https://cdn.rawgit.com/jiffyclub/snakeviz/v0.4.0/snakeviz/static/snakeviz.css" rel="stylesheet">
<!-- DataTables CSS -->
<link href="https://cdn.rawgit.com/jiffyclub/snakeviz/v0.4.0/snakeviz/static/vendor/jquery.dataTables.min.css" rel="stylesheet">
</head>
@jiffyclub
jiffyclub / snakeviz_glob.html
Created April 16, 2015 22:17
A self-contained SnakeViz page showing a profile of glob.glob. See this rendered at https://cdn.rawgit.com/jiffyclub/0a43706c5daed5445d30/raw/c66530d7f0c6a1b1f2c4403c665b192d597b7f0a/snakeviz_glob.html.
<!doctype html>
<html>
<head>
<meta charset="utf-8" .>
<title>g2</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<link href="https://cdn.rawgit.com/jiffyclub/snakeviz/v0.4.0/snakeviz/static/snakeviz.css" rel="stylesheet">
<!-- DataTables CSS -->
<link href="https://cdn.rawgit.com/jiffyclub/snakeviz/v0.4.0/snakeviz/static/vendor/jquery.dataTables.min.css" rel="stylesheet">
</head>
@jiffyclub
jiffyclub / onpypi
Last active November 14, 2017 06:20
Script for checking whether a name is taken on PyPI.
#!/usr/bin/env python
"""
This script checks whether a name is taken on PyPI.
Usage:
onpypi fakename requests
Prints:
@jiffyclub
jiffyclub / tserv
Last active September 3, 2020 09:14
Start a Tornado static file server in a given directory. To start the server in the current directory: `tserv .`. Then go to `http://localhost:8000` to browse the directory.
#!/usr/bin/env python
"""
Starts a Tornado static file server in a given directory.
To start the server in the current directory:
tserv .
Then go to http://localhost:8000 to browse the directory.
Use the --prefix option to add a prefix to the served URL,
@jiffyclub
jiffyclub / scientific_python_demo.ipynb
Created February 19, 2015 04:38
Demo from SF Python Project Night 2015-02-18
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@jiffyclub
jiffyclub / per-capita.ipynb
Last active August 29, 2015 14:15
Plotting Software Carpentry data using plotly: http://software-carpentry.org/blog/2015/02/plot-this.html
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.