Skip to content

Instantly share code, notes, and snippets.

Matt Davis jiffyclub

Block or report user

Report or block jiffyclub

Hide content and notifications from this user.

Learn more about blocking users

Contact Support about this user’s behavior.

Learn more about reporting abuse

Report abuse
View GitHub Profile
@jiffyclub
jiffyclub / precipout.py
Last active Jun 8, 2016
Example of using Orca to download and concatenate multiple datasets from the California Department of Water Resources. More on Orca at https://udst.github.io/orca/.
View precipout.py
import sys
from datetime import datetime
import orca
from caldwr.parsers.precipout import load_precipout
# this will be replaced when calling orca.run, but it's good to
# have a placeholder here so things don't crash when evaluating
# stuff ala carte or with orca-server.
@jiffyclub
jiffyclub / hdf_to_parquet.py
Last active Aug 8, 2019
Do the same thing in Spark and Pandas
View hdf_to_parquet.py
"""
Convert Pandas DFs in an HDFStore to parquet files for better compatibility
with Spark.
Run from the command line with:
spark-submit --driver-memory 4g --master 'local[*]' hdf5_to_parquet.py
"""
import pandas as pd
@jiffyclub
jiffyclub / svstatic
Last active Mar 7, 2019
Convert a SnakeViz HTML file into a self-contained static file that can be hosted anywhere. This script replaces instances of static files being loaded from the local server by having them come from the rawgit CDN.
View svstatic
#!/usr/bin/env python
"""
Prepare an HTML file from SnakeViz for use as a static page.
This makes it so all static files are loaded from a CDN instead
of from the local server.
To get the SnakeViz HTML file run the snakeviz CLI to load a profile
in your browser, than save that page as an HTML file to your computer.
Finally, run this script on that HTML file.
@jiffyclub
jiffyclub / loc_then_groupby.html
Created Apr 16, 2015
SnakeViz profile of running df.loc[df.price > 10000].groupby('cut').carat.mean() on a Pandas DataFrame containing the diamonds dataset. See rendered at https://cdn.rawgit.com/jiffyclub/317ae693c15a0aac6cc4/raw/621f23f778beba7b26537ada73e64eb356901bc1/loc_then_groupby.html.
View loc_then_groupby.html
<!doctype html>
<html>
<head>
<meta charset="utf-8" .>
<title>tmppou4ttv_</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<link href="https://cdn.rawgit.com/jiffyclub/snakeviz/v0.4.0/snakeviz/static/snakeviz.css" rel="stylesheet">
<!-- DataTables CSS -->
<link href="https://cdn.rawgit.com/jiffyclub/snakeviz/v0.4.0/snakeviz/static/vendor/jquery.dataTables.min.css" rel="stylesheet">
</head>
View snakeviz_glob.html
<!doctype html>
<html>
<head>
<meta charset="utf-8" .>
<title>g2</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<link href="https://cdn.rawgit.com/jiffyclub/snakeviz/v0.4.0/snakeviz/static/snakeviz.css" rel="stylesheet">
<!-- DataTables CSS -->
<link href="https://cdn.rawgit.com/jiffyclub/snakeviz/v0.4.0/snakeviz/static/vendor/jquery.dataTables.min.css" rel="stylesheet">
</head>
@jiffyclub
jiffyclub / onpypi
Last active Nov 14, 2017
Script for checking whether a name is taken on PyPI.
View onpypi
#!/usr/bin/env python
"""
This script checks whether a name is taken on PyPI.
Usage:
onpypi fakename requests
Prints:
@jiffyclub
jiffyclub / tserv
Last active Sep 30, 2019
Start a Tornado static file server in a given directory. To start the server in the current directory: `tserv .`. Then go to `http://localhost:8000` to browse the directory.
View tserv
#!/usr/bin/env python
"""
Starts a Tornado static file server in a given directory.
To start the server in the current directory:
tserv .
Then go to http://localhost:8000 to browse the directory.
Use the --prefix option to add a prefix to the served URL,
@jiffyclub
jiffyclub / scientific_python_demo.ipynb
Created Feb 19, 2015
Demo from SF Python Project Night 2015-02-18
View scientific_python_demo.ipynb
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
View per-capita.ipynb
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
You can’t perform that action at this time.