Skip to content

Instantly share code, notes, and snippets.

Greg Werner jgwerner

Block or report user

Report or block jgwerner

Hide content and notifications from this user.

Learn more about blocking users

Contact Support about this user’s behavior.

Learn more about reporting abuse

Report abuse
View GitHub Profile
jgwerner / gist:d2436c3006d11c1953a0ab894691d041
Created Nov 6, 2019
Alternative to docker images purge to remove dangling images.
View gist:d2436c3006d11c1953a0ab894691d041
# remove docker images by line in text file
# alternative to `docker images purge`
list_dangling_images () {
docker images --filter dangling=true | while read IMAGE_ID; do
>> $images_file
# Based in large part from
set -e
print_help() {
echo "Usage: bash OPTION"
echo -e "\nOPTIONS:"
View end_to_end_notebook.ipynb
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
jgwerner /
Created Jun 20, 2017
Standard data science workflow

Data Science Workflow

Define the Problem

  • What is the problem? Provide formal and informal definitions.
  • Why does the problem need to be solved? Motivation, benefits, how it will be used.
  • How would I solve the problem? Describe how the problem would be solved manually to flush domain knowledge.

Prepare Data

jgwerner /
Created May 19, 2017
Jupyter Notebook config enabled for IFrames.
from jupyter_core.paths import jupyter_data_dir
import subprocess
import os
import errno
import stat
c = get_config()
c.NotebookApp.ip = '*'
c.NotebookApp.port = 8888
c.NotebookApp.open_browser = False
jgwerner /
Last active Oct 24, 2017
Nvidia docker installation script
# based in large part from
set -x
set -e
# Update and upgrade
echo "--"
echo "-- Running ubuntu updates and Ugrade"
jgwerner /
Last active Feb 1, 2017
Useful docker commands

Maintenance and troubleshooting commands

Installing Docker

Docker install script is probably the best way to install docker with Ubuntu, as it's the official bash script maintained by Docker and is updated with latest features:

$ curl -sSL | sh

Uninstalling Docker

jgwerner / nginx.conf
Last active Jul 14, 2017
View nginx.conf
worker_processes auto;
error_log /dev/stdout debug;
events {
worker_connections 2048;
http {
upstream flaskserver {
jgwerner / pyspark-config
Created Jul 5, 2016
PySpark Jupyter Notebook configuration
View pyspark-config
# Jupyter Notebook Python, Spark, Mesos Stack
## What it Gives You
* Jupyter Notebook 4.2.x
* Conda Python 3.x and Python 2.7.x environments
* pyspark, pandas, matplotlib, scipy, seaborn, scikit-learn pre-installed
* Spark 1.6.0 for use in local mode or to connect to a cluster of Spark workers
* Mesos client 0.22 binary that can communicate with a Mesos master
You can’t perform that action at this time.