Skip to content

Instantly share code, notes, and snippets.

View dusenberrymw's full-sized avatar

Mike Dusenberry dusenberrymw

212 followers · 90 following
View GitHub Profile
@dusenberrymw
dusenberrymw / Softmax Demo.json
Created August 1, 2016 20:05
{"paragraphs":[{"text":"%md\n## Quick Setup","dateUpdated":"2016-08-01T12:09:19-0700","config":{"colWidth":12,"graph":{"mode":"table","height":300,"optionOpen":false,"keys":[],"values":[],"groups":[],"scatter":{}},"enabled":true,"editorMode":"ace/mode/markdown","editorHide":true},"settings":{"params":{},"forms":{}},"jobName":"paragraph_1470078505348_-349631352","id":"20160801-120825_354862280","result":{"code":"SUCCESS","type":"HTML","msg":"<h2>Quick Setup</h2>\n"},"dateCreated":"2016-08-01T12:08:25-0700","dateStarted":"2016-08-01T12:09:17-0700","dateFinished":"2016-08-01T12:09:17-0700","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:1214"},{"text":"import org.apache.sysml.api.mlcontext._\nimport org.apache.sysml.api.mlcontext.ScriptFactory._\n\n// Create a SystemML MLContext object\nval ml = new MLContext(sc)","dateUpdated":"2016-08-01T12:12:09-0700","config":{"colWidth":12,"graph":{"mode":"table","height":300,"optionOpen":false,"keys":[],"values":[],"groups":[],"scatter":{}},"enabled"
@dusenberrymw
dusenberrymw / jupyter_boilerplate.py
Created August 5, 2016 17:30
# Magics
%matplotlib inline
%load_ext autoreload
%autoreload 2
# Imports
import math
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
@dusenberrymw
dusenberrymw / caffeinate.sh
Last active January 4, 2017 21:49
Keep OS X / macOS from sleeping, even while locked.
#!/usr/bin/env bash
caffeinate -di
@dusenberrymw
dusenberrymw / kernel.json
Last active January 5, 2017 00:29
Python 3 + Spark 1.6 + SystemML Jupyter kernel template (`/usr/local/share/jupyter/kernels/pyspark3_1.6/kernel.json`).
{
"argv": [
"/usr/local/opt/python3/bin/python3.5",
"-m",
"ipykernel",
"-f",
"{connection_file}"
],
"language": "python",
"display_name": "Python 3 Spark 1.6 SystemML",
@dusenberrymw
dusenberrymw / setup_drives.sh
Last active January 5, 2017 22:47
Setup new hard drives for HDFS.
# sign in as root: `sudo -i -u root`
# use `df` to see disk usage
# use `lsblk` to see raw disks
parted -s /dev/sdb mklabel gpt;parted -s /dev/sdc mklabel gpt;parted -s /dev/sdd mklabel gpt;parted -s /dev/sde mklabel gpt;parted -s /dev/sdf mklabel gpt;parted -s /dev/sdg mklabel gpt;parted -s /dev/sdh mklabel gpt;parted -s /dev/sdi mklabel gpt;parted -s /dev/sdj mklabel gpt;parted -s /dev/sdk mklabel gpt;parted -s /dev/sdl mklabel gpt;parted -s /dev/sdm mklabel gpt
parted -s /dev/sdb mkpart primary 1 -- -1;parted -s /dev/sdc mkpart primary 1 -- -1;parted -s /dev/sdd mkpart primary 1 -- -1;parted -s /dev/sde mkpart primary 1 -- -1;parted -s /dev/sdf mkpart primary 1 -- -1;parted -s /dev/sdg mkpart primary 1 -- -1;parted -s /dev/sdh mkpart primary 1 -- -1;parted -s /dev/sdi mkpart primary 1 -- -1;parted -s /dev/sdj mkpart primary 1 -- -1;parted -s /dev/sdk mkpart primary 1 -- -1;parted -s /dev/sdl mkpart primary 1 -- -1;parted -s /dev/sdm mkpart primary 1 -- -1
mkdir /disk1;mkdir /disk2;mkdir /disk3;mkdir /disk4
@dusenberrymw
dusenberrymw / systemml_committer_guide.md
Last active January 9, 2017 22:13
Apache SystemML Committer Git Guide

SystemML Git Guide

Setup Git repo locally

  • Fork Apache SystemML to your personal GitHub account by browsing to [https://github.com/apache/incubator-systemml] and clicking "Fork".
  • Clone your personal GitHub fork of Apache SystemML:
    • git clone git@github.com:USERNAME/incubator-systemml.git // assuming the use of SSH keys with GitHub
  • Add GitHub (read-only mirror) and Apache-owned (committer writeable) Git repositories as remotes:
    • cd incubator-systemml
    • git remote add apache-github https://github.com/apache/incubator-systemml.git
  • git remote add apache https://git-wip-us.apache.org/repos/asf/incubator-systemml.git
@dusenberrymw
dusenberrymw / numpy_memory.py
Created April 14, 2017 20:22
NumPy memory address experiments
# NumPy memory experiments
import numpy as np
a = np.random.rand(1, 2, 3)
b = np.asarray(a) # no copy!
c = np.array(a) # copy!
# The array interface is a map that has a 'data' key returning
# a tuple containing a pointer to the memory address and a
# return flag.
@dusenberrymw
dusenberrymw / jupyter_tips_and_tricks.md
Last active June 20, 2017 19:10
Tips & Tricks for Jupyter

Jupyter Tips & Tricks

Show available kernels:

  • jupyter kernelspec list

Install new custom kernel (such as PySpark):

  1. Install dummy kernel in home directory:
  • ipython kernel install --prefix ~/my_custom_jupyter_kernels
  1. Edit name of ~/my_custom_jupyter_kernels/share/jupyter/kernels/python3 folder as desired.
@dusenberrymw
dusenberrymw / git_tips_and_tricks.md
Last active June 26, 2017 19:02
Git Tips & Tricks

Solid Git PR Contributor Workflow

A solid Git pull request workflow will keep you from having issues when contributing work to projects of interest. At the core, the idea is simple: keep a local master branch simply as a means of getting the latest official updates from the project's official Git repo so that you can create new branches from it to work on your desired changes. Then, always open PRs from these new branches, and once the PR is merged into the official Git repo, you can simply move back to master, pull those official changes, and then checkout a brand new branch for the next item you wish to work on.

@dusenberrymw
dusenberrymw / cuda_tips_and_tricks.md
Last active August 24, 2017 00:38
Cuda + cuDNN Tips & Tricks

Cuda + cuDNN Tips & Tricks

Installation

CUDA + cuDNN Installation:

  • Install Cuda (2 options):
    • Download from NVIDIA website. Will install to /Developer/NVIDIA/CUDA=#-# (macOS / OS X), with symlinks in /usr/local/cuda/.
    • Alternatively, use brew cask install cuda.
  • Download CuDNN from NVIDIA website into local_cudnn_path, then:
    • sudo mkdir /usr/local/cudnn
    • sudo cp -r local_cudnn_path/* /usr/local/cudnn/