Skip to content

Instantly share code, notes, and snippets.

View vanpelt's full-sized avatar
⭐
pro

Chris Van Pelt vanpelt

⭐
pro
271 followers · 20 following
View GitHub Profile
@vanpelt
vanpelt / 01_enable-services.sh
Created February 2, 2022 07:37
Local init scripts
#!/bin/bash
# move all services to runit, was tricky to make this happen in docker without
# overwriting cron / sshd
echo "*** Copying services to runit"
mv /home/wandb/service/* /etc/service/
mv /home/wandb/wandb-logrotate /etc/logrotate.d/
HOST=${HOST:-http://localhost:8080}
@vanpelt
vanpelt / README.md
Last active June 6, 2019 21:53
Deep Learning Resources

Overview

In preperation for our 10 week course we want to offer up some resources for students interested in getting a head start.

Educational Resources

We have a series of online videos that assume no knowledge of deep learning or machine learning and cover all of the basic applications. Each video is around 20 minutes. If you want to get a feel for what the class will be like or brush up on the fundamentals, feel free to watch them at your own pace: https://www.wandb.com/classes/intro/overview. We will cover a lot of these topics in more depth in person.

The only real requirement for these classes is a solid understanding of python. If you can follow along with the videos, you will be in good shape. If not, you may want to brush up on your python with a book or an online course. The books Deep Learning with Python and Hands-on Machine Learning with Scikit-Learn, Keras, and TensorFlow: Concepts, Tools, and Techniques to Build Intelligent Systems are excellent compliments to the class materials

@vanpelt
vanpelt / sweep.py
Created January 11, 2019 20:27
Sagemaker Sweep Monitor
import sagemaker
import wandb
tuner_name = "sagemaker-pytorch-190111-1056"
tuner = sagemaker.HyperparameterTuner.attach(tuner_name)
job_name = tuner.analytics().description()['TrainingJobDefinition']['StaticHyperParameters']['sagemaker_job_name']
client = tuner.sagemaker_session.sagemaker_client
api = wandb.Api()
runs = api.runs("wandb/sm-pytorch-cifar", {"config.sagemaker_job_name": job_name})
for run in runs:
@vanpelt
vanpelt / requirements.txt
Created July 26, 2018 22:13
ENV
absl-py==0.2.2
alembic==1.0.0
argh==0.26.2
astor==0.6.2
async-generator==1.9
backcall==0.1.0
bleach==1.5.0
certifi==2018.4.16
chainer==4.1.0
chardet==3.0.4
@vanpelt
vanpelt / upsert.py
Created March 20, 2018 06:33
Flask SqlAlchemy MySQL ON DUPLICATE KEY UPDATE returning auto increment id UPSERT HOTNESS
from app import db
from sqlalchemy import func
from sqlalchemy.dialects.mysql import insert
def upsert(model, insert_dict):
"""model can be a db.Model or a table(), insert_dict should contain a primary or unique key."""
inserted = insert(model).values(**insert_dict)
upserted = inserted.on_duplicate_key_update(
id=func.LAST_INSERT_ID(model.id), **{k: inserted.inserted[k]
for k, v in insert_dict.items()})
@vanpelt
vanpelt / README.md
Last active October 27, 2017 02:11
W&B Frontend Coding Exercise

Overview

The task at hand is to create a sortable tableview of issues listed in the tensorflow/tensorflow github repository using React, ApolloClient, and the GitHub GraphQL API.

MVP

The MVP should render the first page of issues with a hard coded access token. The look and feel is less important than the function, but still considered.

@vanpelt
vanpelt / README.md
Created September 13, 2017 21:02
Script to enable iframe cross origin communication with CrowdFlower

cml_iframe

To communicate with CrowdFlower run the following whenever events occur requiring data exchange:

window._cf.post("persist", {"key": "value"})
@vanpelt
vanpelt / cml.js
Last active April 12, 2017 00:03
Using require.js in CrowdFlower Jobs
//The test question interface doesn't have require.js, this mimics minimal functionality
var require = window.require || function() {
var args = Array.prototype.slice.call(arguments);
var opts_or_list = args.shift(), opts, list
if(Array.isArray(opts_or_list)) {
opts = {}, list = opts_or_list
} else {
list = args.shift(), opts = opts_or_list
}
var libraries = ['//cdnjs.cloudflare.com/ajax/libs/twitter-bootstrap/2.3.2/js/bootstrap.min.js']
@vanpelt
vanpelt / prediction_schema.json
Last active February 10, 2017 23:03
An overview of the prediction formats usable with the CrowdFlower annotation tool
{
"shapes":[
{"type": "predicted_box", "width": 100, "height": 100, "x": 100, "y": 100, "rotation": 180, "tag": "optional class", "probability": 0.8},
{"type": "predicted_poly", "points": [100, 100, 150, 150, 100, 100], "tag": "optional class", "probability": 0.8},
{"type": "predicted_dots", "dots": [{"x":100, "y":100, "tag": "optional class"}], "tag": "optional class", "probability": 0.8}
]
}
@vanpelt
vanpelt / documentation.md
Last active February 3, 2017 01:41
Options for 3rd Party Integration into CrowdFlower

Option 1: CML iFrame

Include the following javascript snippet in the <head> of your custom interface:

<script src="https://crowdflower.github.io/construe/cml_iframe.js"></script>

Then there are two required calls for your interface to communicate with the CrowdFlower platform: