Skip to content

Instantly share code, notes, and snippets.

Avatar

Mark Kockerbeck xeb

View GitHub Profile
@xeb
xeb / ingest.py
Created Aug 31, 2020
Process all word documents into a single text file (for GPT training)
View ingest.py
#!/usr/bin/env python
import os
import docx2txt
from absl import app
from absl import flags
FLAGS = flags.FLAGS
flags.DEFINE_string("output", "output.txt", "The path of final output.")
flags.DEFINE_string("source", None, "The source path to process.")
@xeb
xeb / fingerprints.txt
Created Sep 29, 2019
OpenPilot Chrysler Pacifica 2019 (non-Hybrid) Fingerprints
View fingerprints.txt
This file has been truncated, but you can view the full file.
number of messages 2:
fingerprint 388: 4, 1024: 8
number of messages 4:
fingerprint 388: 4, 448: 6, 1024: 8, 1025: 8
number of messages 9:
fingerprint 274: 2, 388: 4, 448: 6, 752: 2, 817: 8, 906: 8, 924: 8, 1024: 8, 1025: 8
number of messages 14:
fingerprint 274: 2, 388: 4, 448: 6, 520: 8, 532: 8, 564: 8, 752: 2, 817: 8, 906: 8, 924: 8, 937: 8, 996: 8, 1024: 8, 1025: 8
number of messages 18:
@xeb
xeb / model.py
Created Jul 6, 2019
XGBoost Example
View model.py
#!/usr/bin/env python
#!pip install sklearn xgboost
#!wget https://raw.githubusercontent.com/jbrownlee/Datasets/master/pima-indians-diabetes.data.csv
from numpy import loadtxt
from xgboost import XGBClassifier
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score
import pprint
@xeb
xeb / Dockerfile.py35
Created Aug 23, 2016
fastText with Python 3.5 Example
View Dockerfile.py35
# Modified from: https://github.com/xeb/fastText-docker/blob/master/Dockerfile
# To build & run, do:
# docker build -t fasttext-py3.5 -f Dockerfile.py35 .
# docker run --rm -it fasttext-py3.5 ./eval.py
#
FROM python:3.5
RUN apt-get update && apt-get install -y \
build-essential \
wget \
git \
@xeb
xeb / Dockerfile
Created Jul 10, 2016
TensorFlow Serving Issue 114
View Dockerfile
# Dockerfile to reproduce Issue 114 for TensorFlow Serving
# https://github.com/tensorflow/serving/issues/114
# Base Dockerfile from: https://github.com/tensorflow/serving/blob/master/tensorflow_serving/tools/docker/Dockerfile.devel
# To demonstrate this, just build the container & then run it:
# $ docker build -t tfserv_114 .
# $ docker run -it tfserv_114
FROM ubuntu:14.04
RUN apt-get update && apt-get install -y \
@xeb
xeb / docker_cheatseet.md
Created Jul 10, 2016
Docker Cheatsheet
View docker_cheatseet.md

Remove unused images

docker rmi $(docker images --filter "dangling=true" -q --no-trunc)

@xeb
xeb / Vagrantfile
Last active Jul 12, 2016
Basic Vagrant file
View Vagrantfile
# -*- mode: ruby -*-
# vi: set ft=ruby :
Vagrant.configure(2) do |config|
config.vm.box = "ubuntu/xenial64"
#config.vm.box = "bento/centos-7.2"
#config.vm.box_url = "https://atlas.hashicorp.com/bento/boxes/centos-7.2"
config.vm.network "forwarded_port", guest: 5000, host: 5000
config.vm.synced_folder "scripts", "/home/vagrant/scripts"
@xeb
xeb / ssh_reverse.md
Last active Feb 18, 2016
Reverse SSH
View ssh_reverse.md

SSH Reverse Proxy

Some simple notes to setup SSH reverse proxy

  1. on source machine (behind the network), run: ssh -N -R 2222:localhost:22 YOURUSER@YOURSERVER

  2. on YOURSERVER, run: ssh -l SOURCEUSER -p 2222 localhost

  3. If you want to automated it, do something like...

@xeb
xeb / install-tensorflow.sh
Created Dec 17, 2015
TensorFlow Install on Mac OS X with sudo PIP and Six pre-installed
View install-tensorflow.sh
#!/bin/bash
sudo -H pip install --upgrade https://storage.googleapis.com/tensorflow/mac/tensorflow-0.6.0-py2-none-any.whl --ignore-installed six
# Modified install command from: https://www.tensorflow.org/versions/master/get_started/os_setup.html
# See: https://github.com/pypa/pip/issues/3165
# Because Apple has preinstalled the dependency Six that cannot be removed
# Some of the errors I experienced were the following:
# [Errno 1] Operation not permitted: '/tmp/pip-BriGIp-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib
@xeb
xeb / flac2mp3
Created Dec 7, 2015
Converts FLAC to MP3
View flac2mp3
#!/bin/bash
# from: http://apple.stackexchange.com/questions/9064/how-to-i-go-from-flac-to-mp3-using-lame-flac-using-the-terminal-alone
for f in "$@"; do
[[ "$f" != *.flac ]] && continue
album="$(metaflac --show-tag=album "$f" | sed 's/[^=]*=//')"
artist="$(metaflac --show-tag=artist "$f" | sed 's/[^=]*=//')"
date="$(metaflac --show-tag=date "$f" | sed 's/[^=]*=//')"
title="$(metaflac --show-tag=title "$f" | sed 's/[^=]*=//')"
year="$(metaflac --show-tag=date "$f" | sed 's/[^=]*=//')"