Skip to content

Instantly share code, notes, and snippets.

View geyang's full-sized avatar
🤖
I finish papers.

Ge Yang geyang

🤖
I finish papers.
View GitHub Profile
@geyang
geyang / instrumentation server setup script
Created September 6, 2020 03:49
ML-Dash and ML-Logger Setup Script
1 df
2 ls /dev/sdh
3 cd /dev/sdh
4 sudo mount /dev/sdb1 ~/runs
5 mkdir runs
6 sudo mount /dev/sdb1 ~/runs
7 sudo mount /dev/sh~/runs
8 sudo mount /dev/sdh ~/runs
9 ls
10 cd runs
@geyang
geyang / README.md
Last active August 26, 2020 19:49
CommonMark-X Landmark Example

Example for Rendering the Landmarks

@geyang
geyang / crop_image.bash
Created July 15, 2020 05:33
use ffmpeg to crop mp4 files
ffmpeg -i input.mp4 -filter:v "crop=674:64:0:0" output.mp4
@geyang
geyang / mac_vanilla.diff
Created May 13, 2020 00:02
diff for the mac implementation
diff --git a/mac/mac_vanilla.py b/mac/mac_vanilla.py
index 1565db7..28d5970 100644
--- a/mac/mac_vanilla.py
+++ b/mac/mac_vanilla.py
@@ -1,3 +1,5 @@
+from copy import deepcopy
+
import torch
from torch import nn
from torch import optim
@geyang
geyang / Upgrading HDF5 and h5py to use SWMR with h5 files.md
Last active March 17, 2020 04:34
hdf5 v1.10 upgrade/installation instruction for using single-write-multiple-read (SWMR) in h5py.

The new 1.10 version of hdf5 library is stable. You can now install from the source to use this new capability.

The installation process has a few caveats, so here is how to do it.

Note:

  1. you need to first remove the hdf5 libraries currently installed in ubuntu.
  2. Then you need to reinstall h5py.
  3. when you reinstall h5py via pip, use the --no-binary flag. This is because the h5py wheel comes with its own hdf5 binary.

Instructions:

@geyang
geyang / transformer.py
Last active March 13, 2020 00:47
quick sketch of a transformer layer.
class MultiHeadedMlp(nn.Module):
def __init__(self, input_dim, *out_dims):
super().__init__()
self.out_dims = out_dims
self.heads = nn.ModuleList([nn.Linear(input_dim, dim) for dim in out_dims])
def forward(self, i):
return [h(i) for h in self.heads]
#!/bin/bash
echo "Restarting bluetooth service..."
blueutil -p 0 && sleep 1 && blueutil -p 1
echo "Waiting bluetooth service to be restored..."
until blueutil -p | grep "1" >/dev/null; do sleep 1; done
echo "Searching for devices not connected..."
devices=($(blueutil --paired | grep "not connected" | awk -F '[ ,]' '{print $2}'))
@geyang
geyang / shared folder logging setup.md
Last active June 30, 2019 19:51
How to setup shared folder writing with ml-logger.md

firs create a group, add users to that group se the folder to this group, recursively if this takes too long, use glob patterns to select the top k-levels

  1. sudo chgrp -R /checkpoint/geyang
  2. sudo chmod -R 2777 /checkpoint/geyang
  3. when running logger, use umask 0 to make sure all files are written with 777 permission.

For all instances writing, really need to umask 0 to make sure that files are writable from multible accounds. For exisiting files written by different user accounts, use chmod -R 2777 to amend the permissions.

@geyang
geyang / jaynes.yml
Created April 22, 2019 06:31
updated jaynes file for amy.
version: 0
hosts:
fair: &devfair_host
ip: localhost
port: 41000
username: geyang
pem: ~/.ssh/fair-loaner
mounts:
- !mounts.SSHCode &plan2vec_code
<<: *devfair_host
@geyang
geyang / tensorflow_data_loading_README.md
Created December 11, 2018 02:23
tensorflow dataloading scripts

List of Data Loader Patterns for Tensorflow

These code come from the Tensorflow documentation the (experimental) new Dataset API.

General types:

  • one_shot
  • initializable
  • reinitializable
  • feedable