Skip to content

Instantly share code, notes, and snippets.

View NegatioN's full-sized avatar

Joakim Rishaug NegatioN

View GitHub Profile
@NegatioN
NegatioN / lesstimewaste.js
Last active May 27, 2023 09:34
WeDontLikeWastingTime
var portalMain = window.frames['mainwindow'].frames['portalmain'];
var rightSideFrame; // This is named of the form *_rightside, so we search
for(let i = 0; i < portalMain.frames.length; i++) {
if(portalMain.frames[i].name.endsWith('_rightside')) {
rightSideFrame = portalMain.frames[i];
break;
}
}
var frameDoc = rightSideFrame.frames['componentframe'].document;
@NegatioN
NegatioN / dedup.py
Created August 2, 2022 17:08
Delete all but one of jpgs taken at the same second
from PIL import Image, ExifTags
from collections import defaultdict
from datetime import datetime
import os
from tqdm import tqdm
from glob import glob
import sys
glob_string = sys.argv[1]
print(f'Searching in globstring: {glob_string}')
@NegatioN
NegatioN / scriptable_minilm_tokenizer.py
Created May 12, 2022 13:26
A Torchscript-able MiniLM Tokenizer
class MiniLMTokenizer(torch.nn.Module):
def __init__(self, vocab: Dict[str, int], vocab_scores: Dict[str, float]):
'''
:param vocab: A dictionary mapping from string to index of token
:param vocab_scores: A dictionary mapping from string to the score of a given token. This is used to decide
which tokenization is most probable for our input string.
For unigram models this should be avaiable under `~/.cache/torch/sentence_transformers/$YOURMODEL/unigram.json`
You might also need to flip the scores if they're negative.
```
with open(f'~/.cache/torch/sentence_transformers/{org}_{model_name}/unigram.json', 'r') as f:
@NegatioN
NegatioN / script.sh
Created March 31, 2022 09:34
Debugging segfaults in Python
python -q -X faulthandler -c "#Faulty Code"
@NegatioN
NegatioN / t4r.py
Last active January 13, 2022 11:39
MVP Inconsistent output shape
'''
#Conda environment file (env.yml). Install with `conda env create -f env.yml`
channels:
- nvidia
- rapidsai
- anaconda
- conda-forge
dependencies:
- python=3.8
@NegatioN
NegatioN / vim.txt
Last active August 4, 2023 08:24
Vim stuff
\%23c Matches in a specific column. (23), example :%s/\%1c/#/ (insert # on first column of all rows)
:% (all rows), :. (this row), :$ (last row), :.,$ (this->last row), :5,$ (5th->last row)
s, substitute. :%s/a/b/g, change all a->b on all rows.
u, undo
p, paste
m+a, set mark a [works for a-z]
'+a, go to mark a [works for a-z]
<<, de-indent row
>>, indent row
J, join rows
@NegatioN
NegatioN / rename.sh
Created November 30, 2021 19:37
Rename sed script
for f in *; do NEW_NAME=$(echo $f | sed -e "s/.*EP\([0-9][0-9]\).*/S06E\1.mkv/g"); mv ${f} ${NEW_NAME}; done;
@NegatioN
NegatioN / install.sh
Last active September 27, 2021 13:48
Installing Canon Printers Ubuntu
# Origin http://ubuntuhandbook.org/index.php/2020/05/canon-printer-scangear-mp-ubuntu-20-04/
# Ubuntu
sudo apt install printer-driver-gutenprint
sudo add-apt-repository ppa:thierry-f/fork-michael-gruz
sudo apt install scangearmp2
sudo apt install scangearmp-mp495series
# Arch / Manjaro
@NegatioN
NegatioN / kwargs_pytorch.cpp
Last active December 1, 2020 16:30
How to input python-esque kwargs to a pytorch scripted c++ model
torch::jit::IValue myIvalue;
torch::jit::IValue myIvalue2;
torch::jit::script::Module module;
std::unordered_map<std::string, torch::jit::IValue> umap = {{"x", myIvalue}, {"opt", myIvalue2}};
auto result = module.get_method("forward")({}, umap);
// shows all potential arguments to model forward
std::cout << module.get_method("forward").function().getSchema().arguments() << std::endl;
@NegatioN
NegatioN / help.py
Created November 13, 2020 09:48
help.py
df = ### assumes you have a df
category_column = 'main_category'
number = 1000
dfz = []
for cat in df[category_column].unique():
dfz.append(df[df[category_column]==cat].sample(n=number))
final_df = pd.concat(dfz)