Skip to content

Instantly share code, notes, and snippets.

@mobilemind
mobilemind / git-tag-delete-local-and-remote.sh
Last active July 2, 2024 00:02
how to delete a git tag locally and remote
# delete local tag '12345'
git tag -d 12345
# delete remote tag '12345' (eg, GitHub version too)
git push origin :refs/tags/12345
# alternative approach
git push --delete origin tagName
git tag -d tagName
@MInner
MInner / parallel.py
Last active August 9, 2022 02:14
Executing jobs in parallel with a nice progress bar: a tqdm wrapper for joblib.Parallel
from tqdm import tqdm_notebook as tqdm
from joblib import Parallel, delayed
import time
import random
def func(x):
time.sleep(random.randint(1, 10))
return x
@niranjv
niranjv / install_python_36_amazon_linux.sh
Last active January 30, 2023 21:49
Install Python 3.6 in Amazon Linux
# A virtualenv running Python3.6 on Amazon Linux/EC2 (approximately) simulates the Python 3.6 Docker container used by Lambda
# and can be used for developing/testing Python 3.6 Lambda functions
# This script installs Python 3.6 on an EC2 instance running Amazon Linux and creates a virtualenv running this version of Python
# This is required because Amazon Linux does not come with Python 3.6 pre-installed
# and several packages available in Amazon Linux are not available in the Lambda Python 3.6 runtime
# The script has been tested successfully on a t2.micro EC2 instance (Root device type: ebs; Virtualization type: hvm)
# running Amazon Linux AMI 2017.03.0 (HVM), SSD Volume Type - ami-c58c1dd3
# and was developed with the help of AWS Support
@johnhw
johnhw / umap_sparse.py
Last active January 6, 2024 16:09
1 million prime UMAP layout
### JHW 2018
import numpy as np
import umap
# This code from the excellent module at:
# https://stackoverflow.com/questions/4643647/fast-prime-factorization-module
import random
# Install go
$ sudo apt-get install golang-go
# Download goofys package
$ go get github.com/kahing/goofys
# Install goofys
$ go install github.com/kahing/goofys
# Copy the goofys binary to somewhere in your path
@tamuhey
tamuhey / tokenizations_post.md
Last active June 26, 2024 01:00
How to calculate the alignment between BERT and spaCy tokens effectively and robustly

How to calculate the alignment between BERT and spaCy tokens effectively and robustly

image

site: https://tamuhey.github.io/tokenizations/

Natural Language Processing (NLP) has made great progress in recent years because of neural networks, which allows us to solve various tasks with end-to-end architecture. However, many NLP systems still require language-specific pre- and post-processing, especially in tokenizations. In this article, I describe an algorithm that simplifies calculating correspondence between tokens (e.g. BERT vs. spaCy), one such process. And I introduce Python and Rust libraries that implement this algorithm. Here are the library and the demo site links:

@alexalouit
alexalouit / goofys.txt
Last active December 8, 2021 00:46
scaleway s3
# be careful, goofys does not support fsync, software performing check will return an error
$ wget https://github.com/kahing/goofys/releases/latest/download/goofys
$ chmod +x goofys
$ mv goofys /usr/bin/goofys
$ mkdir /root/.aws
$ echo -e "[default]\naws_access_key_id = $accesskey\naws_secret_access_key = $secretkey" > /root/.aws/credentials
$ echo 'goofys#bucket /mnt/bucket-name fuse _netdev,allow_other,--endpoint=https://s3.fr-par.scw.cloud/,--file-mode=0755,--dir-mode=0755,--uid=65534,--gid=65534 0 0' >> /etc/fstab