ohjho

PIP

installing from Git Repo

in general the command is as follows but see this blog post for the most detailed explanation:

pip install git+https://github.com/user/repositor.git@branch#subdirectory=src

• note that the @branch is completely optional
• also #subdirectory=src is also optional (sometimes the project to install is not in the root of the repo, e.g. Streamlit)
• either options like --ignore-requires-python might be useful if you think the python version requirement is not neccessary

ohjho

Replace a color with transparency

for example, to convert white to transparent

convert target.png -transparent white result.png

if not perfectly white:

convert target.png -fuzz {}% -transparent white result.png

where the smaller the fuzz %, the closer to true white or conversely, the larger the %, the more variation from white is allowed to become transparent.

ohjho

Tips and Tricks for using CUDA

nvidia-smi is very useful, in fact you can hack together a little nvtop with just watch -n 1 "nvidia-smi". Note, however, that the CUDA version in the top right corner of the output is the highest CUDA version supported, NOT installed.

You get also get your GPU's name with nvidia-smi --query-gpu=gpu_name --format=csv (credit to this interesting forumn thread)

And to get your CUDA driver version use nvcc --version

Docker resources

nvidia publishs a lot of linux docker images with different versions of CUDA installed on dockerhub

ohjho

Find Differences in two Text Files

grep -Fxvf file1 file2

Flags mean:

-F, --fixed-strings
              Interpret PATTERN as a list of fixed strings, separated by newlines, any of which is to be matched.    
-x, --line-regexp

ohjho

basics

• check that docker is working: docker run hello-world
• prune images: docker image prune -a
• prune containers: docker container prune
• check nvidia-docker2 is working (with CUDA 10.0 in base): docker run --rm --gpus all nvidia/cuda:10.0-runtime nvidia-smi
• build with Dockerfile: docker build -t your_image_name .
• run built image as container: docker run --name your_container_name your_image_name (for extra options see here)
• kill running container: docker kill your_container_name

ohjho

Installation

MacOS

Get FFMPEG with all the options which is a little more complicated in brew because the Homebrew team removed all options from the core formula:

brew update
brew uninstall --force --ignore-dependencies ffmpeg
brew install chromaprint amiaopensource/amiaos/decklinksdk
brew tap homebrew-ffmpeg/ffmpeg
brew upgrade homebrew-ffmpeg/ffmpeg/ffmpeg $(brew options homebrew-ffmpeg/ffmpeg/ffmpeg | grep -vE '\s' | grep -- '--with-' | grep -vi chromaprint | tr '\n' ' ')

ohjho

# to remove the file on git without deleting from local
#  use -> git rm --cached 
find . -name .ipynb_checkpoints -print0 | xargs -0 git rm -r --ignore-unmatch
echo .ipynb_checkpoints >> .gitignore

ohjho

Building PyTorch from Master/ Source using VirtualEnv

with reference to this

make sure that you have cloned the PyTorch repo. Let's assume that we'll install into ~/git/

$ cd ~/git
$ git clone --recursive https://github.com/pytorch/pytorch.git

activate your Python3 virtualenv and install PyTorch requirements

ohjho

Google Drive's Image URL

Use this tool which is from the site that wrote these simple Google Drive hacks you should know

Google Photo's Image URL

see this solution here

Other Tools & Packages

• youtube-drive: Store files as YouTube videos == infinite disk space.
• gdown: download large files from Google Drive

ohjho

Installing TorchServe

Machine Type

GCP, Ubuntu instance (Canonical, Ubuntu, 16.04 LTS, amd64 xenial image built on 2020-06-10, supports Shielded VM features)

To get this example to actually work, I followed the official documentation, this blog post by AWS, and this YouTube demo

1. Install Java 11

sudo add-apt-repository ppa:openjdk-r/ppa
sudo apt-get update