Skip to content

Instantly share code, notes, and snippets.

Avatar
🚀
Lightspeed

Roger Qiu CMCDragonkai

🚀
Lightspeed
View GitHub Profile
@CMCDragonkai
CMCDragonkai / machine.js
Last active Oct 21, 2020
Backup Restore Machine
View machine.js
const canBackup = (context, event) => {
return !context.backupLock
};
const canRestore = (context, event) => {
return !context.restoreLock
};
const backupMachine = Machine(
{
@CMCDragonkai
CMCDragonkai / sqlparams.py
Last active Sep 23, 2020
Dynamic Binding for PostgreSQL with AsyncPG #python #postgresql #asyncpg
View sqlparams.py
from collections import OrderdDict
from typings import Hashable, List
class SQLParams:
def __init__(self):
self.__values = OrderedDict()
self.__count = 1
@CMCDragonkai
CMCDragonkai / labelencoder_onehotencoder.py
Created Jul 7, 2020
LabelEncoder and OneHotEncoder #python #sklearn
View labelencoder_onehotencoder.py
import numpy as np
import pandas as pd
from sklearn.preprocessing import LabelEncoder, OneHotEncoder
classes_df = pd.DataFrame({
"class_id": ['n01669191', 'n01812337', 'n02007558', 'n02871439', 'n04306847',
'n10226413', 'n10267311', 'n12360108', 'n12662772', 'n13918274']
})
@CMCDragonkai
CMCDragonkai / batch_size_image_size.md
Created Jul 3, 2020
Batch size and Image size optimisation for CNNs
View batch_size_image_size.md

Batch size and Image size optimisation for CNNs

Given an initial size of an image. If we drop the square image size by side_p percentage points.

How much does the area percentage drop by?

The formula is as below:

area_p = [side_p * x^2 * (2 - side_p)] / 100
@CMCDragonkai
CMCDragonkai / roc_curves.md
Last active Jul 2, 2020
ROC Curves #python
View roc_curves.md

ROC Curves

Receiver Operating Characteristic curves are useful in measuring the effectiveness of a classifier.

To calculate a ROC curve, you first need to prepare dataset and pass it through the classifier.

You will have these 2 vectors:

  1. truth vector with values of 0 (negative-class) and 1 (positive-class).
  2. score vector with values with "smaller" or "negative" values pointing to the negative-class and "larger" or "positive" values pointing to the positive-class
@CMCDragonkai
CMCDragonkai / load_image_array.py
Created Jun 24, 2020
Load RGB image into numpy array from a file stream #python
View load_image_array.py
import io
import numpy as np
from PIL import Image
from typing import IO
def load_image(image_f: IO[bytes]) -> np.ndarray:
peek = image_f.peek(6)
if image_f.closed:
# if peek caused the underlying stream to be closed
@CMCDragonkai
CMCDragonkai / pycoco.py
Created Jun 15, 2020
Reading images from COCO #python
View pycoco.py
import zipfile
import numpy as np
from PIL import Image
from pycocotools import coco
# coco releases annotations and images in zip files
# you can keep the images in the zip archives because they allow random access
# however the annotations should be extracted and processed as is
c = coco.COCO("./annotations/instances_train2017.json")
@CMCDragonkai
CMCDragonkai / acquiring_nix_hash_large_file.md
Created Jun 11, 2020
Acquiring the Nix hash of a large file #nix
View acquiring_nix_hash_large_file.md

Acquiring the Nix hash of a large file

Normally you could use nix-prefetch-url. However it doesn't work when the files don't fit into memory.

For those cases, you should use:

wget https://url-to-large-file.com/file.zip
nix-hash --flat --base32 --type sha256 ./file.zip
@CMCDragonkai
CMCDragonkai / shannon_entropy.py
Last active Jun 10, 2020
Shannon Entropy implemented in Python #python
View shannon_entropy.py
import numpy as np
# these functions reify shannon information and shannon entropy
# the results are in units of "bits" because we are using log with base 2
# prob has to have the [0.0, 1.0] range
# probs should be the array of all probabilities of all possible outcomes of a random variable
# it is assumed that the random variable is discrete
You can’t perform that action at this time.