CMCDragonkai

const canBackup = (context, event) => {
  return !context.backupLock
};

const canRestore = (context, event) => {
  return !context.restoreLock
};

const backupMachine = Machine(
  {

CMCDragonkai

See: http://charlesnagy.info/it/postgresql/group-by-limit-per-group-in-postgresql

CMCDragonkai

from collections import OrderdDict
from typings import Hashable, List


class SQLParams:

    def __init__(self):
        self.__values = OrderedDict()
        self.__count = 1

CMCDragonkai

import numpy as np
import pandas as pd
from sklearn.preprocessing import LabelEncoder, OneHotEncoder


classes_df = pd.DataFrame({
    "class_id": ['n01669191', 'n01812337', 'n02007558', 'n02871439', 'n04306847',
                 'n10226413', 'n10267311', 'n12360108', 'n12662772', 'n13918274']
})

CMCDragonkai

Batch size and Image size optimisation for CNNs

Given an initial size of an image. If we drop the square image size by side_p percentage points.

How much does the area percentage drop by?

The formula is as below:

area_p = [side_p * x^2 * (2 - side_p)] / 100

CMCDragonkai

ROC Curves

Receiver Operating Characteristic curves are useful in measuring the effectiveness of a classifier.

To calculate a ROC curve, you first need to prepare dataset and pass it through the classifier.

You will have these 2 vectors:

1. truth vector with values of 0 (negative-class) and 1 (positive-class).
2. score vector with values with "smaller" or "negative" values pointing to the negative-class and "larger" or "positive" values pointing to the positive-class

CMCDragonkai

import io
import numpy as np
from PIL import Image
from typing import IO


def load_image(image_f: IO[bytes]) -> np.ndarray:
    peek = image_f.peek(6)
    if image_f.closed:
        # if peek caused the underlying stream to be closed

CMCDragonkai

import zipfile
import numpy as np
from PIL import Image
from pycocotools import coco

# coco releases annotations and images in zip files
# you can keep the images in the zip archives because they allow random access
# however the annotations should be extracted and processed as is

c = coco.COCO("./annotations/instances_train2017.json")

CMCDragonkai

Acquiring the Nix hash of a large file

Normally you could use nix-prefetch-url. However it doesn't work when the files don't fit into memory.

For those cases, you should use:

wget https://url-to-large-file.com/file.zip
nix-hash --flat --base32 --type sha256 ./file.zip

CMCDragonkai

import numpy as np


# these functions reify shannon information and shannon entropy
# the results are in units of "bits" because we are using log with base 2
# prob has to have the [0.0, 1.0] range
# probs should be the array of all probabilities of all possible outcomes of a random variable
# it is assumed that the random variable is discrete