Skip to content

Instantly share code, notes, and snippets.

View hsahovic's full-sized avatar

Haris Sahovic hsahovic

View GitHub Profile
@hsahovic
hsahovic / tf_dataset_cutmix.py
Created March 19, 2020 02:30
This function takes a tensorflow dataset and returns a corresponding dataset implementing cutmix
def tf_ds_cutmix(ds, shuffling=1024):
ds_shuffled = ds.shuffle(shuffling)
def cutmix(p1, p2):
img_1, label_1 = p1
img_2, label_2 = p2
lambda_ = tf.random.uniform((1,))
@hsahovic
hsahovic / ohem_crossentropy_loss_tensorflow.py
Created March 18, 2020 23:53
A custom tensorflow / keras loss implementing OHEM (https://arxiv.org/abs/1604.03540) with cross-entropy.
import tensorflow as tf
from tensorflow.keras.losses import categorical_crossentropy
@tf.function
def ohem_crossentropy_loss(y_true, y_pred):
# You can apply OHEM with any loss you want
# To do so, you just need to change the base loss below
cross_entropy = categorical_crossentropy(y_true, y_pred)
# You can tune how many examples are rejected by modifying the `80` value below
@hsahovic
hsahovic / Keras applications cafee image preprocessing.js
Created December 8, 2019 23:31
This gist reproduces the cafee (which is default) image preprocessing used in keras.applications (up to a rescaling factor) with tensorflow.js.
let img = await webcam.capture();
let rChannel = img.slice([0,0,0],[192,192,1]).reshape([192,192]);
let gChannel = img.slice([0,0,1],[192,192,1]).reshape([192,192]);
let bChannel = img.slice([0,0,2],[192,192,1]).reshape([192,192]);
img = tf.stack([bChannel, gChannel, rChannel], 2);
img = tf.mul(img, 1/255);
img = tf.add(img, -.5);
@hsahovic
hsahovic / trained_to_sequential.py
Created December 8, 2019 09:37
This snippet was used to recover a proper keras model from a saved model which contained a submodel (ie., one of its layers was actually another model), in order to apply model optimization a posteriori (quantization, pruning). It can be extended to handle more type of layers.
from tensorflow.keras.models import Sequential
import tensorflow.keras.layers as keras_layers
clone = Sequential()
# Here, layer[1] is a model.
layers = [model.layers[0]] + model.layers[1].layers + model.layers[2:]
for layer in layers:
if str(type(layer)).endswith("InputLayer'>"):
@hsahovic
hsahovic / split_train_test_directories.py
Last active December 9, 2019 02:52
Split files from a directory into two directories with the same structure.
import os
from tqdm.notebook import tqdm
from shutil import copyfile
def split_dir(source_dir, target_dir_train, target_dir_test, ratio=.2):
files = os.listdir(source_dir)
to_test = int(len(files) * ratio + .5)
assert files[:to_test] + files[to_test:] == files
for file in files[:to_test]:
@hsahovic
hsahovic / image_generator_to_tf_ds.py
Created December 1, 2019 02:53
Converting a Keras image generator to a Tensorflow Dataset
from typing import Generator
import tensorflow as tf
def image_generator_to_tf_ds(generator: Generator) -> tf.data.Dataset:
"""Converts a initialized keras Image Data Generator to an equivalent tf Dataset.
Example usage:
>>> img_generator = ImageDataGenerator()
>>> img_generator = img_generator.flow_from_directory(