Skip to content

Instantly share code, notes, and snippets.

View RaphaelMeudec's full-sized avatar

Raphael Meudec RaphaelMeudec

View GitHub Profile
@RaphaelMeudec
RaphaelMeudec / tensorflow_dataset_for_images.py
Last active October 25, 2020 10:12
Create a simple tf.data Dataset for an image deblurring task
from pathlib import Path
import tensorflow as tf
def select_patch(sharp, blur, patch_size_x, patch_size_y):
"""
Select a patch on both sharp and blur images at the same localization.
Args:
sharp (tf.Tensor): Tensor for the sharp image
@RaphaelMeudec
RaphaelMeudec / optimized_tensorflow_dataset_for_image.py
Created January 16, 2020 11:17
Create an optimized version of tf.data Dataset for an image deblurring task
from pathlib import Path
import tensorflow as tf
def select_patch(sharp, blur, patch_size_x, patch_size_y):
"""
Select a patch on both sharp and blur images at the same localization.
Args:
sharp (tf.Tensor): Tensor for the sharp image
blur (tf.Tensor): Tensor for the blur image
# Define multi-gpu strategy
mirrored_strategy = tf.distribute.MirroredStrategy()
# Update batch size value
batch_size *= mirrored_strategy.num_replicas_in_sync
# Create strategy scope to perform training
with mirrored_strategy.scope():
model = [...]
model.fit(...)
@RaphaelMeudec
RaphaelMeudec / mixed_preciision.py
Created January 16, 2020 16:07
Lines to turn float32 into mixed precision training
policy = tf.keras.mixed_precision.experimental.Policy('mixed_float16')
tf.keras.mixed_precision.experimental.set_policy(policy)
@RaphaelMeudec
RaphaelMeudec / training_performance_inputs_type.py
Last active April 21, 2020 16:37
Comparison of training time based on what you feed to the .fit method (np.array, ImageDataGenerator with different arguments, tf.data)
"""
Performance comparison between ImageDataGenerator and tf.data.Dataset on a simple task
Dependencies available at https://www.github.com/sicara/tf2-evonorm
"""
import math
from pathlib import Path
import click
import tensorflow as tf
@RaphaelMeudec
RaphaelMeudec / renice_python_processes.py
Last active April 13, 2021 09:16
Python script to automatically renice all python processes
#!/usr/bin/python3
from datetime import datetime
import psutil
DEFAULT_DRAGO_NICENESS_VALUE = 5
DURATION_BEFORE_RENICING = 30