Skip to content

Instantly share code, notes, and snippets.

View kretes's full-sized avatar

Tomasz Bartczak kretes

32 followers · 5 following
View GitHub Profile
@kretes
kretes / pyradiomics.ipynb
Created October 23, 2023 21:03
pyradiomics.ipynb
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@kretes
kretes / pandas_dataframes_diff_dolt.sh
Last active May 4, 2022 17:37
Commands needed to run to get a dolt-diff of two csvs
dolt init
dolt schema import --create --pks <primary_key_column> <table_name> original_data.csv
dolt table import -u <table_name> original_data.csv
dolt commit -am "initial version"
dolt table import -u <table_name> changed_data.csv
dolt diff
@kretes
kretes / multi_gpu_custom_train_step_model.py
Created October 13, 2021 12:00
A modified version of Keras model that allows for a different reduction of the loss values
# Copyright 2015 The TensorFlow Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
@kretes
kretes / Dockerfile
Created October 6, 2021 16:32
Poetry trying to install tf 2.0.2
FROM ubuntu:20.04
# Set working dir
RUN mkdir -p /work
WORKDIR /work
ARG PYTHON_VERSION=3.9
ENV DEBIAN_FRONTEND=noninteractive
RUN apt update \
@kretes
kretes / ModelWeightsPrinter.py
Last active September 6, 2021 06:55
tensorflow callback that prints some basic statistics about weights
class ModelWeightsPrinter(Callback):
def __init__(self, model) -> None:
super().__init__()
self.model = model
def print_stats(self, hist):
if hist:
allw = np.hstack([x.flatten() for x in self.model.get_weights()])
h = np.histogram(allw, bins=np.linspace(-1, 1, 5))
print("weights_histogram")
@kretes
kretes / tf_47032.py
Created May 21, 2021 12:57
tensorflow 47032 issue yet another reproduce https://github.com/tensorflow/tensorflow/issues/47032
import tensorflow as tf
from tensorflow import keras
import numpy as np
N = 6
CLASSES = 3
FEATURES = 4
model = keras.Sequential([
keras.layers.Dense(10, input_shape=(FEATURES,)),
@kretes
kretes / tf_measure.py
Created May 12, 2021 16:25
function wrapper to measure execution time in tf graph
def measure_time_tf(fun, name):
def time_measuring(*args):
start = tf.timestamp()
result = fun(*args)
end = tf.timestamp()
diff_times100 = 100.0 * (end - start)
tf.print(name, tf.cast(tf.cast(diff_times100, tf.int32), tf.float32) / 100.0)
return result
return time_measuring
@kretes
kretes / print_tf_ds.py
Created May 10, 2021 08:07
A short utility to apply over a tf.data.Dataset to display what is it processing at given stage
def print_ds(ds, what):
def print_elems(*args):
tf.print(f"len of {what}", len(args), *[tf.shape(a) for a in args], args[-1])
return tuple(args)
return ds.map(print_elems)
ds = print_ds(ds, "1")
@kretes
kretes / explain_structure_tf.py
Created April 16, 2021 13:47
Explains structure with collections/numpy/tensorflow tensors in it
import tensorflow as tf
def explain_structure(obj, how_many=3):
if hasattr(obj, 'shape'):
repr = f"{type(obj)}, {obj.shape}"
if type(obj) == tf.Tensor:
repr = f"{repr} {obj.device}"
return repr
elif type(obj) in [type([]), type(())]:
inner = obj
@kretes
kretes / ipython.out
Last active March 1, 2021 07:32
In [7]: import numpy as np
...: import tensorflow as tf
...: print(tf.version.GIT_VERSION, tf.version.VERSION)
...:
...: dim = 300
...:
...: def return_single(*args):
...: x = np.random.rand(dim,dim,dim)
...: return x,1
...: