Skip to content

Instantly share code, notes, and snippets.

@dmitrysarov
dmitrysarov / batch_norm_test.ipynb
Last active December 26, 2018 13:27
How Keras BatchNorm should be set for transfer learning.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@dmitrysarov
dmitrysarov / fold_split_equally.py
Created January 9, 2019 09:59
It could help when your data set not balanced over one of parameters, but you have to split it over equally sized folds
#iteratively fill fold with balancing probobilities of next item respectivly it size
#df - dataframe of subjects CT slices, different subjects can have different number of slices
#folds splits can't have same subject in it
value_count = df['subject'].value_counts() #count how much each subject have slices
number_of_folds = 5
np.random.seed(42)
for num, chunk in enumerate(np.array_split(value_count, range(number_of_folds, len(value_count), number_of_folds))):
# chunk - is sorted by count
if num == 0:
#initialize folds
@dmitrysarov
dmitrysarov / gray2hot.py
Last active January 11, 2019 09:31
These lines can help some one to transform grayscale mask to multichannel (equal to number of classes) one hot mask.
def gray2hot(graymask, num_classes=2, include_background=True):
'''Convert grayscale image of mask to multichannel mask
if include_background=True number of channels in output will be
num_classes + 1
'''
graymask = graymask.astype(np.int32)
onehot_mask = np.zeros((graymask.shape[0], graymask.shape[1], num_classes+include_background), dtype=np.int32) # +1 relate to background
if include_background:
onehot_mask[:, :, 0] = 1 # initialize all as background at first
if np.all(graymask == 0):
@dmitrysarov
dmitrysarov / matplotlib_line_styles.py
Last active February 20, 2019 08:00
bunch of line styles for matplotlib
from itertools import combinations
styles = map(lambda x: x+'-', list('.^o<>1234sp*hH+xDd|_'))
for num, line_style in enumerate(styles, 1):
plt.plot(range(10), [num]*10, line_style, label=line_style)
plt.legend(loc='upper left', bbox_to_anchor=(1, 1))
plt.show()
@dmitrysarov
dmitrysarov / python_widgets_GUI.py
Last active April 10, 2019 22:11
first experience with widgets
import ipywidgets as widgets
from IPython.display import display, clear_output
from fxd import FxdInputFile
import matplotlib.pyplot as plt
import matplotlib.animation as animation
from PIL import Image
import pickle
import numpy as np
from copy import deepcopy
%config InlineBackend.close_figures=False
@dmitrysarov
dmitrysarov / snakeviz_usage
Last active May 31, 2019 14:24
snakeviz usage snipet
snakeviz - tool helping to visualize data of after you script profile.
to install: python3 -m pip install snakeviz
to profile create profile log file do: python3 -m cProfile -o profile.log you_script.py
to run snakeviz (not in browser but run server): python3 -m snakeviz -s -H 0.0.0.0 -p 9022 profile.log
from multiprocessing import Process, Queue
from time import sleep
def loader(init_q, cache_q):
while not init_q.empty():
if cache_q.full():
continue
cache_q.put(init_q.get())
def predicter(init_q, cache_q):
while True:
first cosulution found was https://askubuntu.com/questions/1032633/18-04-screen-remains-blank-after-wake-up-from-suspend
but it switch off second screen
second i decide to install nvidia driver, and switch off Nouveau driver (open source nvidia river) http://us.download.nvidia.com/XFree86/Linux-x86_64/378.13/README/commonproblems.html#nouveau
https://medium.com/@antonioszeto/how-to-install-nvidia-driver-on-ubuntu-18-04-7b464bab43e6
@dmitrysarov
dmitrysarov / image_analiser.py
Created October 3, 2019 12:06
Simple tool implemented in jupyter for image dynamic range and frequency analysis.
# %matplotlib inline
%matplotlib notebook
from ipywidgets import interact, widgets, Layout
import matplotlib.pyplot as plt
from matplotlib.image import AxesImage
from IPython.display import display
from numpy.fft import fftshift, fft2, ifft2, ifftshift
import time
from skimage import transform
@dmitrysarov
dmitrysarov / iterative_hist.py
Last active October 7, 2019 13:53
If data is too large, this class can iteratively absorb data
import numpy as np
import matplotlib.pyplot as plt
class Iterative_hist(object):
'''
Collect distribution information in iterative manner.
Useful when whole data not fit memory.
Usage:
if min and max data value is not known, call set_min_max() method, iterative, on hall dataset
call add_data() method, for data addition