Skip to content

Instantly share code, notes, and snippets.

View Luonic's full-sized avatar
:shipit:

Alexandr Kislinskiy Luonic

:shipit:
22 followers · 4 following
View GitHub Profile
@Luonic
Luonic / ilm_norm.py
Created May 29, 2019 20:31
Instance-Level Meta Normalization for Tensorflow 1.1x [https://arxiv.org/abs/1904.03516]
import tensorflow as tf
def group_norm(x, G=32, eps=1e-5):
# x: input features with shape [N,C,H,W]
# G: number of groups for GN
N, H, W, C = x.shape
gamma = tf.get_variable('gamma', [1, 1, 1, C], initializer=tf.initializers.ones)
beta = tf.get_variable('B_beta', [1, 1, 1, C], initializer=tf.initializers.zeros)
x = tf.reshape(x, [N, H, W, G, C // G])
mean, var = tf.nn.moments(x, [1, 2, 4], keep_dims=True)
@Luonic
Luonic / group_norm.py
Created June 15, 2018 09:57
Group normalization for Tensorflow with adaptive groups count and data format 'channels_last'
def group_normalization(input_tensor, num_groups, gamma=1.0, beta=0.0, epsilon=1e-5):
channels_int = input_tensor.get_shape().as_list()[3]
while channels_int % num_groups != 0 and num_groups != 0:
num_groups -= 1
batch, height, width, channels = input_tensor.shape
input_tensor = tf.reshape(input_tensor, shape=(batch, height, width, channels // num_groups, num_groups))
mean, var = tf.nn.moments(input_tensor, [1, 2, 3], keep_dims=True)
input_tensor = (input_tensor - mean) / tf.sqrt(var + epsilon)
input_tensor = tf.reshape(input_tensor, [batch, height, width, channels])
@Luonic
Luonic / subpixel.py
Created June 21, 2017 21:09
import matplotlib.pyplot as plt
import tensorflow as tf
import numpy as np
def _phase_shift(I, r):
bsize, a, b, c = I.get_shape().as_list()
bsize = tf.shape(I)[0] # Handling Dimension(None) type for undefined batch dim
X = tf.reshape(I, (bsize, a, b, r, r))
X = tf.transpose(X, (0, 1, 2, 4, 3)) # bsize, a, b, 1, 1
@Luonic
Luonic / img_ocr_keras.py
Created February 14, 2017 22:51
'''This example uses a convolutional stack followed by a recurrent stack
and a CTC logloss function to perform optical character recognition
of generated text images. I have no evidence of whether it actually
learns general shapes of text, or just is able to recognize all
the different fonts thrown at it...the purpose is more to demonstrate CTC
inside of Keras. Note that the font list may need to be updated
for the particular OS in use.
This starts off with 4 letter words. For the first 12 epochs, the
difficulty is gradually increased using the TextImageGenerator class
@Luonic
Luonic / caffe_log_2.txt
Created December 20, 2016 15:02
I1220 16:01:43.502776 3961783232 solver.cpp:228] Iteration 0, loss = 5.0178
I1220 16:01:43.502820 3961783232 solver.cpp:244] Train net output #0: loss = 5.0178 (* 1 = 5.0178 loss)
I1220 16:01:43.502847 3961783232 sgd_solver.cpp:106] Iteration 0, lr = 0.001
I1220 16:02:51.024165 3961783232 solver.cpp:228] Iteration 100, loss = 1.87306
I1220 16:02:51.063248 3961783232 solver.cpp:244] Train net output #0: loss = 1.87306 (* 1 = 1.87306 loss)
I1220 16:02:51.063267 3961783232 sgd_solver.cpp:106] Iteration 100, lr = 0.001
I1220 16:03:58.084491 3961783232 solver.cpp:228] Iteration 200, loss = 1.01183
I1220 16:03:58.084777 3961783232 solver.cpp:244] Train net output #0: loss = 1.01183 (* 1 = 1.01183 loss)
I1220 16:03:58.084800 3961783232 sgd_solver.cpp:106] Iteration 200, lr = 0.001
I1220 16:05:05.282796 3961783232 solver.cpp:228] Iteration 300, loss = 0.772216
@Luonic
Luonic / caffe_log.txt
Last active March 20, 2017 14:32
I0320 14:39:22.460748 8660 solver.cpp:219] Iteration 0 (-6.57593e+33 iter/s, 0.322088s/250 iters), loss = 0.901091
I0320 14:39:22.460778 8660 solver.cpp:238] Train net output #0: loss1 = 0.693134 (* 0.3 = 0.20794 loss)
I0320 14:39:22.460798 8660 solver.cpp:238] Train net output #1: loss2 = 0.693151 (* 1 = 0.693151 loss)
I0320 14:39:22.460814 8660 sgd_solver.cpp:105] Iteration 0, lr = 0.0001
I0320 14:40:56.452066 8660 solver.cpp:219] Iteration 250 (2.65983 iter/s, 93.9909s/250 iters), loss = 0.133513
I0320 14:40:56.452141 8660 solver.cpp:238] Train net output #0: loss1 = 0.119867 (* 0.3 = 0.0359602 loss)
I0320 14:40:56.452162 8660 solver.cpp:238] Train net output #1: loss2 = 0.0975531 (* 1 = 0.0975531 loss)
I0320 14:40:56.452167 8660 sgd_solver.cpp:105] Iteration 250, lr = 0.0001
I0320 14:42:58.538004 8660 solver.cpp:448] Snapshotting to binary proto file snapshots/sale_iter_500.caffemodel
I0320 14:42:58.732300 8660 sgd_solver.cpp:273] Snapshotting solver state to binary proto file sn
@Luonic
Luonic / log_12312.txt
Created November 13, 2016 14:48
I1113 13:59:14.466996 15355 solver.cpp:337] Iteration 0, Testing net (#0)
I1113 13:59:14.481719 15355 net.cpp:693] Ignoring source layer prob
I1113 13:59:44.028913 15355 solver.cpp:404] Test net output #0: accuracy = 0.013875
I1113 13:59:44.168041 15355 solver.cpp:228] Iteration 0, loss = 4.52858
I1113 13:59:44.168081 15355 solver.cpp:244] Train net output #0: loss = 4.52858 (* 1 = 4.52858 loss)
I1113 13:59:44.168093 15355 sgd_solver.cpp:106] Iteration 0, lr = 0.001
I1113 14:03:18.589840 15355 solver.cpp:228] Iteration 1000, loss = 0.235305
I1113 14:03:18.589912 15355 solver.cpp:244] Train net output #0: loss = 0.235305 (* 1 = 0.235305 loss)
I1113 14:03:18.589920 15355 sgd_solver.cpp:106] Iteration 1000, lr = 0.001
I1113 14:06:52.377185 15355 solver.cpp:228] Iteration 2000, loss = 0.0354668
@Luonic
Luonic / orig_net_log.txt
Created November 12, 2016 21:12
I1112 18:26:20.476073 22658 solver.cpp:337] Iteration 0, Testing net (#0)
I1112 18:26:20.486620 22658 net.cpp:693] Ignoring source layer prob
I1112 18:26:22.844864 22658 solver.cpp:404] Test net output #0: accuracy = 0.02416
I1112 18:26:22.862859 22658 solver.cpp:228] Iteration 0, loss = 38.3147
I1112 18:26:22.862937 22658 solver.cpp:244] Train net output #0: loss = 38.3147 (* 1 = 38.3147 loss)
I1112 18:26:22.862980 22658 sgd_solver.cpp:106] Iteration 0, lr = 0.01
I1112 18:27:25.690100 22658 solver.cpp:228] Iteration 1000, loss = 4.31486
I1112 18:27:25.690229 22658 solver.cpp:244] Train net output #0: loss = 4.31486 (* 1 = 4.31486 loss)
I1112 18:27:25.690245 22658 sgd_solver.cpp:106] Iteration 1000, lr = 0.01
I1112 18:28:28.546875 22658 solver.cpp:228] Iteration 2000, loss = 4.29191
@Luonic
Luonic / my_net_log.txt
Last active November 12, 2016 21:12
I1112 07:17:41.890250 17193 solver.cpp:337] Iteration 0, Testing net (#0)
I1112 07:17:41.896697 17193 net.cpp:693] Ignoring source layer drop1
I1112 07:17:41.901717 17193 net.cpp:693] Ignoring source layer prob
I1112 07:17:49.161095 17193 solver.cpp:404] Test net output #0: accuracy = 0.02056
I1112 07:17:49.283001 17193 solver.cpp:228] Iteration 0, loss = 5.03651
I1112 07:17:49.283051 17193 solver.cpp:244] Train net output #0: loss = 5.03651 (* 1 = 5.03651 loss)
I1112 07:17:49.283078 17193 sgd_solver.cpp:106] Iteration 0, lr = 0.01
I1112 07:19:59.706967 17193 solver.cpp:228] Iteration 1000, loss = 1.17764
I1112 07:19:59.707039 17193 solver.cpp:244] Train net output #0: loss = 1.17765 (* 1 = 1.17765 loss)
I1112 07:19:59.707053 17193 sgd_solver.cpp:106] Iteration 1000, lr = 0.000187328