Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
GoogLeNet in Keras

GoogLeNet in Keras

Here is a Keras model of GoogLeNet (a.k.a Inception V1). I created it by converting the GoogLeNet model from Caffe. Note: this model currently only runs with the Theano backend for Keras. I hope to get the model working with the TensorFlow backend in the future.

GoogLeNet paper:

Going deeper with convolutions.
Szegedy, Christian, et al. 
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2015.

Requirements

The code now runs with Python 3.6, Keras 2.2.4, and Theano 1.0.4. You will also need to install the following:

pip install pillow numpy imageio

To switch to the Theano backend, change your ~/.keras/keras.json file to

{"epsilon": 1e-07, "floatx": "float32", "backend": "theano", "image_data_format": "channels_first"}

Note that Theano requires the channels_first option for image_data_format.

Running the Demo (googlenet.py)

To create a GoogLeNet model, call the following from within Python:

from googlenet import create_googlenet
model = create_googlenet()

googlenet.py also contains a demo image classification. To run the demo, you will need to install the pre-trained weights and the class labels. You will also need a test image. Once these are downloaded and moved to the working directory, you can run googlenet.py from the terminal:

$ python googlenet.py

which will output the predicted class label for the image.

from __future__ import print_function
import imageio
from PIL import Image
import numpy as np
from keras.layers import Input, Dense, Conv2D, MaxPooling2D, AveragePooling2D, ZeroPadding2D, Dropout, Flatten, Concatenate, Reshape, Activation
from keras.models import Model
from keras.regularizers import l2
from keras.optimizers import SGD
from googlenet_custom_layers import PoolHelper,LRN
def create_googlenet(weights_path=None):
# creates GoogLeNet a.k.a. Inception v1 (Szegedy, 2015)
input = Input(shape=(3, 224, 224))
conv1_7x7_s2 = Conv2D(64, (7,7), strides=(2,2), padding='same', activation='relu', name='conv1/7x7_s2', kernel_regularizer=l2(0.0002))(input)
conv1_zero_pad = ZeroPadding2D(padding=(1, 1))(conv1_7x7_s2)
pool1_helper = PoolHelper()(conv1_zero_pad)
pool1_3x3_s2 = MaxPooling2D(pool_size=(3,3), strides=(2,2), padding='valid', name='pool1/3x3_s2')(pool1_helper)
pool1_norm1 = LRN(name='pool1/norm1')(pool1_3x3_s2)
conv2_3x3_reduce = Conv2D(64, (1,1), padding='same', activation='relu', name='conv2/3x3_reduce', kernel_regularizer=l2(0.0002))(pool1_norm1)
conv2_3x3 = Conv2D(192, (3,3), padding='same', activation='relu', name='conv2/3x3', kernel_regularizer=l2(0.0002))(conv2_3x3_reduce)
conv2_norm2 = LRN(name='conv2/norm2')(conv2_3x3)
conv2_zero_pad = ZeroPadding2D(padding=(1, 1))(conv2_norm2)
pool2_helper = PoolHelper()(conv2_zero_pad)
pool2_3x3_s2 = MaxPooling2D(pool_size=(3,3), strides=(2,2), padding='valid', name='pool2/3x3_s2')(pool2_helper)
inception_3a_1x1 = Conv2D(64, (1,1), padding='same', activation='relu', name='inception_3a/1x1', kernel_regularizer=l2(0.0002))(pool2_3x3_s2)
inception_3a_3x3_reduce = Conv2D(96, (1,1), padding='same', activation='relu', name='inception_3a/3x3_reduce', kernel_regularizer=l2(0.0002))(pool2_3x3_s2)
inception_3a_3x3 = Conv2D(128, (3,3), padding='same', activation='relu', name='inception_3a/3x3', kernel_regularizer=l2(0.0002))(inception_3a_3x3_reduce)
inception_3a_5x5_reduce = Conv2D(16, (1,1), padding='same', activation='relu', name='inception_3a/5x5_reduce', kernel_regularizer=l2(0.0002))(pool2_3x3_s2)
inception_3a_5x5 = Conv2D(32, (5,5), padding='same', activation='relu', name='inception_3a/5x5', kernel_regularizer=l2(0.0002))(inception_3a_5x5_reduce)
inception_3a_pool = MaxPooling2D(pool_size=(3,3), strides=(1,1), padding='same', name='inception_3a/pool')(pool2_3x3_s2)
inception_3a_pool_proj = Conv2D(32, (1,1), padding='same', activation='relu', name='inception_3a/pool_proj', kernel_regularizer=l2(0.0002))(inception_3a_pool)
inception_3a_output = Concatenate(axis=1, name='inception_3a/output')([inception_3a_1x1,inception_3a_3x3,inception_3a_5x5,inception_3a_pool_proj])
inception_3b_1x1 = Conv2D(128, (1,1), padding='same', activation='relu', name='inception_3b/1x1', kernel_regularizer=l2(0.0002))(inception_3a_output)
inception_3b_3x3_reduce = Conv2D(128, (1,1), padding='same', activation='relu', name='inception_3b/3x3_reduce', kernel_regularizer=l2(0.0002))(inception_3a_output)
inception_3b_3x3 = Conv2D(192, (3,3), padding='same', activation='relu', name='inception_3b/3x3', kernel_regularizer=l2(0.0002))(inception_3b_3x3_reduce)
inception_3b_5x5_reduce = Conv2D(32, (1,1), padding='same', activation='relu', name='inception_3b/5x5_reduce', kernel_regularizer=l2(0.0002))(inception_3a_output)
inception_3b_5x5 = Conv2D(96, (5,5), padding='same', activation='relu', name='inception_3b/5x5', kernel_regularizer=l2(0.0002))(inception_3b_5x5_reduce)
inception_3b_pool = MaxPooling2D(pool_size=(3,3), strides=(1,1), padding='same', name='inception_3b/pool')(inception_3a_output)
inception_3b_pool_proj = Conv2D(64, (1,1), padding='same', activation='relu', name='inception_3b/pool_proj', kernel_regularizer=l2(0.0002))(inception_3b_pool)
inception_3b_output = Concatenate(axis=1, name='inception_3b/output')([inception_3b_1x1,inception_3b_3x3,inception_3b_5x5,inception_3b_pool_proj])
inception_3b_output_zero_pad = ZeroPadding2D(padding=(1, 1))(inception_3b_output)
pool3_helper = PoolHelper()(inception_3b_output_zero_pad)
pool3_3x3_s2 = MaxPooling2D(pool_size=(3,3), strides=(2,2), padding='valid', name='pool3/3x3_s2')(pool3_helper)
inception_4a_1x1 = Conv2D(192, (1,1), padding='same', activation='relu', name='inception_4a/1x1', kernel_regularizer=l2(0.0002))(pool3_3x3_s2)
inception_4a_3x3_reduce = Conv2D(96, (1,1), padding='same', activation='relu', name='inception_4a/3x3_reduce', kernel_regularizer=l2(0.0002))(pool3_3x3_s2)
inception_4a_3x3 = Conv2D(208, (3,3), padding='same', activation='relu', name='inception_4a/3x3' ,kernel_regularizer=l2(0.0002))(inception_4a_3x3_reduce)
inception_4a_5x5_reduce = Conv2D(16, (1,1), padding='same', activation='relu', name='inception_4a/5x5_reduce', kernel_regularizer=l2(0.0002))(pool3_3x3_s2)
inception_4a_5x5 = Conv2D(48, (5,5), padding='same', activation='relu', name='inception_4a/5x5', kernel_regularizer=l2(0.0002))(inception_4a_5x5_reduce)
inception_4a_pool = MaxPooling2D(pool_size=(3,3), strides=(1,1), padding='same', name='inception_4a/pool')(pool3_3x3_s2)
inception_4a_pool_proj = Conv2D(64, (1,1), padding='same', activation='relu', name='inception_4a/pool_proj', kernel_regularizer=l2(0.0002))(inception_4a_pool)
inception_4a_output = Concatenate(axis=1, name='inception_4a/output')([inception_4a_1x1,inception_4a_3x3,inception_4a_5x5,inception_4a_pool_proj])
loss1_ave_pool = AveragePooling2D(pool_size=(5,5), strides=(3,3), name='loss1/ave_pool')(inception_4a_output)
loss1_conv = Conv2D(128, (1,1), padding='same', activation='relu', name='loss1/conv', kernel_regularizer=l2(0.0002))(loss1_ave_pool)
loss1_flat = Flatten()(loss1_conv)
loss1_fc = Dense(1024, activation='relu', name='loss1/fc', kernel_regularizer=l2(0.0002))(loss1_flat)
loss1_drop_fc = Dropout(0.7)(loss1_fc)
loss1_classifier = Dense(1000, name='loss1/classifier', kernel_regularizer=l2(0.0002))(loss1_drop_fc)
loss1_classifier_act = Activation('softmax')(loss1_classifier)
inception_4b_1x1 = Conv2D(160, (1,1), padding='same', activation='relu', name='inception_4b/1x1', kernel_regularizer=l2(0.0002))(inception_4a_output)
inception_4b_3x3_reduce = Conv2D(112, (1,1), padding='same', activation='relu', name='inception_4b/3x3_reduce', kernel_regularizer=l2(0.0002))(inception_4a_output)
inception_4b_3x3 = Conv2D(224, (3,3), padding='same', activation='relu', name='inception_4b/3x3', kernel_regularizer=l2(0.0002))(inception_4b_3x3_reduce)
inception_4b_5x5_reduce = Conv2D(24, (1,1), padding='same', activation='relu', name='inception_4b/5x5_reduce', kernel_regularizer=l2(0.0002))(inception_4a_output)
inception_4b_5x5 = Conv2D(64, (5,5), padding='same', activation='relu', name='inception_4b/5x5', kernel_regularizer=l2(0.0002))(inception_4b_5x5_reduce)
inception_4b_pool = MaxPooling2D(pool_size=(3,3), strides=(1,1), padding='same', name='inception_4b/pool')(inception_4a_output)
inception_4b_pool_proj = Conv2D(64, (1,1), padding='same', activation='relu', name='inception_4b/pool_proj', kernel_regularizer=l2(0.0002))(inception_4b_pool)
inception_4b_output = Concatenate(axis=1, name='inception_4b/output')([inception_4b_1x1,inception_4b_3x3,inception_4b_5x5,inception_4b_pool_proj])
inception_4c_1x1 = Conv2D(128, (1,1), padding='same', activation='relu', name='inception_4c/1x1', kernel_regularizer=l2(0.0002))(inception_4b_output)
inception_4c_3x3_reduce = Conv2D(128, (1,1), padding='same', activation='relu', name='inception_4c/3x3_reduce', kernel_regularizer=l2(0.0002))(inception_4b_output)
inception_4c_3x3 = Conv2D(256, (3,3), padding='same', activation='relu', name='inception_4c/3x3', kernel_regularizer=l2(0.0002))(inception_4c_3x3_reduce)
inception_4c_5x5_reduce = Conv2D(24, (1,1), padding='same', activation='relu', name='inception_4c/5x5_reduce', kernel_regularizer=l2(0.0002))(inception_4b_output)
inception_4c_5x5 = Conv2D(64, (5,5), padding='same', activation='relu', name='inception_4c/5x5', kernel_regularizer=l2(0.0002))(inception_4c_5x5_reduce)
inception_4c_pool = MaxPooling2D(pool_size=(3,3), strides=(1,1), padding='same', name='inception_4c/pool')(inception_4b_output)
inception_4c_pool_proj = Conv2D(64, (1,1), padding='same', activation='relu', name='inception_4c/pool_proj', kernel_regularizer=l2(0.0002))(inception_4c_pool)
inception_4c_output = Concatenate(axis=1, name='inception_4c/output')([inception_4c_1x1,inception_4c_3x3,inception_4c_5x5,inception_4c_pool_proj])
inception_4d_1x1 = Conv2D(112, (1,1), padding='same', activation='relu', name='inception_4d/1x1', kernel_regularizer=l2(0.0002))(inception_4c_output)
inception_4d_3x3_reduce = Conv2D(144, (1,1), padding='same', activation='relu', name='inception_4d/3x3_reduce', kernel_regularizer=l2(0.0002))(inception_4c_output)
inception_4d_3x3 = Conv2D(288, (3,3), padding='same', activation='relu', name='inception_4d/3x3', kernel_regularizer=l2(0.0002))(inception_4d_3x3_reduce)
inception_4d_5x5_reduce = Conv2D(32, (1,1), padding='same', activation='relu', name='inception_4d/5x5_reduce', kernel_regularizer=l2(0.0002))(inception_4c_output)
inception_4d_5x5 = Conv2D(64, (5,5), padding='same', activation='relu', name='inception_4d/5x5', kernel_regularizer=l2(0.0002))(inception_4d_5x5_reduce)
inception_4d_pool = MaxPooling2D(pool_size=(3,3), strides=(1,1), padding='same', name='inception_4d/pool')(inception_4c_output)
inception_4d_pool_proj = Conv2D(64, (1,1), padding='same', activation='relu', name='inception_4d/pool_proj', kernel_regularizer=l2(0.0002))(inception_4d_pool)
inception_4d_output = Concatenate(axis=1, name='inception_4d/output')([inception_4d_1x1,inception_4d_3x3,inception_4d_5x5,inception_4d_pool_proj])
loss2_ave_pool = AveragePooling2D(pool_size=(5,5), strides=(3,3), name='loss2/ave_pool')(inception_4d_output)
loss2_conv = Conv2D(128, (1,1), padding='same', activation='relu', name='loss2/conv', kernel_regularizer=l2(0.0002))(loss2_ave_pool)
loss2_flat = Flatten()(loss2_conv)
loss2_fc = Dense(1024, activation='relu', name='loss2/fc', kernel_regularizer=l2(0.0002))(loss2_flat)
loss2_drop_fc = Dropout(0.7)(loss2_fc)
loss2_classifier = Dense(1000, name='loss2/classifier', kernel_regularizer=l2(0.0002))(loss2_drop_fc)
loss2_classifier_act = Activation('softmax')(loss2_classifier)
inception_4e_1x1 = Conv2D(256, (1,1), padding='same', activation='relu', name='inception_4e/1x1', kernel_regularizer=l2(0.0002))(inception_4d_output)
inception_4e_3x3_reduce = Conv2D(160, (1,1), padding='same', activation='relu', name='inception_4e/3x3_reduce', kernel_regularizer=l2(0.0002))(inception_4d_output)
inception_4e_3x3 = Conv2D(320, (3,3), padding='same', activation='relu', name='inception_4e/3x3', kernel_regularizer=l2(0.0002))(inception_4e_3x3_reduce)
inception_4e_5x5_reduce = Conv2D(32, (1,1), padding='same', activation='relu', name='inception_4e/5x5_reduce', kernel_regularizer=l2(0.0002))(inception_4d_output)
inception_4e_5x5 = Conv2D(128, (5,5), padding='same', activation='relu', name='inception_4e/5x5', kernel_regularizer=l2(0.0002))(inception_4e_5x5_reduce)
inception_4e_pool = MaxPooling2D(pool_size=(3,3), strides=(1,1), padding='same', name='inception_4e/pool')(inception_4d_output)
inception_4e_pool_proj = Conv2D(128, (1,1), padding='same', activation='relu', name='inception_4e/pool_proj', kernel_regularizer=l2(0.0002))(inception_4e_pool)
inception_4e_output = Concatenate(axis=1, name='inception_4e/output')([inception_4e_1x1,inception_4e_3x3,inception_4e_5x5,inception_4e_pool_proj])
inception_4e_output_zero_pad = ZeroPadding2D(padding=(1, 1))(inception_4e_output)
pool4_helper = PoolHelper()(inception_4e_output_zero_pad)
pool4_3x3_s2 = MaxPooling2D(pool_size=(3,3), strides=(2,2), padding='valid', name='pool4/3x3_s2')(pool4_helper)
inception_5a_1x1 = Conv2D(256, (1,1), padding='same', activation='relu', name='inception_5a/1x1', kernel_regularizer=l2(0.0002))(pool4_3x3_s2)
inception_5a_3x3_reduce = Conv2D(160, (1,1), padding='same', activation='relu', name='inception_5a/3x3_reduce', kernel_regularizer=l2(0.0002))(pool4_3x3_s2)
inception_5a_3x3 = Conv2D(320, (3,3), padding='same', activation='relu', name='inception_5a/3x3', kernel_regularizer=l2(0.0002))(inception_5a_3x3_reduce)
inception_5a_5x5_reduce = Conv2D(32, (1,1), padding='same', activation='relu', name='inception_5a/5x5_reduce', kernel_regularizer=l2(0.0002))(pool4_3x3_s2)
inception_5a_5x5 = Conv2D(128, (5,5), padding='same', activation='relu', name='inception_5a/5x5', kernel_regularizer=l2(0.0002))(inception_5a_5x5_reduce)
inception_5a_pool = MaxPooling2D(pool_size=(3,3), strides=(1,1), padding='same', name='inception_5a/pool')(pool4_3x3_s2)
inception_5a_pool_proj = Conv2D(128, (1,1), padding='same', activation='relu', name='inception_5a/pool_proj', kernel_regularizer=l2(0.0002))(inception_5a_pool)
inception_5a_output = Concatenate(axis=1, name='inception_5a/output')([inception_5a_1x1,inception_5a_3x3,inception_5a_5x5,inception_5a_pool_proj])
inception_5b_1x1 = Conv2D(384, (1,1), padding='same', activation='relu', name='inception_5b/1x1', kernel_regularizer=l2(0.0002))(inception_5a_output)
inception_5b_3x3_reduce = Conv2D(192, (1,1), padding='same', activation='relu', name='inception_5b/3x3_reduce', kernel_regularizer=l2(0.0002))(inception_5a_output)
inception_5b_3x3 = Conv2D(384, (3,3), padding='same', activation='relu', name='inception_5b/3x3', kernel_regularizer=l2(0.0002))(inception_5b_3x3_reduce)
inception_5b_5x5_reduce = Conv2D(48, (1,1), padding='same', activation='relu', name='inception_5b/5x5_reduce', kernel_regularizer=l2(0.0002))(inception_5a_output)
inception_5b_5x5 = Conv2D(128, (5,5), padding='same', activation='relu', name='inception_5b/5x5', kernel_regularizer=l2(0.0002))(inception_5b_5x5_reduce)
inception_5b_pool = MaxPooling2D(pool_size=(3,3), strides=(1,1), padding='same', name='inception_5b/pool')(inception_5a_output)
inception_5b_pool_proj = Conv2D(128, (1,1), padding='same', activation='relu', name='inception_5b/pool_proj', kernel_regularizer=l2(0.0002))(inception_5b_pool)
inception_5b_output = Concatenate(axis=1, name='inception_5b/output')([inception_5b_1x1,inception_5b_3x3,inception_5b_5x5,inception_5b_pool_proj])
pool5_7x7_s1 = AveragePooling2D(pool_size=(7,7), strides=(1,1), name='pool5/7x7_s2')(inception_5b_output)
loss3_flat = Flatten()(pool5_7x7_s1)
pool5_drop_7x7_s1 = Dropout(0.4)(loss3_flat)
loss3_classifier = Dense(1000, name='loss3/classifier', kernel_regularizer=l2(0.0002))(pool5_drop_7x7_s1)
loss3_classifier_act = Activation('softmax', name='prob')(loss3_classifier)
googlenet = Model(inputs=input, outputs=[loss1_classifier_act,loss2_classifier_act,loss3_classifier_act])
if weights_path:
googlenet.load_weights(weights_path)
return googlenet
if __name__ == "__main__":
img = imageio.imread('cat.jpg', pilmode='RGB')
img = np.array(Image.fromarray(img).resize((224, 224))).astype(np.float32)
img[:, :, 0] -= 123.68
img[:, :, 1] -= 116.779
img[:, :, 2] -= 103.939
img[:,:,[0,1,2]] = img[:,:,[2,1,0]]
img = img.transpose((2, 0, 1))
img = np.expand_dims(img, axis=0)
# Test pretrained model
model = create_googlenet('googlenet_weights.h5')
sgd = SGD(lr=0.1, decay=1e-6, momentum=0.9, nesterov=True)
model.compile(optimizer=sgd, loss='categorical_crossentropy')
out = model.predict(img) # note: the model has three outputs
labels = np.loadtxt('synset_words.txt', str, delimiter='\t')
predicted_label = np.argmax(out[2])
predicted_class_name = labels[predicted_label]
print('Predicted Class: ', predicted_label, ', Class Name: ', predicted_class_name)
from keras.layers.core import Layer
import theano.tensor as T
class LRN(Layer):
def __init__(self, alpha=0.0001,k=1,beta=0.75,n=5, **kwargs):
self.alpha = alpha
self.k = k
self.beta = beta
self.n = n
super(LRN, self).__init__(**kwargs)
def call(self, x, mask=None):
b, ch, r, c = x.shape
half_n = self.n // 2 # half the local region
input_sqr = T.sqr(x) # square the input
extra_channels = T.alloc(0., b, ch + 2*half_n, r, c) # make an empty tensor with zero pads along channel dimension
input_sqr = T.set_subtensor(extra_channels[:, half_n:half_n+ch, :, :],input_sqr) # set the center to be the squared input
scale = self.k # offset for the scale
norm_alpha = self.alpha / self.n # normalized alpha
for i in range(self.n):
scale += norm_alpha * input_sqr[:, i:i+ch, :, :]
scale = scale ** self.beta
x = x / scale
return x
def get_config(self):
config = {"alpha": self.alpha,
"k": self.k,
"beta": self.beta,
"n": self.n}
base_config = super(LRN, self).get_config()
return dict(list(base_config.items()) + list(config.items()))
class PoolHelper(Layer):
def __init__(self, **kwargs):
super(PoolHelper, self).__init__(**kwargs)
def call(self, x, mask=None):
return x[:,:,1:,1:]
def get_config(self):
config = {}
base_config = super(PoolHelper, self).get_config()
return dict(list(base_config.items()) + list(config.items()))
@rikkuporta

This comment has been minimized.

Copy link

commented Jul 7, 2016

Hi!

I changed the last layers to a Dense(8) instead of Dense(1000)
Now I am trying to retrain the Network with

ghist = googlemodel.fit_generator(
        get_train_gen(224,64),
        samples_per_epoch=4416,
        nb_epoch=30,
        validation_data=get_val_gen(224),
        nb_val_samples=1152)

but I get an Error Message:
Exception: The model expects 3 input arrays, but only received one array. Found: array with shape (64, 8).
[64 = BatchSize, 8=Classes]

Could you help me or give me a hint?

@joelouismarino

This comment has been minimized.

Copy link
Owner Author

commented Jul 11, 2016

The GoogLeNet model requires three output vectors, one for each of the classifiers. You can see this from the line

googlenet = Model(input=input, output=[loss1_classifier_act,loss2_classifier_act,loss3_classifier_act])

in googlenet.py. In order to train GoogLeNet in Keras, you need to feed three copies of your labels into the model. I'm not sure what your get_train_gen() function is doing, but it should be returning an ImageDataGenerator object. If you have X_train and Y_train and a generator datagen defined using

datagen = ImageDataGenerator()
datagen.fit(X_train)

then somewhere in get_train_gen(), you should be running something like

datagen.flow(X_train,Y_train, batch_size=64).

Something similar should be done in your get_val_gen() function. Now, I'm not sure you can use the fit_generator method with GoogLeNet. I would instead suggest using a for loop with the ImageDataGenerator flow method, as shown here. The three outputs come in when using one of the model's training methods, such as train_on_batch:

loss = model.train_on_batch(X_batch, [Y_batch,Y_batch,Y_batch]).

This would look something like:

for e in range(nb_epoch):
    batches = 0
    for X_batch, Y_batch in datagen.flow(X_train, Y_train, batch_size=64):
        loss = model.train_on_batch(X_batch, [Y_batch,Y_batch,Y_batch]) # note the three outputs
        batches += 1
        if batches >= len(X_train) / 64:
            # we need to break the loop by hand because
            # the generator loops indefinitely
            break
@rikkuporta

This comment has been minimized.

Copy link

commented Jul 14, 2016

Thank you for your answer!
Now I understand, but unfortunately I use flow_from_directory, where it reads from a directory. Do you know something for this?
I also tried to use the flow method but this also seems to fail with the error X (images tensor) and y (labels) should have the same length. Found: X.shape = (17999, 3, 224, 224), y.shape = (3, 17999) I'm not sure what to make out of this message.

This is the code:

def get_train_gen_google(batch):
    (X, Y) = load_data('training')

    datagen = ImageDataGenerator(
        rescale=1. / 255,
        shear_range=0.2,
        zoom_range=0.2)

    datagen.fit(X)

    datagen = datagen.flow(X, [Y, Y, Y], batch_size=batch)

    return datagen
def load_data(what):
    file = pp + '/dataset_2/'+what+'.txt'
    labels = list()
    data = list()
    with open(file, 'r') as f:
        reader = csv.reader(f, delimiter=';')
        for row in reader:
            labels.append(row[1])

            i = imread(pp+'/dataset_2/'+what+'/'+row[0], mode='RGB')
            img = imresize(i, (224, 224)).astype(np.float32)
            img[:, :, [0, 1, 2]] = img[:, :, [2, 1, 0]]
            img = img.transpose((2, 0, 1))
            nimg = np.asarray(img, dtype="uint8")
            data.append(nimg)
    Y = np.asarray(labels)
    Xb = np.asarray(data)
    X = np.array(Xb)
    return (X,Y)

Edit:
I also tried transposing, but then I get object of type 'ImageDataGenerator' has no len() error

@joelouismarino

This comment has been minimized.

Copy link
Owner Author

commented Jul 27, 2016

My apologies! I had confused ImageDataGenerator's flow method with the model's fit method. I updated my previous comment to reflect this.

@jsalbert

This comment has been minimized.

Copy link

commented Oct 6, 2016

Hi, I have this error when creating the net:
in create_googlenet
mode='concat', concat_axis=1, name='inception_3a/output')
File "/imatge/ajimenez/workspace/ITR/keras_env/local/lib/python2.7/site-packages/keras/engine/topology.py", line 1528, in merge
name=name)
File "/imatge/ajimenez/workspace/ITR/keras_env/local/lib/python2.7/site-packages/keras/engine/topology.py", line 1186, in init
node_indices, tensor_indices)
File "/imatge/ajimenez/workspace/ITR/keras_env/local/lib/python2.7/site-packages/keras/engine/topology.py", line 1253, in _arguments_validation
'Layer shapes: %s' % (input_shapes))

Exception: "concat" mode can only merge layers with matching output shapes except for the concat axis. Layer shapes: [(None, 1, 28, 64), (None, 1, 28, 128), (None, 1, 28, 32), (None, 1, 28, 32)]

Could you help me to solve it?

@leviswind

This comment has been minimized.

Copy link

commented Oct 25, 2016

the configuration of your keras is not correct, follow the instruction here.
File "Your user folder"/.keras/keras.json The default setting is:
{
"image_dim_ordering": "tf",
"epsilon": 1e-07,
"floatx": "float32",
"backend": "tensorflow"
}
just set "backend" to "theano" And set "image_dim_ordering" to "th"

@amanrana20

This comment has been minimized.

Copy link

commented Oct 29, 2016

Can you please tell me what ZeroPadding2D() and PoolHelper() are doing ?
Thanks

@JGuillaumin

This comment has been minimized.

Copy link

commented Dec 21, 2016

This is explained in this post : http://joelouismarino.github.io/blog_posts/blog_googlenet_keras.html

ZeroPadding2D : add zeros around feature maps after convolution layers
PoolHelper : remove the first row and column

@zaid478

This comment has been minimized.

Copy link

commented Jan 13, 2017

Can anyone tell me about how to perform Localization task with the model . I want bounding box parameters of all the objects in the image . Can anyone help please.

@Miladiouss

This comment has been minimized.

Copy link

commented Mar 27, 2017

When I run this code, I get the following error:

.../googlenet_custom_layers.py in call(self, x, mask)
     12 
     13     def call(self, x, mask=None):
---> 14         b, ch, r, c = x.shape
     15         half_n = self.n // 2 # half the local region
     16         input_sqr = T.sqr(x) # square the input

AttributeError: 'Tensor' object has no attribute 'shape'

What should I do?

@mdda

This comment has been minimized.

Copy link

commented Apr 19, 2017

Would you be open to someone (like me, for instance) trying to get this model into https://github.com/fchollet/deep-learning-models (which is MIT licensed)?

Like you, I think the Googlenet original model has a nice mix of low-parameters vs. performance (and FLOPs), so it would be good to have it in the 'standard library'.

@ernest-s

This comment has been minimized.

Copy link

commented May 10, 2017

Can you please let me know how to use this model for new dataset with 16 classes?

@infiniteperplexity

This comment has been minimized.

Copy link

commented Feb 13, 2018

I get TypeError: ('The following error happened while compiling the node', DotModulo(A, s, m, A2, s2, m2), '\n', "can't concat bytes to str")

@Mohamedsabry109

This comment has been minimized.

Copy link

commented Oct 7, 2018

this code gives different outputs when running with theano backend and tensorflow backend !!!!

@akshay-varshney

This comment has been minimized.

Copy link

commented Feb 14, 2019

ValueError: Improper config format: {'l2': 0.00019999999494757503, 'name': 'WeightRegularizer', 'l1': 0.0}
I am getting this error please help me to resolve this.

@williamsnick606

This comment has been minimized.

Copy link

commented Feb 16, 2019

I tried to run this and it gave me endless errors. I think it might be due to version issues. Which versions of libraries did you use?

@mamangxtc

This comment has been minimized.

Copy link

commented Feb 27, 2019

Hi i'm new to python and getting this error while to run the code :

model = create_googlenet('googlenet_weights.h5')
TypeError: 'module' object is not callable

what should i do? please help
i'm using python 3.6 and keras 2.2.4 in Visual Studio 2017

@MTarekShaaban

This comment has been minimized.

Copy link

commented Apr 12, 2019

How can I retrain using this code and weights? Urgent HELP needed please!

@yw3379

This comment has been minimized.

Copy link

commented Jun 4, 2019

Hi I get the error when I run the code:
theano.tensor.var.AsTensorError: ('Cannot convert Tensor("pool1/3x3_s2/MaxPool:0", shape=(?, 1, 56, 63), dtype=float32) to TensorType', <class 'tensorflow.python.framework.ops.Tensor'>)
what does it mean?plz help
I am using python 3.6 and keras2.2.4 in pycharm

@joelouismarino

This comment has been minimized.

Copy link
Owner Author

commented Jun 4, 2019

I've updated the Gist to work with the latest versions of Keras and Theano in Python 3.6. I have confirmed that this runs on the CPU.

@GianniDanese

This comment has been minimized.

Copy link

commented Jun 16, 2019

Hi I get the error when I run the code:
theano.tensor.var.AsTensorError: ('Cannot convert Tensor("pool1/3x3_s2/MaxPool:0", shape=(?, 1, 56, 63), dtype=float32) to TensorType', <class 'tensorflow.python.framework.ops.Tensor'>)
what does it mean?plz help
I am using python 3.6 and keras2.2.4 in pycharm

I have the same problem

@joelouismarino

This comment has been minimized.

Copy link
Owner Author

commented Jun 16, 2019

@yw3379 @GianniDanese it looks like its trying to convert between Theano and Tensorflow tensors. Currently, the LRN layer only works with Theano. Are you using the Theano backend in Keras?

@GianniDanese

This comment has been minimized.

Copy link

commented Jun 16, 2019

@yw3379 @GianniDanese it looks like its trying to convert between Theano and Tensorflow tensors. Currently, the LRN layer only works with Theano. Are you using the Theano backend in Keras?

Yes now. I've adding following lines at the starting of code:

import os
os.environ['KERAS_BACKEND'] = 'theano'

But I get the error when I run the code:

ValueError: A Concatenate layer requires inputs with matching shapes except for the concat axis. Got inputs shapes: [(None, 1, 28, 64), (None, 1, 28, 128), (None, 1, 28, 32), (None, 1, 28, 32)]

@joelouismarino

This comment has been minimized.

Copy link
Owner Author

commented Jun 16, 2019

Are you resizing your input image to (1, 3, 224, 224)?

@GianniDanese

This comment has been minimized.

Copy link

commented Jun 16, 2019

Are you resizing your input image to (1, 3, 224, 224)?

Yes, I haven't changed the code.

img = imageio.imread('cat.jpg', pilmode='RGB') img = np.array(Image.fromarray(img).resize((224, 224))).astype(np.float32)

Is it correct, right? this execute the resize.

@joelouismarino

This comment has been minimized.

Copy link
Owner Author

commented Jun 16, 2019

Yes, that's correct. I did some testing, and your error is actually coming from your configuration of channels. Theano uses channels as the first dimension, whereas Tensorflow uses channels as the last dimension. You can change this using the image_data_format entry in your keras.json file in the ~/.keras directory. It should look like the following:

{"epsilon": 1e-07, "floatx": "float32", "backend": "theano", "image_data_format": "channels_first"}
@GianniDanese

This comment has been minimized.

Copy link

commented Jun 16, 2019

Yes, that's correct. I did some testing, and your error is actually coming from your configuration of channels. Theano uses channels as the first dimension, whereas Tensorflow uses channels as the last dimension. You can change this using the image_data_format entry in your keras.json file in the ~/.keras directory. It should look like the following:

{"epsilon": 1e-07, "floatx": "float32", "backend": "theano", "image_data_format": "channels_first"}

I'm sorry, I did as you told me but now appear this new error:

OSError: Unable to open file (unable to open file: name = 'googlenet_weights.h5', errno = 2, error message = 'No such file or directory', flags = 0, o_flags = 0)

@joelouismarino

This comment has been minimized.

Copy link
Owner Author

commented Jun 16, 2019

You will need to download the weights (see the description of the Gist) and move them to your working directory. The main script calls

model = create_googlenet('googlenet_weights.h5')

which assumes the weights file is in the same directory. If you don't want to use a pre-trained model, you can instead just call

model = create_googlenet()
@GianniDanese

This comment has been minimized.

Copy link

commented Jun 16, 2019

You will need to download the weights (see the description of the Gist) and move them to your working directory. The main script calls

model = create_googlenet('googlenet_weights.h5')

which assumes the weights file is in the same directory. If you don't want to use a pre-trained model, you can instead just call

model = create_googlenet()

Sorry for the trouble, I'm a beginner with these things. thank you so much!

@joelouismarino

This comment has been minimized.

Copy link
Owner Author

commented Jun 16, 2019

No problem; I'm happy to help. Glad you're getting some use out of it!

@joelouismarino

This comment has been minimized.

Copy link
Owner Author

commented Jun 18, 2019

@utkarsh1148 you're getting the same error as above. This is due to not changing the image_data_format field in the ~/.keras/keras.json file to channels_first. It is set to channels_last by default. See the readme.md file for more details.

@utkarsh1148

This comment has been minimized.

Copy link

commented Jun 18, 2019

Using Theano backend.
WARNING (theano.configdefaults): g++ not available, if using conda: conda install m2w64-toolchain

Warning (from warnings module):
File "C:\Users\Utkarsh\AppData\Local\Programs\Python\Python36\lib\site-packages\theano\configdefaults.py", line 560
warnings.warn("DeprecationWarning: there is no c++ compiler."
UserWarning: DeprecationWarning: there is no c++ compiler.This is deprecated and with Theano 0.11 a c++ compiler will be mandatory
WARNING (theano.configdefaults): g++ not detected ! Theano will be unable to execute optimized C-implementations (for both CPU and GPU) and will default to Python implementations. Performance will be severely degraded. To remove this warning, set Theano flags cxx to an empty string.
WARNING (theano.tensor.blas): Using NumPy C-API based implementation for BLAS functions.
ERROR (theano.gof.opt): Optimization failure due to: local_abstractconv_check
ERROR (theano.gof.opt): node: AbstractConv2d{convdim=2, border_mode='half', subsample=(2, 2), filter_flip=True, imshp=(None, 3, 224, 224), kshp=(64, 3, 7, 7), filter_dilation=(1, 1), num_groups=1, unshared=False}(/input_1, InplaceDimShuffle{3,2,0,1}.0)
ERROR (theano.gof.opt): TRACEBACK:
ERROR (theano.gof.opt): Traceback (most recent call last):
File "C:\Users\Utkarsh\AppData\Local\Programs\Python\Python36\lib\site-packages\theano\gof\opt.py", line 2034, in process_node
replacements = lopt.transform(node)
File "C:\Users\Utkarsh\AppData\Local\Programs\Python\Python36\lib\site-packages\theano\tensor\nnet\opt.py", line 500, in local_abstractconv_check
node.op.class.name)
theano.gof.opt.LocalMetaOptimizerSkipAssertionError: AbstractConv2d Theano optimization failed: there is no implementation available supporting the requested options. Did you exclude both "conv_dnn" and "conv_gemm" from the optimizer? If on GPU, is cuDNN available and does the GPU support it? If on CPU, do you have a BLAS library installed Theano can link against? On the CPU we do not support float16.

Traceback (most recent call last):
File "E:/DL/google_net.py", line 164, in
out = model.predict(img) # note: the model has three outputs
File "C:\Users\Utkarsh\AppData\Local\Programs\Python\Python36\lib\site-packages\keras\engine\training.py", line 1164, in predict
self._make_predict_function()
File "C:\Users\Utkarsh\AppData\Local\Programs\Python\Python36\lib\site-packages\keras\engine\training.py", line 554, in _make_predict_function
**kwargs)
File "C:\Users\Utkarsh\AppData\Local\Programs\Python\Python36\lib\site-packages\keras\backend\theano_backend.py", line 1397, in function
return Function(inputs, outputs, updates=updates, **kwargs)
File "C:\Users\Utkarsh\AppData\Local\Programs\Python\Python36\lib\site-packages\keras\backend\theano_backend.py", line 1383, in init
**kwargs)
File "C:\Users\Utkarsh\AppData\Local\Programs\Python\Python36\lib\site-packages\theano\compile\function.py", line 317, in function
output_keys=output_keys)
File "C:\Users\Utkarsh\AppData\Local\Programs\Python\Python36\lib\site-packages\theano\compile\pfunc.py", line 486, in pfunc
output_keys=output_keys)
File "C:\Users\Utkarsh\AppData\Local\Programs\Python\Python36\lib\site-packages\theano\compile\function_module.py", line 1839, in orig_function
name=name)
File "C:\Users\Utkarsh\AppData\Local\Programs\Python\Python36\lib\site-packages\theano\compile\function_module.py", line 1519, in init
optimizer_profile = optimizer(fgraph)
File "C:\Users\Utkarsh\AppData\Local\Programs\Python\Python36\lib\site-packages\theano\gof\opt.py", line 108, in call
return self.optimize(fgraph)
File "C:\Users\Utkarsh\AppData\Local\Programs\Python\Python36\lib\site-packages\theano\gof\opt.py", line 97, in optimize
ret = self.apply(fgraph, *args, **kwargs)
File "C:\Users\Utkarsh\AppData\Local\Programs\Python\Python36\lib\site-packages\theano\gof\opt.py", line 251, in apply
sub_prof = optimizer.optimize(fgraph)
File "C:\Users\Utkarsh\AppData\Local\Programs\Python\Python36\lib\site-packages\theano\gof\opt.py", line 97, in optimize
ret = self.apply(fgraph, *args, **kwargs)
File "C:\Users\Utkarsh\AppData\Local\Programs\Python\Python36\lib\site-packages\theano\gof\opt.py", line 2143, in apply
nb += self.process_node(fgraph, node)
File "C:\Users\Utkarsh\AppData\Local\Programs\Python\Python36\lib\site-packages\theano\gof\opt.py", line 2039, in process_node
lopt, node)
File "C:\Users\Utkarsh\AppData\Local\Programs\Python\Python36\lib\site-packages\theano\gof\opt.py", line 1933, in warn_inplace
return NavigatorOptimizer.warn(exc, nav, repl_pairs, local_opt, node)
File "C:\Users\Utkarsh\AppData\Local\Programs\Python\Python36\lib\site-packages\theano\gof\opt.py", line 1919, in warn
raise exc
File "C:\Users\Utkarsh\AppData\Local\Programs\Python\Python36\lib\site-packages\theano\gof\opt.py", line 2034, in process_node
replacements = lopt.transform(node)
File "C:\Users\Utkarsh\AppData\Local\Programs\Python\Python36\lib\site-packages\theano\tensor\nnet\opt.py", line 500, in local_abstractconv_check
node.op.class.name)
theano.gof.opt.LocalMetaOptimizerSkipAssertionError: AbstractConv2d Theano optimization failed: there is no implementation available supporting the requested options. Did you exclude both "conv_dnn" and "conv_gemm" from the optimizer? If on GPU, is cuDNN available and does the GPU support it? If on CPU, do you have a BLAS library installed Theano can link against? On the CPU we do not support float16.

pls help

@joelouismarino

This comment has been minimized.

Copy link
Owner Author

commented Jun 18, 2019

This looks to be an issue with Theano. To test things, I would try running on the CPU. Make sure you're using float32 (again, see readme.md). If that still doesn't work, I would go back and check your installation here.

@utkarsh1148

This comment has been minimized.

Copy link

commented Jun 19, 2019

@joelouismarino

This comment has been minimized.

Copy link
Owner Author

commented Jun 19, 2019

Did you follow the installation instructions for Theano? First, install Anaconda. Then install the requirements:

conda install numpy scipy mkl

(this will install mkl, which is a BLAS library). Then install Theano:

conda install theano
@Flamingwizard4

This comment has been minimized.

Copy link

commented Jul 4, 2019

Hi so I am trying to implement GoogleNet for 6 classes and have added the following before each activation layer:

E.g.

loss1_classifier = Dense(1000, name='loss1/classifier', kernel_regularizer=l2(0.0002))(loss1_drop_fc)
loss1_classifier_final = Dense(6, name='loss1/classifier_final', kernel_regularizer=l2(0.0002))(loss1_classifier)
loss1_classifier_act = Activation('softmax')(loss1_classifier_final)

However, I am getting this error:

ValueError: Error when checking target: expected activation_1 to have shape (6,) but got array with shape (1,)

Is this a problem with my labels? Do I have to reformat them so the network can properly compare its predicted activations or is it something else?

Thanks

@Flamingwizard4

This comment has been minimized.

Copy link

commented Jul 4, 2019

Hi so I am trying to implement GoogleNet for 6 classes and have added the following before each activation layer:

E.g.

loss1_classifier = Dense(1000, name='loss1/classifier', kernel_regularizer=l2(0.0002))(loss1_drop_fc)
loss1_classifier_final = Dense(6, name='loss1/classifier_final', kernel_regularizer=l2(0.0002))(loss1_classifier)
loss1_classifier_act = Activation('softmax')(loss1_classifier_final)

However, I am getting this error:

ValueError: Error when checking target: expected activation_1 to have shape (6,) but got array with shape (1,)

Is this a problem with my labels? Do I have to reformat them so the network can properly compare its predicted activations or is it something else?

Thanks

Nevermind, I just had to use sparse_categorical_crossentropy :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.