Create a gist now

Instantly share code, notes, and snippets.

Updated to the Keras 2.0 API.
'''This script goes along the blog post
"Building powerful image classification models using very little data"
from blog.keras.io.
It uses data that can be downloaded at:
https://www.kaggle.com/c/dogs-vs-cats/data
In our setup, we:
- created a data/ folder
- created train/ and validation/ subfolders inside data/
- created cats/ and dogs/ subfolders inside train/ and validation/
- put the cat pictures index 0-999 in data/train/cats
- put the cat pictures index 1000-1400 in data/validation/cats
- put the dogs pictures index 12500-13499 in data/train/dogs
- put the dog pictures index 13500-13900 in data/validation/dogs
So that we have 1000 training examples for each class, and 400 validation examples for each class.
In summary, this is our directory structure:
```
data/
train/
dogs/
dog001.jpg
dog002.jpg
...
cats/
cat001.jpg
cat002.jpg
...
validation/
dogs/
dog001.jpg
dog002.jpg
...
cats/
cat001.jpg
cat002.jpg
...
```
'''
import numpy as np
from keras.preprocessing.image import ImageDataGenerator
from keras.models import Sequential
from keras.layers import Dropout, Flatten, Dense
from keras import applications
# dimensions of our images.
img_width, img_height = 150, 150
top_model_weights_path = 'bottleneck_fc_model.h5'
train_data_dir = 'data/train'
validation_data_dir = 'data/validation'
nb_train_samples = 2000
nb_validation_samples = 800
epochs = 50
batch_size = 16
def save_bottlebeck_features():
datagen = ImageDataGenerator(rescale=1. / 255)
# build the VGG16 network
model = applications.VGG16(include_top=False, weights='imagenet')
generator = datagen.flow_from_directory(
train_data_dir,
target_size=(img_width, img_height),
batch_size=batch_size,
class_mode=None,
shuffle=False)
bottleneck_features_train = model.predict_generator(
generator, nb_train_samples // batch_size)
np.save(open('bottleneck_features_train.npy', 'w'),
bottleneck_features_train)
generator = datagen.flow_from_directory(
validation_data_dir,
target_size=(img_width, img_height),
batch_size=batch_size,
class_mode=None,
shuffle=False)
bottleneck_features_validation = model.predict_generator(
generator, nb_validation_samples // batch_size)
np.save(open('bottleneck_features_validation.npy', 'w'),
bottleneck_features_validation)
def train_top_model():
train_data = np.load(open('bottleneck_features_train.npy'))
train_labels = np.array(
[0] * (nb_train_samples / 2) + [1] * (nb_train_samples / 2))
validation_data = np.load(open('bottleneck_features_validation.npy'))
validation_labels = np.array(
[0] * (nb_validation_samples / 2) + [1] * (nb_validation_samples / 2))
model = Sequential()
model.add(Flatten(input_shape=train_data.shape[1:]))
model.add(Dense(256, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(1, activation='sigmoid'))
model.compile(optimizer='rmsprop',
loss='binary_crossentropy', metrics=['accuracy'])
model.fit(train_data, train_labels,
epochs=epochs,
batch_size=batch_size,
validation_data=(validation_data, validation_labels))
model.save_weights(top_model_weights_path)
save_bottlebeck_features()
train_top_model()
@lgondara

I really love this example. I had some problems getting it running on python 3.5, but everything ran smooth on 2.7. Great work.

@jdelange

I get an error at the point where it starts train_top_model():, at this line: train_data = np.load(open('bottleneck_features_train.npy'))

ValueError was unhandled by user code
Message: total size of new array must be unchanged

It seems to suggest that the numpy array saved in the save_bottlebeck_features(): has somehow changed?

@fabioperez

How to get the labels when training more than two categories? Is it possible to infer the labels from the generator?

@manakatie

I'm getting an error in the line datagen = ImageDataGenerator(rescale=1./255). Error is TypeError: __init__() got an unexpected keyword argument 'rescale'.

More generally, I get a similar unexpected keyword error if I uncomment any of the commented lines in the following, which is taken from the ImageDataGenerator documentation.

from keras.preprocessing.image import ImageDataGenerator
datagen = ImageDataGenerator(featurewise_center=False,
    samplewise_center=False,
    featurewise_std_normalization=False,
    samplewise_std_normalization=False,
    zca_whitening=False,
    rotation_range=0.,
    width_shift_range=0.,
    height_shift_range=0.,
#     shear_range=0.,
#     zoom_range=0.,
#     channel_shift_range=0.,
#     fill_mode='nearest',
#     cval=0.,
#     rescale=None,
#     dim_ordering=False,
    horizontal_flip=False,
    vertical_flip=False)

I'm using Python 2.7 with Tensor Flow backend. Appreciate any help.

(Francois, these gists and your blog posts have been really helpful to me, thanks for taking the time to share them!)

@assaflehr
assaflehr commented Jul 6, 2016 edited

@jdelange To solve the ValueError

ValueError was unhandled by user code
Message: total size of new array must be unchanged

If you are using Windows platform, you must use the binary mode of files when saving numpy array, so just add the char 'b' to the mode, like this:
np.save(open(file_name, 'wb'), saved)
loaded = np.load(open(file_name,'rb'))

This weird non cross-platform python behavior explained here

@Solarwinds1991

It's awesome!Thank you so much

@yurkor
yurkor commented Jul 10, 2016

Ideally we want use original VGG image prepossessing mean and no scale
i.e.

datagen = ImageDataGenerator(rescale=1., featurewise_center=True) #(rescale=1./255)
datagen.mean=np.array([103.939, 116.779, 123.68],dtype=np.float32).reshape(3,1,1)

it gives final accuracy 95 instead of 90
Epoch 3/10
0s - loss: 1.3352 - acc: 0.9100 - val_loss: 0.7913 - val_acc: 0.9470

Epoch 3/10
0s - loss: 0.3059 - acc: 0.8725 - val_loss: 0.2171 - val_acc: 0.9140

@mawangyi
mawangyi commented Sep 1, 2016 edited

I really love this example. Thank You Very Much!

@mawangyi
mawangyi commented Sep 5, 2016 edited

How do I use this model to predict classification
my code:

weights_path = 'vgg16_weights.h5'
top_model_weights_path = 'bottleneck_fc_model.h5'
img_width, img_height = 150, 150

def VGG16_bottlebeck_features():
input_shape = (3, 150, 150)
img_input = Input(shape=input_shape)

x = Flatten(name='flatten')(img_input)
x = Dense(256, activation='relu', name='fc1')(x)
x = Dense(1, activation='sigmoid', name='predictions')(x)

model = Model(img_input, x)
model.load_weights("bottleneck_fc_model.h5")
return model

if name == 'main':
model = VGG16_bottlebeck_features()
img_path = 'image/cat.jpg'
img = image.load_img(img_path, target_size=(150, 150))
x = image.img_to_array(img)

x = np.expand_dims(x, axis=0)
x = preprocess_input(x)

preds = model.predict(x)

print('Predicted:', decode_predictions(preds))

And ERROR:
Traceback (most recent call last):
File "vgg16.py", line 194, in
preds = model.predict(x)
File "/usr/local/lib/python2.7/site-packages/keras/engine/training.py", line 1177, in predict
batch_size=batch_size, verbose=verbose)
File "/usr/local/lib/python2.7/site-packages/keras/engine/training.py", line 876, in _predict_loop
batch_outs = f(ins_batch)
File "/usr/local/lib/python2.7/site-packages/keras/backend/theano_backend.py", line 672, in call
return self.function(*inputs)
File "/usr/local/lib/python2.7/site-packages/theano/compile/function_module.py", line 871, in call
storage_map=getattr(self.fn, 'storage_map', None))
File "/usr/local/lib/python2.7/site-packages/theano/gof/link.py", line 314, in raise_with_op
reraise(exc_type, exc_value, exc_trace)
File "/usr/local/lib/python2.7/site-packages/theano/compile/function_module.py", line 859, in call
outputs = self.fn()
File "/usr/local/lib/python2.7/site-packages/theano/gof/op.py", line 912, in rval
r = p(n, [x[0] for x in i], o)
File "/usr/local/lib/python2.7/site-packages/theano/tensor/blas.py", line 1552, in perform
z[0] = numpy.asarray(numpy.dot(x, y))
ValueError: ('shapes (1,67500) and (8192,256) not aligned: 67500 (dim 1) != 8192 (dim 0)', (1, 67500), (8192, 256))
Apply node that caused the error: Dot22(Reshape{2}.0, fc1_W)
Toposort index: 11
Inputs types: [TensorType(float32, matrix), TensorType(float32, matrix)]
Inputs shapes: [(1, 67500), (8192, 256)]
Inputs strides: [(270000, 4), (1024, 4)]
Inputs values: ['not shown', 'not shown']
Outputs clients: [[Elemwise{Composite{(i0 * (Abs((i1 + i2)) + i1 + i2))}}[(0, 1)](TensorConstant{%281, 1%29 of 0.5}, Dot22.0, InplaceDimShuffle{x,0}.0)]]

HINT: Re-running with most Theano optimization disabled could give you a back-trace of when this node was created. This can be done with by setting the Theano flag 'optimizer=fast_compile'. If that does not work, Theano optimizations can be disabled with 'optimizer=None'.
HINT: Use the Theano flag 'exception_verbosity=high' for a debugprint and storage map footprint of this apply node.

@michelyang

I really love this simple example. But I got a problem now. Can anyone tell me how to make prediction? Here is what I did.
I use a picture of cat. Input it to the VGG model (bottle neck one). I got a feature. Then load the feature to the trained model (the added one). But got wrong result. The prediction is always 0 (the first one).

load json and create model

json_file = open('model_vgg.json', 'r')
loaded_model_json = json_file.read()
json_file.close()
model1 = model_from_json(loaded_model_json)

load weights into new model

assert os.path.exists(weights_path), 'Model weights not found (see "weights_path" variable in script).'
f = h5py.File(weights_path)

for k in range(f.attrs['nb_layers']):
if k >= len(model1.layers):
# we don't look at the last (fully-connected) layers in the savefile
break
g = f['layer_{}'.format(k)]
weights = [g['param_{}'.format(p)] for p in range(g.attrs['nb_params'])]
model1.layers[k].set_weights(weights)
f.close()
print('Model VGG loaded.')

load json and create model

json_file = open('model_catdog.json', 'r')
loaded_model_json = json_file.read()
json_file.close()
model2 = model_from_json(loaded_model_json)

load weights into new model

model2.load_weights("bottleneck_fc_model.h5")
print('Model CAT DOG loaded.')

img = cv2.imread("dog.1000.jpg")
img = cv2.resize(img, (150, 150)).astype(np.float32) / 255.
print img
img = img.reshape(1 , img.shape[2], img.shape[0], img.shape[1])

feature = model1.predict(img)
probs = model2.predict(feature)
print "probs: ", probs
prediction = probs.argmax(axis=1)
print "prediction: ", prediction

@srikar2097
srikar2097 commented Sep 24, 2016 edited

I have done exactly what this code does, by the end of 50th epoch I am getting high training accuracy but ≈ 50% validation accuracy. In fact the validation accuracy hovers around 50% throughout. See the last 3 epochs. See the non-changing val_loss and val_acc. What am i doing wrong?

Epoch 48/50
10000/10000 [==============================] - 2s - loss: 0.3180 - acc: 0.9131 - val_loss: 7.9712 - val_acc: 0.5000
Epoch 49/50
10000/10000 [==============================] - 2s - loss: 0.2903 - acc: 0.9172 - val_loss: 7.9712 - val_acc: 0.5000
Epoch 50/50
10000/10000 [==============================] - 2s - loss: 0.3272 - acc: 0.9132 - val_loss: 7.9712 - val_acc: 0.5000

Facts -
0. followed the post as mentioned here. This is to get a model with pre-trained Image-net bottleneck features.

  1. I have used the above code by François.
  2. Got VGG16 pre-trained weights from here. As mentioned in post.
  3. I did not see a pre-processing step where Image is zero-centered around mean. I tried with and without this step but no change in validation performance.
  4. Using Theano backend on a GPU.

What am i doing wrong?

@paho67
paho67 commented Sep 29, 2016

I also wonder, how to make predictions.
I tried to load saved model and then give it to my own image:
...
img = load_img(img_path,target_size=(img_width,img_height))
x = img_to_array(img)
x = np.expand_dims(x, axis=0)
x = preprocess_input(x)
model = load_model('.\models' + model_name + '.h5')
preds = model.predict_classes(x)
prob = model.predict_proba(x)

But I got error:
Exception: Error when checking : expected flatten_input_1 to have shape (None, 512, 4, 4) but got array with shape (1, 3, 150, 150)

@xcmax
xcmax commented Sep 29, 2016

Question:how to use this to multiclass classification? In order to apply it to multiclass problem, I changed the loss function t'binary_crossentropy' to 'categorical_crossentropy', but the acc could only reach 50% or so? I used 700 images for training 69 classes.

@shackenberg

@srikar2097, did you find out a solution? Facing the same issue.

@bayraktare

I carefully followed the given procedure given both at blog and here. Unfortunately, I am getting the following error which I couldn't find a way to solve:
Traceback (most recent call last):
File "", line 1, in
File "/usr/lib/python2.7/dist-packages/spyderlib/widgets/externalshell/sitecustomize.py", line 540, in runfile
execfile(filename, namespace)
File "/home/santez/Desktop/DL Examples/KERAS/Object Recognition/Classifier From Little Data KeraBlog/classifierfromlittledata3.py", line 118, in
model.layers[k].set_weights(weights)
File "/usr/local/lib/python2.7/dist-packages/keras/engine/topology.py", line 889, in set_weights
'provided weight shape ' + str(w.shape))
Exception: Layer weight shape (3, 3, 150, 64) not compatible with provided weight shape (64, 3, 3, 3)

@AakashKumarNain

@paho67 I am also facing the same issue

@jessicaowensby

@paho67 @AakashKumarNain
I ran into the same issue and documented the fix here.

When I do the prediction, it looks like this:

img = load_img('my_img.jpg',False, (img_width, img_height))
x = img_to_array(img)
prediction = model.predict(x.reshape((1,3,img_width, img_height)),batch_size=32, verbose=0)
print(prediction)

I am not sure this is 100% correct since I am not getting accurate predictions, but I am getting a prediction nonetheless.

@shackenberg

@srikar2097, I found the reason for that issue. flow_from_directory selects from one folder at a time and as there are more then 2000 images in the first folder (13k for me) it just takes images from one folder/class but still pretends like the val data is half from one class half from the other. See line 145.

@davidpickup

I'm getting the following error for line 79:

ValueError: Filter must not be larger than the input: Filter: (2, 2) Input: (1, 75)

@bayraktare

Hi everybody!
I think I have confronted with most of the dimension errors you wrote and probably you have to change the configuration of the ~/keras/keras.json file as following:
{
"image_dim_ordering": "th",
"epsilon": 1e-07,
"floatx": "float32",
"backend": "tensorflow"
}

@paho67
paho67 commented Oct 13, 2016

@jessicaowensby I think that this is not solution for my problem.
I guess that loaded model assume that input have to be this type:
model.add(Flatten(input_shape=train_data.shape[1:])) <=> which is (None, 512, 4, 4).

I wonder how to change this that way that I could give it 150x150 image (array with shape (1, 3, 150, 150))?

@fabioperez

@bayraktare Thanks for this fix!

@daveselinger
daveselinger commented Oct 22, 2016 edited

@srikar2097 and @shackenberg So the specific way to fix this is to enter the correct number in nb_train_samples and nb_validation_samples.

@bayraktare Thanks--that is a better way than what I'm doing which is modifying every file with:

from keras import backend as K
K.set_image_dim_ordering('th')

FINAL NOTE: it appears that numpy has changed its interface so that np.save now takes a filename. The np.save appears to add '.npy' to the end (whereas load does not), so instead of opening the file, you have to use this interface:
np.save("example") followed by:
np.load("example.npy")

I was getting: TypeError: must be str, not bytes

@edmundronald
edmundronald commented Oct 25, 2016 edited

I'm just going from fixing one bug to the next and not getting the code to work. With the installation etc I am easily at 20h work now (Vagrant Mac) installing one package after the other and fixing things. I don't think the code is fixable with Theano, I am now going to install tf, but am growing tired. Would it be possible to make a VM which just runs and has a complete set of everything? I would be willing to work on that. I have a PhD in AI, and I'm 60 years old, but this is situation is more for a 15 year old with time on his hands and knowledge of Python version issues.

@edmundronald

Here is the error I am now getting
RuntimeError: BaseCorrMM: Failed to allocate output of 32 x 64 x 150 x 150
Apply node that caused the error: CorrMM{valid, (1, 1)}(IncSubtensor{InplaceSet;::, ::, int64:int64:, int64:int64:}.0, Subtensor{::, ::, ::int64, ::int64}.0)
Toposort index: 55
Inputs types: [TensorType(float32, 4D), TensorType(float32, 4D)]
Inputs shapes: [(32, 3, 152, 152), (64, 3, 3, 3)]
Inputs strides: [(277248, 92416, 608, 4), (108, 36, -12, -4)]
Inputs values: ['not shown', 'not shown']
Outputs clients: [[Elemwise{Composite{(i0 * (Abs((i1 + i2)) + i1 + i2))}}[(0, 1)](TensorConstant{%281, 1, 1, 1%29 of 0.5}, CorrMM{valid, %281, 1%29}.0, Reshape{4}.0)]]

@trancept

@edmundronald : I think you have an installation problem. I use Ubuntu 16.04, Python3.5, Tensorfow and the only modification I have to made was changing from "tf" to "th" in ~/.keras/keras.json
And made the script run under Python3.5 with small modifications : https://gist.github.com/trancept/010665252efcdffc5d886325b36fc951

@bluelight773

@xcmax (or anyone who attempted applying the code for multiple classes) did you figure out how to make it work for multi-class? Thanks

@ManasHardas

I am getting the following error

Using TensorFlow backend.
Model loaded.
Found 573 images belonging to 2 classes.
Traceback (most recent call last):
File "with_vgg16_weights.py", line 144, in
save_bottlebeck_features()
File "with_vgg16_weights.py", line 90, in save_bottlebeck_features
bottleneck_features_train = model.predict_generator(generator, nb_train_samples)
File "/Users/ml/lib/python3.5/site-packages/keras/models.py", line 945, in predict_generator
pickle_safe=pickle_safe)
File "/Users/ml/lib/python3.5/site-packages/keras/engine/training.py", line 1655, in predict_generator
all_outs[i][processed_samples:(processed_samples + nb_samples)] = out
ValueError: could not broadcast input array from shape (32,512,4,4) into shape (10,512,4,4)

This seems to be a numpy error not a keras error. My settings are exactly like the example except the number of training and validation examples. Has anyone else encountered this before?
Thanks

@tlind
tlind commented Nov 23, 2016 edited

@ManasHardas I am having exactly the same issue on my own dataset. Did you find a solution?

The problem seems to be related to nb_train_samples not being a multiple of batch_size. If I increase nb_train_samples to a larger number (without actually adding more training images), I guess the labels would end up being incorrect because the assumption is that the first nb_train_samples/2 samples are of the first class, and the second nb_train_samples/2 are of the second class.

What I don't understand is why the original code was supposed to work like this, since 2000 is also not divisible by 32. Maybe something changed in Keras internally?

@tlind
tlind commented Nov 24, 2016

Problem solved! Both my nb_train_samples and nb_validation_samples were set incorrectly (I had to subtract 2 from both). With the correct number of samples set, the issue has disappeared. I think while I was counting the number of files inside the train and validation directories, my file browser was also considering the folders of the two classes as extra items. The count provided by ImageDataGenerator in the console ("Found 5946 images belonging to 2 classes.") was correct, though.

@ManasHardas
ManasHardas commented Nov 25, 2016 edited

@tlind I am still getting error. nb_train_samples=600 and nb_validation_samples=100 and batch_size=32. This means I have 300 images of each class in a folder for training and 50 images for each class for validation. Now I am getting this error.

Using TensorFlow backend.
Model loaded.
Found 600 images belonging to 2 classes.
Traceback (most recent call last):
File "with_vgg16_weights.py", line 144, in
save_bottlebeck_features()
File "with_vgg16_weights.py", line 95, in save_bottlebeck_features
bottleneck_features_train
File "/Users/manas/ml/lib/python3.5/site-packages/numpy/lib/npyio.py", line 491, in save
pickle_kwargs=pickle_kwargs)
File "/Users/manas/ml/lib/python3.5/site-packages/numpy/lib/format.py", line 555, in write_array
version)
File "/Users/manas/ml/lib/python3.5/site-packages/numpy/lib/format.py", line 328, in _write_array_header
fp.write(header_prefix)
TypeError: write() argument must be str, not bytes
Exception ignored in: <bound method BaseSession.del of <tensorflow.python.client.session.Session object at 0x111c13da0>>
Traceback (most recent call last):
File "/Users/manas/ml/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 532, in del
AttributeError: 'NoneType' object has no attribute 'TF_DeleteStatus'

@rajshah4

@davidpickup I think the errors might be due to a tensorflow/theano issue. It took me a while, but I put together a notebook that runs all the code with a tensorflow backend. Its available here: https://github.com/rajshah4/image_keras

@ManasHardas

I resolved --- TypeError: write() argument must be str, not bytes --- by following @daveselinger suggestion. Create model without .npy extension
np.save("example") np.load("example.npy")

For prediction I am doing the following
top_models_weight_path = 'bottleneck_fc_model.h5'
img = image.load_img(test_img, target_size=(img_width, img_height))
x = image.img_to_array(img)
x = np.expand_dims(x, axis=0)
x = preprocess_input(x)
model = load_weights(top_models_weight_path)
class_preds = model.predict_classes(x)
prob_preds = model.predict_proba(x)

But getting this error
Using TensorFlow backend.
Traceback (most recent call last):
File "predict.py", line 27, in <module> model = load_model(top_model_weights_path)
File "/Users/ml/lib/python3.5/site-packages/keras/models.py", line 126, in load_model raise ValueError('No model found in config file.')
ValueError: No model found in config file.

@rajivshah4 - were you able to load the classifier layer weights and actually predict the class for a single test image?

@MayorPain

Don't you have to pass the test images to the same generator if you want to predict on them, like the train and validation images. So you get the (None, 512L, 2L, 2L) shape. Like:

generator = datagen.flow(
X_test,
batch_size=32,

        shuffle=False)
bottleneck_features_validation = model.predict_generator(generator, len(X_test))
np.save('bottleneck_features_test.npy', bottleneck_features_validation)

test = np.load('bottleneck_features_test.npy')

predictions = model.predict(test, verbose=1)

@sulaimanvesal
sulaimanvesal commented Dec 8, 2016 edited

@MayorPain - I have created as follow the generator and send through the convolutional layer as follow:

generator = datagen.flow_from_directory(
test_data_dir,
batch_size=32,
target_size=(img_width, img_height),
class_mode=None,
shuffle=False)
test_data_features = model.predict_generator(generator, 12500)
np.save(open('test_data_features.npy', 'w'), test_data_features)

My question is that when this generator searches test-folder, it ask for a subfolder and count that subfolder as a class. I have run my program and everything is smooth but during the predication accuracy is really low. Although I have trained over entire Cat-&-Dog dataset with 50 Epochs and my validation accuracy for train data and validation data reaches to 99.2 %. But when I predict the test data it's really worse.

Would you please share what is wrong with my test-images or the generator provide some ambiguity problem.

@projektneo2016

Thanks for the tutorial , its nice one.
I need help to predict an image. I try everything and nothing works. :(

I get this error:
Exception: Error when checking : expected flatten_input_1 to have shape (None, 512, 4, 4) but got array with shape (1, 3, 150, 150)

@austinchencym

@bluelight773
@xcmax
did you guys (or anyone who attempted applying the code for multiple classes) figure out how to make it work for multi-class? Thanks

@austinchencym
austinchencym commented Dec 28, 2016 edited

I got

File "data/Classifier2.py", line 100, in train_top_model
train_data = np.load(open('bottleneck_features_train.npy'))

UnicodeDecodeError: 'charmap' codec can't decode byte 0x8d in position 142: character maps to undefined

did any one have the same issue and get solved?>

@dzorlu
dzorlu commented Jan 7, 2017

For multiclass prediction you need to convert your labels via to_categorical. For more info see this:

http://stackoverflow.com/questions/36927025/how-to-use-keras-multi-layer-perceptron-for-multi-class-classification

@shan4224
shan4224 commented Jan 9, 2017 edited

I am getting some errors while predicting test data. How to predict the images for test data in Using the bottleneck features of a pre-trained network section.

I have tried the following :

test_data_dir = '/Dogs_Cats/data/test' ## There are 100 imagesin test data
datagen = ImageDataGenerator(rescale=1., featurewise_center=True)
datagen.mean=np.array([103.939, 116.779, 123.68],dtype=np.float32).reshape(3,1,1)
generator = datagen.flow_from_directory(
test_data_dir,
target_size=(img_width, img_height),
batch_size=32,
shuffle=False,
classes = None)

test_data_features = model.predict_generator(generator, 100)
np.save(open('/Dogs_Cats/data/test_data_features.npy','wb'), test_data_features)
test_data =np.load(open('/Dogs_Cats/data/test_data_features.npy','rb'))
predictions = model.predict_classes(test_data, verbose=1)

I am getting errors:
Exception: Error when checking : expected flatten_input_5 to have shape (None, 512, 4, 4) but got array with shape (100, 3, 150, 150)

@GuminEgor

@shan4224, @paho67

I am getting errors:
Exception: Error when checking : expected flatten_input_5 to have shape (None, 512, 4, 4) but got array with shape (100, 3, 150, 150)

that's why our little 2-layers network is not specified for getting pictures as input.
We need to add these two layers to the existing 13 ones and to compile ALL the 15-layers network.
If you look at the third part of the tutorial, you will understand how to do it

@biswatherockstar

Hi Folks,

Anyone know the reason why I get loss : nan for some end epochs . I was trying this code with a multiclass classification

Epoch 44/50
3021/3021 [==============================] - 8s - loss: 0.0414 - acc: 0.9841 - val_loss: 0.5106 - val_acc: 0.9346
Epoch 45/50
3021/3021 [==============================] - 11s - loss: 0.0402 - acc: 0.9864 - val_loss: 0.5089 - val_acc: 0.9432
Epoch 46/50
3021/3021 [==============================] - 9s - loss: nan - acc: 0.7306 - val_loss: nan - val_acc: 0.4247
Epoch 47/50
3021/3021 [==============================] - 10s - loss: nan - acc: 0.4551 - val_loss: nan - val_acc: 0.4247
Epoch 48/50
3021/3021 [==============================] - 9s - loss: nan - acc: 0.4551 - val_loss: nan - val_acc: 0.4247
Epoch 49/50
3021/3021 [==============================] - 11s - loss: nan - acc: 0.4551 - val_loss: nan - val_acc: 0.4247
Epoch 50/50
3021/3021 [==============================] - 11s - loss: nan - acc: 0.4551 - val_loss: nan - val_acc: 0.4247

Thanks,
Biswa

@al3xsh
al3xsh commented Jan 26, 2017 edited

Hi,

First of all, thanks for a great tutorial! I have only just discovered keras and this example has shown me how simple and powerful development using the keras framework can be.

I have followed the tutorial (and made the modification to use the published vgg16 image preprocessing means suggested by @yurkor) but i seem to get significantly different results using the theano backend and the tensorflow backend - does this happen to anybody else?

Using the vgg16 image preprocessing means and the theano backend I get a validation accuracy of ~95% after 50 epochs, but using the tensorflow backend the validation accuracy is between 88 - 90% after 50 epochs.

Without the vgg16 image preprocessing means the accuracy is around 90% for the theano backend and 86% for tensorflow.

Any ideas why theano seems to outperform tensorflow? I thought that they would be almost identical.

Regards,

Alex

@al3xsh
al3xsh commented Jan 26, 2017

Follow up:

If I use the vgg16 example with imagenet weights provided with keras:

from keras.applications.vgg16 import VGG16
model = VGG16(weights='imagenet', include_top=False)

to do the feature extraction and then build the classifier on top of that (as in this tutorial), then I get the same accuracy using the tensorflow backend as with using the theano backend. I assume I'm messing up the conversion of the vgg 16 weights from theano format to tensorflow format. At the moment I am doing:

        # the saved weights are in theano format and we are using the 
        # tensorflow backend so we need to manually reorder them because 
        # tensorflow implements convolution differently ...
        if layer.__class__.__name__ in ['Convolution1D', 
                                        'Convolution2D', 
                                        'Convolution3D', 
                                        'AtrousConvolution2D']:
            weights[0] = np.transpose(weights[0], (2, 3, 1, 0))

which came from:
https://gist.github.com/baraldilorenzo/07d7802847aaad0a35d3#gistcomment-1905315

I want to use something like convert_kernel - but am having difficulty going from the h5 file to the numpy array.

Any ideas how I should be doing it?

Regards,

Alex

@rk-ka
rk-ka commented Jan 29, 2017

TypeError: can't multiply sequence by non-int of type 'float'

TypeError Traceback (most recent call last)
in ()
169
170 save_bottlebeck_features()
--> 171 train_top_model()

in train_top_model()
149 def train_top_model():
150 train_data = np.load('C:/toolkits/data/model/bottleneck/bottleneck_features_train.npy')
--> 151 train_labels = np.array([0] * (nb_train_samples / 2) + [1] * (nb_train_samples / 2))
152
153 validation_data = np.load('C:/toolkits/data/model/bottleneck/bottleneck_features_validation.npy')

TypeError: can't multiply sequence by non-int of type 'float'


Help please

@jamiejwsmith

@rk-ka Are you running this under Python 3.5? I had this exact issue when I ran it under Python 3.5, however I was able to find a solution to the problem.

Change the following line:
train_labels = np.array([0] * (nb_train_samples / 2) + [1] * (nb_train_samples / 2))

To:
train_labels = np.array([0] * (nb_train_samples // 2) + [1] * (nb_train_samples // 2))

And also change the following line:
validation_labels = np.array([0] * (nb_validation_samples / 2) + [1] * (nb_validation_samples / 2))

To:
validation_labels = np.array([0] * (nb_validation_samples // 2) + [1] * (nb_validation_samples // 2))

@tovalinder
tovalinder commented Feb 9, 2017 edited

@srikar2097 I had the same problem. My problem was not the number of images in the folders, however removing the file type in the filename solved the problem, see example below.

np.save(open('bottleneck_features_train.npy', 'wb'), bottleneck_features_train)
changed into
np.save(open('bottleneck_features_train', 'wb'), bottleneck_features_train)

Loading the bottleneck features with train_data = np.load(open('bottleneck_features_train', 'rb'))

@rk-ka Changing the second argument to open() into 'wb' for writing and 'rb' for reading (as in the example above) also solves the TypeError.

@rahul15495

Here is my code for building the model

import h5py
import numpy as np
from keras.preprocessing.image import ImageDataGenerator
from keras.models import Sequential
from keras.layers import Convolution2D, MaxPooling2D, ZeroPadding2D
from keras.layers import Activation, Dropout, Flatten, Dense
from keras.applications.vgg16 import VGG16
from keras.models import model_from_json

def save_bottleneck_features():
datagen= ImageDataGenerator(rescale= 1./255)

base_model= VGG16(weights='imagenet', include_top=False)
print("model loaded.")

generator = datagen.flow_from_directory(
	'data/train',
	target_size=(150, 150),
	batch_size=32,
	class_mode=None,
	shuffle=False

)
bottleneck_features_train= base_model.predict_generator(generator,2000)

generator = datagen.flow_from_directory(
	'data/valid',
	target_size=(150, 150),
	batch_size=32,
	class_mode=None,
	shuffle=False)


bottleneck_features_validation= base_model.predict_generator(generator,800)

h5f= h5py.File('catsVSdogs_bottleNeck1.h5', 'w')
h5f.create_dataset('bottleneck_features_train', data=bottleneck_features_train)
h5f.create_dataset('bottleneck_features_validation', data=bottleneck_features_validation)
h5f.close()

def train_top_model():

h5f= h5py.File('catsVSdogs_bottleNeck1.h5', 'r')
train_data= h5f['bottleneck_features_train'][:]
train_labels= np.array([0]*1000+ [1]*1000)

validation_data= h5f['bottleneck_features_validation'][:]
validation_labels= np.array([0]*400+ [1]*400)
h5f.close()

model= Sequential()
model.add(Flatten(input_shape= train_data.shape[1:]))
model.add(Dense(256, activation= 'relu'))
model.add(Dropout(0.5))
model.add(Dense(1, activation= 'sigmoid'))

model.compile(loss='binary_crossentropy',
				optimizer= 'rmsprop',
				metrics= ['accuracy'])

model.fit( train_data,train_labels,
		nb_epoch= 30,
		batch_size= 50,
		validation_data=(validation_data, validation_labels))
model.save_weights('bottleneck_fc_model.h5')
model_json = model.to_json()
with open("model_catsVSdogs_bottleNeck1.json", "w") as json_file:
	json_file.write(model_json)
	print("saved model to disk")

#save_bottleneck_features()
train_top_model()

I used h5 format to save the weights.

for testing, I created a test directory with 20 images

import h5py
import numpy as np
from keras.preprocessing.image import ImageDataGenerator
from keras.models import Sequential
from keras.layers import Activation, Dropout, Flatten, Dense
from keras.applications.vgg16 import VGG16
from keras.models import model_from_json

datagen= ImageDataGenerator(rescale= 1./255)
base_model= VGG16(weights='imagenet', include_top=False)
print("model loaded.")
generator = datagen.flow_from_directory(
'data/test',
target_size=(150, 150),
batch_size=20,
class_mode=None,
shuffle= True)

bottleneck_features_test= base_model.predict_generator(generator,20)
test_labels= np.array([0]*10+ [1]*10)

json_file = open('model_catsVSdogs_bottleNeck1.json', 'r')
model_json= json_file.read()
json_file.close()
model = model_from_json(model_json)

model.compile(loss='binary_crossentropy', optimizer='rmsprop', metrics=['accuracy'])
model.load_weights('bottleneck_fc_model.h5')
score = model.predict_classes(bottleneck_features_test).reshape((1,20))
print(score)
print(len(bottleneck_features_test))
print(test_labels)

@Wheele9
Wheele9 commented Mar 1, 2017 edited

I am getting an Error, when trying to runk the save_bottlebeck_features() function:

Model loaded.
Found 2000 images belonging to 2 classes.

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-9-1e8bd729719b> in <module>()
      1 
----> 2 save_bottlebeck_features()

<ipython-input-6-5c4d92c63b13> in save_bottlebeck_features()
     64             shuffle=False)
     65     bottleneck_features_train = model.predict_generator(generator, nb_train_samples)
---> 66     np.save(open('bottleneck_features_train.npy', 'w'), bottleneck_features_train)
     67 
     68     generator = datagen.flow_from_directory(

/home/matyi/miniconda3/envs/tf1/lib/python3.5/site-packages/numpy/lib/npyio.py in save(file, arr, allow_pickle, fix_imports)
    489         arr = np.asanyarray(arr)
    490         format.write_array(fid, arr, allow_pickle=allow_pickle,
--> 491                            pickle_kwargs=pickle_kwargs)
    492     finally:
    493         if own_fid:

/home/matyi/miniconda3/envs/tf1/lib/python3.5/site-packages/numpy/lib/format.py in write_array(fp, array, version, allow_pickle, pickle_kwargs)
    553     _check_version(version)
    554     used_ver = _write_array_header(fp, header_data_from_array_1_0(array),
--> 555                                    version)
    556     # this warning can be removed when 1.9 has aged enough
    557     if version != (2, 0) and used_ver == (2, 0):

/home/matyi/miniconda3/envs/tf1/lib/python3.5/site-packages/numpy/lib/format.py in _write_array_header(fp, d, version)
    326         raise ValueError(msg)
    327 
--> 328     fp.write(header_prefix)
    329     fp.write(header)
    330     return version


TypeError: write() argument must be str, not bytes

I am using python 3.5 with anaconda on ubuntu 16.04
tensorflow-gpu version 1.0.0
keras : 1.2.2
Theano version 0.8.2 , my keras.json file :
"backend": "tensorflow",
"epsilon": 1e-07,
"floatx": "float32",
"image_dim_ordering": "th"

Can someone help me with that?

@BrianMowrey

@Wheele9 It seems numpy changed the save/load methods to use a filename. You can change each np.save and np.load (2 of each) to something like

np.save('bottleneck_features_train.npy', bottleneck_features_train)
and
train_data = np.load('bottleneck_features_train.npy')

@galactica147

I've got an error running the program, can someone help why this happened?
i'm using keras 1.2.2, numpy1.12.0, tensorflow1.0.0

python classifier_from_little_data_script_2.py
Using TensorFlow backend.
Traceback (most recent call last):
  File "classifier_from_little_data_script_2.py", line 163, in <module>
    save_bottlebeck_features()
  File "classifier_from_little_data_script_2.py", line 81, in save_bottlebeck_features
    model.add(MaxPooling2D((2, 2), strides=(2, 2)))
  File "/path_to_python_virtualenv/lib/python2.7/site-packages/keras/models.py", line 332, in add
    output_tensor = layer(self.outputs[0])
  File "/path_to_python_virtualenv/lib/python2.7/site-packages/keras/engine/topology.py", line 572, in __call__
    self.add_inbound_node(inbound_layers, node_indices, tensor_indices)
  File "/path_to_python_virtualenv/lib/python2.7/site-packages/keras/engine/topology.py", line 635, in add_inbound_node
    Node.create_node(self, inbound_layers, node_indices, tensor_indices)
  File "/path_to_python_virtualenv/lib/python2.7/site-packages/keras/engine/topology.py", line 166, in create_node
    output_tensors = to_list(outbound_layer.call(input_tensors[0], mask=input_masks[0]))
  File "/path_to_python_virtualenv/lib/python2.7/site-packages/keras/layers/pooling.py", line 160, in call
    dim_ordering=self.dim_ordering)
  File "/path_to_python_virtualenv/lib/python2.7/site-packages/keras/layers/pooling.py", line 210, in _pooling_function
    pool_mode='max')
  File "/path_to_python_virtualenv/lib/python2.7/site-packages/keras/backend/tensorflow_backend.py", line 2866, in pool2d
    x = tf.nn.max_pool(x, pool_size, strides, padding=padding)
  File "/path_to_python_virtualenv/lib/python2.7/site-packages/tensorflow/python/ops/nn_ops.py", line 1793, in max_pool
    name=name)
  File "/path_to_python_virtualenv/lib/python2.7/site-packages/tensorflow/python/ops/gen_nn_ops.py", line 1598, in _max_pool
    data_format=data_format, name=name)
  File "/path_to_python_virtualenv/lib/python2.7/site-packages/tensorflow/python/framework/op_def_library.py", line 763, in apply_op
    op_def=op_def)
  File "/path_to_python_virtualenv/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 2329, in create_op
    set_shapes_for_outputs(ret)
  File "/path_to_python_virtualenv/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1717, in set_shapes_for_outputs
    shapes = shape_func(op)
  File "/path_to_python_virtualenv/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1667, in call_with_requiring
    return call_cpp_shape_fn(op, require_shape_fn=True)
  File "/path_to_python_virtualenv/lib/python2.7/site-packages/tensorflow/python/framework/common_shapes.py", line 610, in call_cpp_shape_fn
    debug_python_shape_fn, require_shape_fn)
  File "/path_to_python_virtualenv/lib/python2.7/site-packages/tensorflow/python/framework/common_shapes.py", line 675, in _call_cpp_shape_fn_impl
    raise ValueError(err.message)
ValueError: Negative dimension size caused by subtracting 2 from 1 for 'MaxPool_1' (op: 'MaxPool') with input shapes: [?,1,75,128].
@xiaokaisq

@galactica147 You need to change input_shape=(3, img_width, img_height) to input_shape=(img_width, img_height,3)

@xiaokaisq

@galactica147 You need to change input_shape=(3, img_width, img_height) to input_shape=(img_width, img_height,3)

@galactica147

@xiaokaisq, thank you for the tip, but now i ran into another error:

File "classifier_from_little_data_script_2.py", line 119, in save_bottlebeck_features
    model.layers[k].set_weights(weights)
ValueError: Layer weight shape (3, 3, 3, 64) not compatible with provided weight shape (64, 3, 3, 3)

Any ideas how to further fix this? (new to keras)

@galactica147

ok, I fixed the crash by this tip: fchollet/keras#3945

@suissemaxx
suissemaxx commented Mar 12, 2017 edited

@yurkor @daveselinger Thanks for your hints! Worked like a charm.

@yurkor I had to change the .reshape for Tensorflow to:

datagen.mean=np.array([103.939, 116.779, 123.68],dtype=np.float32).reshape(1,1,3)

Could improve the accuracy like you suggested.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment