Create a gist now

Instantly share code, notes, and snippets.

What would you like to do?
Updated to the Keras 2.0 API.
'''This script goes along the blog post
"Building powerful image classification models using very little data"
from blog.keras.io.
It uses data that can be downloaded at:
https://www.kaggle.com/c/dogs-vs-cats/data
In our setup, we:
- created a data/ folder
- created train/ and validation/ subfolders inside data/
- created cats/ and dogs/ subfolders inside train/ and validation/
- put the cat pictures index 0-999 in data/train/cats
- put the cat pictures index 1000-1400 in data/validation/cats
- put the dogs pictures index 12500-13499 in data/train/dogs
- put the dog pictures index 13500-13900 in data/validation/dogs
So that we have 1000 training examples for each class, and 400 validation examples for each class.
In summary, this is our directory structure:
```
data/
train/
dogs/
dog001.jpg
dog002.jpg
...
cats/
cat001.jpg
cat002.jpg
...
validation/
dogs/
dog001.jpg
dog002.jpg
...
cats/
cat001.jpg
cat002.jpg
...
```
'''
import numpy as np
from keras.preprocessing.image import ImageDataGenerator
from keras.models import Sequential
from keras.layers import Dropout, Flatten, Dense
from keras import applications
# dimensions of our images.
img_width, img_height = 150, 150
top_model_weights_path = 'bottleneck_fc_model.h5'
train_data_dir = 'data/train'
validation_data_dir = 'data/validation'
nb_train_samples = 2000
nb_validation_samples = 800
epochs = 50
batch_size = 16
def save_bottlebeck_features():
datagen = ImageDataGenerator(rescale=1. / 255)
# build the VGG16 network
model = applications.VGG16(include_top=False, weights='imagenet')
generator = datagen.flow_from_directory(
train_data_dir,
target_size=(img_width, img_height),
batch_size=batch_size,
class_mode=None,
shuffle=False)
bottleneck_features_train = model.predict_generator(
generator, nb_train_samples // batch_size)
np.save(open('bottleneck_features_train.npy', 'w'),
bottleneck_features_train)
generator = datagen.flow_from_directory(
validation_data_dir,
target_size=(img_width, img_height),
batch_size=batch_size,
class_mode=None,
shuffle=False)
bottleneck_features_validation = model.predict_generator(
generator, nb_validation_samples // batch_size)
np.save(open('bottleneck_features_validation.npy', 'w'),
bottleneck_features_validation)
def train_top_model():
train_data = np.load(open('bottleneck_features_train.npy'))
train_labels = np.array(
[0] * (nb_train_samples / 2) + [1] * (nb_train_samples / 2))
validation_data = np.load(open('bottleneck_features_validation.npy'))
validation_labels = np.array(
[0] * (nb_validation_samples / 2) + [1] * (nb_validation_samples / 2))
model = Sequential()
model.add(Flatten(input_shape=train_data.shape[1:]))
model.add(Dense(256, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(1, activation='sigmoid'))
model.compile(optimizer='rmsprop',
loss='binary_crossentropy', metrics=['accuracy'])
model.fit(train_data, train_labels,
epochs=epochs,
batch_size=batch_size,
validation_data=(validation_data, validation_labels))
model.save_weights(top_model_weights_path)
save_bottlebeck_features()
train_top_model()

I really love this example. I had some problems getting it running on python 3.5, but everything ran smooth on 2.7. Great work.

I get an error at the point where it starts train_top_model():, at this line: train_data = np.load(open('bottleneck_features_train.npy'))

ValueError was unhandled by user code
Message: total size of new array must be unchanged

It seems to suggest that the numpy array saved in the save_bottlebeck_features(): has somehow changed?

How to get the labels when training more than two categories? Is it possible to infer the labels from the generator?

I'm getting an error in the line datagen = ImageDataGenerator(rescale=1./255). Error is TypeError: __init__() got an unexpected keyword argument 'rescale'.

More generally, I get a similar unexpected keyword error if I uncomment any of the commented lines in the following, which is taken from the ImageDataGenerator documentation.

from keras.preprocessing.image import ImageDataGenerator
datagen = ImageDataGenerator(featurewise_center=False,
    samplewise_center=False,
    featurewise_std_normalization=False,
    samplewise_std_normalization=False,
    zca_whitening=False,
    rotation_range=0.,
    width_shift_range=0.,
    height_shift_range=0.,
#     shear_range=0.,
#     zoom_range=0.,
#     channel_shift_range=0.,
#     fill_mode='nearest',
#     cval=0.,
#     rescale=None,
#     dim_ordering=False,
    horizontal_flip=False,
    vertical_flip=False)

I'm using Python 2.7 with Tensor Flow backend. Appreciate any help.

(Francois, these gists and your blog posts have been really helpful to me, thanks for taking the time to share them!)

assaflehr commented Jul 6, 2016 edited

@jdelange To solve the ValueError

ValueError was unhandled by user code
Message: total size of new array must be unchanged

If you are using Windows platform, you must use the binary mode of files when saving numpy array, so just add the char 'b' to the mode, like this:
np.save(open(file_name, 'wb'), saved)
loaded = np.load(open(file_name,'rb'))

This weird non cross-platform python behavior explained here

It's awesome!Thank you so much

yurkor commented Jul 10, 2016

Ideally we want use original VGG image prepossessing mean and no scale
i.e.

datagen = ImageDataGenerator(rescale=1., featurewise_center=True) #(rescale=1./255)
datagen.mean=np.array([103.939, 116.779, 123.68],dtype=np.float32).reshape(3,1,1)

it gives final accuracy 95 instead of 90
Epoch 3/10
0s - loss: 1.3352 - acc: 0.9100 - val_loss: 0.7913 - val_acc: 0.9470

Epoch 3/10
0s - loss: 0.3059 - acc: 0.8725 - val_loss: 0.2171 - val_acc: 0.9140

mawangyi commented Sep 1, 2016 edited

I really love this example. Thank You Very Much!

mawangyi commented Sep 5, 2016 edited

How do I use this model to predict classification
my code:

weights_path = 'vgg16_weights.h5'
top_model_weights_path = 'bottleneck_fc_model.h5'
img_width, img_height = 150, 150

def VGG16_bottlebeck_features():
input_shape = (3, 150, 150)
img_input = Input(shape=input_shape)

x = Flatten(name='flatten')(img_input)
x = Dense(256, activation='relu', name='fc1')(x)
x = Dense(1, activation='sigmoid', name='predictions')(x)

model = Model(img_input, x)
model.load_weights("bottleneck_fc_model.h5")
return model

if name == 'main':
model = VGG16_bottlebeck_features()
img_path = 'image/cat.jpg'
img = image.load_img(img_path, target_size=(150, 150))
x = image.img_to_array(img)

x = np.expand_dims(x, axis=0)
x = preprocess_input(x)

preds = model.predict(x)

print('Predicted:', decode_predictions(preds))

And ERROR:
Traceback (most recent call last):
File "vgg16.py", line 194, in
preds = model.predict(x)
File "/usr/local/lib/python2.7/site-packages/keras/engine/training.py", line 1177, in predict
batch_size=batch_size, verbose=verbose)
File "/usr/local/lib/python2.7/site-packages/keras/engine/training.py", line 876, in _predict_loop
batch_outs = f(ins_batch)
File "/usr/local/lib/python2.7/site-packages/keras/backend/theano_backend.py", line 672, in call
return self.function(*inputs)
File "/usr/local/lib/python2.7/site-packages/theano/compile/function_module.py", line 871, in call
storage_map=getattr(self.fn, 'storage_map', None))
File "/usr/local/lib/python2.7/site-packages/theano/gof/link.py", line 314, in raise_with_op
reraise(exc_type, exc_value, exc_trace)
File "/usr/local/lib/python2.7/site-packages/theano/compile/function_module.py", line 859, in call
outputs = self.fn()
File "/usr/local/lib/python2.7/site-packages/theano/gof/op.py", line 912, in rval
r = p(n, [x[0] for x in i], o)
File "/usr/local/lib/python2.7/site-packages/theano/tensor/blas.py", line 1552, in perform
z[0] = numpy.asarray(numpy.dot(x, y))
ValueError: ('shapes (1,67500) and (8192,256) not aligned: 67500 (dim 1) != 8192 (dim 0)', (1, 67500), (8192, 256))
Apply node that caused the error: Dot22(Reshape{2}.0, fc1_W)
Toposort index: 11
Inputs types: [TensorType(float32, matrix), TensorType(float32, matrix)]
Inputs shapes: [(1, 67500), (8192, 256)]
Inputs strides: [(270000, 4), (1024, 4)]
Inputs values: ['not shown', 'not shown']
Outputs clients: [[Elemwise{Composite{(i0 * (Abs((i1 + i2)) + i1 + i2))}}[(0, 1)](TensorConstant{%281, 1%29 of 0.5}, Dot22.0, InplaceDimShuffle{x,0}.0)]]

HINT: Re-running with most Theano optimization disabled could give you a back-trace of when this node was created. This can be done with by setting the Theano flag 'optimizer=fast_compile'. If that does not work, Theano optimizations can be disabled with 'optimizer=None'.
HINT: Use the Theano flag 'exception_verbosity=high' for a debugprint and storage map footprint of this apply node.

I really love this simple example. But I got a problem now. Can anyone tell me how to make prediction? Here is what I did.
I use a picture of cat. Input it to the VGG model (bottle neck one). I got a feature. Then load the feature to the trained model (the added one). But got wrong result. The prediction is always 0 (the first one).

load json and create model

json_file = open('model_vgg.json', 'r')
loaded_model_json = json_file.read()
json_file.close()
model1 = model_from_json(loaded_model_json)

load weights into new model

assert os.path.exists(weights_path), 'Model weights not found (see "weights_path" variable in script).'
f = h5py.File(weights_path)

for k in range(f.attrs['nb_layers']):
if k >= len(model1.layers):
# we don't look at the last (fully-connected) layers in the savefile
break
g = f['layer_{}'.format(k)]
weights = [g['param_{}'.format(p)] for p in range(g.attrs['nb_params'])]
model1.layers[k].set_weights(weights)
f.close()
print('Model VGG loaded.')

load json and create model

json_file = open('model_catdog.json', 'r')
loaded_model_json = json_file.read()
json_file.close()
model2 = model_from_json(loaded_model_json)

load weights into new model

model2.load_weights("bottleneck_fc_model.h5")
print('Model CAT DOG loaded.')

img = cv2.imread("dog.1000.jpg")
img = cv2.resize(img, (150, 150)).astype(np.float32) / 255.
print img
img = img.reshape(1 , img.shape[2], img.shape[0], img.shape[1])

feature = model1.predict(img)
probs = model2.predict(feature)
print "probs: ", probs
prediction = probs.argmax(axis=1)
print "prediction: ", prediction

srikar2097 commented Sep 24, 2016 edited

I have done exactly what this code does, by the end of 50th epoch I am getting high training accuracy but ≈ 50% validation accuracy. In fact the validation accuracy hovers around 50% throughout. See the last 3 epochs. See the non-changing val_loss and val_acc. What am i doing wrong?

Epoch 48/50
10000/10000 [==============================] - 2s - loss: 0.3180 - acc: 0.9131 - val_loss: 7.9712 - val_acc: 0.5000
Epoch 49/50
10000/10000 [==============================] - 2s - loss: 0.2903 - acc: 0.9172 - val_loss: 7.9712 - val_acc: 0.5000
Epoch 50/50
10000/10000 [==============================] - 2s - loss: 0.3272 - acc: 0.9132 - val_loss: 7.9712 - val_acc: 0.5000

Facts -
0. followed the post as mentioned here. This is to get a model with pre-trained Image-net bottleneck features.

  1. I have used the above code by François.
  2. Got VGG16 pre-trained weights from here. As mentioned in post.
  3. I did not see a pre-processing step where Image is zero-centered around mean. I tried with and without this step but no change in validation performance.
  4. Using Theano backend on a GPU.

What am i doing wrong?

paho67 commented Sep 29, 2016

I also wonder, how to make predictions.
I tried to load saved model and then give it to my own image:
...
img = load_img(img_path,target_size=(img_width,img_height))
x = img_to_array(img)
x = np.expand_dims(x, axis=0)
x = preprocess_input(x)
model = load_model('.\models' + model_name + '.h5')
preds = model.predict_classes(x)
prob = model.predict_proba(x)

But I got error:
Exception: Error when checking : expected flatten_input_1 to have shape (None, 512, 4, 4) but got array with shape (1, 3, 150, 150)

xcmax commented Sep 29, 2016

Question:how to use this to multiclass classification? In order to apply it to multiclass problem, I changed the loss function t'binary_crossentropy' to 'categorical_crossentropy', but the acc could only reach 50% or so? I used 700 images for training 69 classes.

@srikar2097, did you find out a solution? Facing the same issue.

I carefully followed the given procedure given both at blog and here. Unfortunately, I am getting the following error which I couldn't find a way to solve:
Traceback (most recent call last):
File "", line 1, in
File "/usr/lib/python2.7/dist-packages/spyderlib/widgets/externalshell/sitecustomize.py", line 540, in runfile
execfile(filename, namespace)
File "/home/santez/Desktop/DL Examples/KERAS/Object Recognition/Classifier From Little Data KeraBlog/classifierfromlittledata3.py", line 118, in
model.layers[k].set_weights(weights)
File "/usr/local/lib/python2.7/dist-packages/keras/engine/topology.py", line 889, in set_weights
'provided weight shape ' + str(w.shape))
Exception: Layer weight shape (3, 3, 150, 64) not compatible with provided weight shape (64, 3, 3, 3)

@paho67 I am also facing the same issue

@paho67 @AakashKumarNain
I ran into the same issue and documented the fix here.

When I do the prediction, it looks like this:

img = load_img('my_img.jpg',False, (img_width, img_height))
x = img_to_array(img)
prediction = model.predict(x.reshape((1,3,img_width, img_height)),batch_size=32, verbose=0)
print(prediction)

I am not sure this is 100% correct since I am not getting accurate predictions, but I am getting a prediction nonetheless.

@srikar2097, I found the reason for that issue. flow_from_directory selects from one folder at a time and as there are more then 2000 images in the first folder (13k for me) it just takes images from one folder/class but still pretends like the val data is half from one class half from the other. See line 145.

I'm getting the following error for line 79:

ValueError: Filter must not be larger than the input: Filter: (2, 2) Input: (1, 75)

Hi everybody!
I think I have confronted with most of the dimension errors you wrote and probably you have to change the configuration of the ~/keras/keras.json file as following:
{
"image_dim_ordering": "th",
"epsilon": 1e-07,
"floatx": "float32",
"backend": "tensorflow"
}

paho67 commented Oct 13, 2016

@jessicaowensby I think that this is not solution for my problem.
I guess that loaded model assume that input have to be this type:
model.add(Flatten(input_shape=train_data.shape[1:])) <=> which is (None, 512, 4, 4).

I wonder how to change this that way that I could give it 150x150 image (array with shape (1, 3, 150, 150))?

@bayraktare Thanks for this fix!

daveselinger commented Oct 22, 2016 edited

@srikar2097 and @shackenberg So the specific way to fix this is to enter the correct number in nb_train_samples and nb_validation_samples.

@bayraktare Thanks--that is a better way than what I'm doing which is modifying every file with:

from keras import backend as K
K.set_image_dim_ordering('th')

FINAL NOTE: it appears that numpy has changed its interface so that np.save now takes a filename. The np.save appears to add '.npy' to the end (whereas load does not), so instead of opening the file, you have to use this interface:
np.save("example") followed by:
np.load("example.npy")

I was getting: TypeError: must be str, not bytes

edmundronald commented Oct 25, 2016 edited

I'm just going from fixing one bug to the next and not getting the code to work. With the installation etc I am easily at 20h work now (Vagrant Mac) installing one package after the other and fixing things. I don't think the code is fixable with Theano, I am now going to install tf, but am growing tired. Would it be possible to make a VM which just runs and has a complete set of everything? I would be willing to work on that. I have a PhD in AI, and I'm 60 years old, but this is situation is more for a 15 year old with time on his hands and knowledge of Python version issues.

Here is the error I am now getting
RuntimeError: BaseCorrMM: Failed to allocate output of 32 x 64 x 150 x 150
Apply node that caused the error: CorrMM{valid, (1, 1)}(IncSubtensor{InplaceSet;::, ::, int64:int64:, int64:int64:}.0, Subtensor{::, ::, ::int64, ::int64}.0)
Toposort index: 55
Inputs types: [TensorType(float32, 4D), TensorType(float32, 4D)]
Inputs shapes: [(32, 3, 152, 152), (64, 3, 3, 3)]
Inputs strides: [(277248, 92416, 608, 4), (108, 36, -12, -4)]
Inputs values: ['not shown', 'not shown']
Outputs clients: [[Elemwise{Composite{(i0 * (Abs((i1 + i2)) + i1 + i2))}}[(0, 1)](TensorConstant{%281, 1, 1, 1%29 of 0.5}, CorrMM{valid, %281, 1%29}.0, Reshape{4}.0)]]

@edmundronald : I think you have an installation problem. I use Ubuntu 16.04, Python3.5, Tensorfow and the only modification I have to made was changing from "tf" to "th" in ~/.keras/keras.json
And made the script run under Python3.5 with small modifications : https://gist.github.com/trancept/010665252efcdffc5d886325b36fc951

@xcmax (or anyone who attempted applying the code for multiple classes) did you figure out how to make it work for multi-class? Thanks

I am getting the following error

Using TensorFlow backend.
Model loaded.
Found 573 images belonging to 2 classes.
Traceback (most recent call last):
File "with_vgg16_weights.py", line 144, in
save_bottlebeck_features()
File "with_vgg16_weights.py", line 90, in save_bottlebeck_features
bottleneck_features_train = model.predict_generator(generator, nb_train_samples)
File "/Users/ml/lib/python3.5/site-packages/keras/models.py", line 945, in predict_generator
pickle_safe=pickle_safe)
File "/Users/ml/lib/python3.5/site-packages/keras/engine/training.py", line 1655, in predict_generator
all_outs[i][processed_samples:(processed_samples + nb_samples)] = out
ValueError: could not broadcast input array from shape (32,512,4,4) into shape (10,512,4,4)

This seems to be a numpy error not a keras error. My settings are exactly like the example except the number of training and validation examples. Has anyone else encountered this before?
Thanks

tlind commented Nov 23, 2016 edited

@ManasHardas I am having exactly the same issue on my own dataset. Did you find a solution?

The problem seems to be related to nb_train_samples not being a multiple of batch_size. If I increase nb_train_samples to a larger number (without actually adding more training images), I guess the labels would end up being incorrect because the assumption is that the first nb_train_samples/2 samples are of the first class, and the second nb_train_samples/2 are of the second class.

What I don't understand is why the original code was supposed to work like this, since 2000 is also not divisible by 32. Maybe something changed in Keras internally?

tlind commented Nov 24, 2016

Problem solved! Both my nb_train_samples and nb_validation_samples were set incorrectly (I had to subtract 2 from both). With the correct number of samples set, the issue has disappeared. I think while I was counting the number of files inside the train and validation directories, my file browser was also considering the folders of the two classes as extra items. The count provided by ImageDataGenerator in the console ("Found 5946 images belonging to 2 classes.") was correct, though.

ManasHardas commented Nov 25, 2016 edited

@tlind I am still getting error. nb_train_samples=600 and nb_validation_samples=100 and batch_size=32. This means I have 300 images of each class in a folder for training and 50 images for each class for validation. Now I am getting this error.

Using TensorFlow backend.
Model loaded.
Found 600 images belonging to 2 classes.
Traceback (most recent call last):
File "with_vgg16_weights.py", line 144, in
save_bottlebeck_features()
File "with_vgg16_weights.py", line 95, in save_bottlebeck_features
bottleneck_features_train
File "/Users/manas/ml/lib/python3.5/site-packages/numpy/lib/npyio.py", line 491, in save
pickle_kwargs=pickle_kwargs)
File "/Users/manas/ml/lib/python3.5/site-packages/numpy/lib/format.py", line 555, in write_array
version)
File "/Users/manas/ml/lib/python3.5/site-packages/numpy/lib/format.py", line 328, in _write_array_header
fp.write(header_prefix)
TypeError: write() argument must be str, not bytes
Exception ignored in: <bound method BaseSession.del of <tensorflow.python.client.session.Session object at 0x111c13da0>>
Traceback (most recent call last):
File "/Users/manas/ml/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 532, in del
AttributeError: 'NoneType' object has no attribute 'TF_DeleteStatus'

@davidpickup I think the errors might be due to a tensorflow/theano issue. It took me a while, but I put together a notebook that runs all the code with a tensorflow backend. Its available here: https://github.com/rajshah4/image_keras

I resolved --- TypeError: write() argument must be str, not bytes --- by following @daveselinger suggestion. Create model without .npy extension
np.save("example") np.load("example.npy")

For prediction I am doing the following
top_models_weight_path = 'bottleneck_fc_model.h5'
img = image.load_img(test_img, target_size=(img_width, img_height))
x = image.img_to_array(img)
x = np.expand_dims(x, axis=0)
x = preprocess_input(x)
model = load_weights(top_models_weight_path)
class_preds = model.predict_classes(x)
prob_preds = model.predict_proba(x)

But getting this error
Using TensorFlow backend.
Traceback (most recent call last):
File "predict.py", line 27, in <module> model = load_model(top_model_weights_path)
File "/Users/ml/lib/python3.5/site-packages/keras/models.py", line 126, in load_model raise ValueError('No model found in config file.')
ValueError: No model found in config file.

@rajivshah4 - were you able to load the classifier layer weights and actually predict the class for a single test image?

Don't you have to pass the test images to the same generator if you want to predict on them, like the train and validation images. So you get the (None, 512L, 2L, 2L) shape. Like:

generator = datagen.flow(
X_test,
batch_size=32,

        shuffle=False)
bottleneck_features_validation = model.predict_generator(generator, len(X_test))
np.save('bottleneck_features_test.npy', bottleneck_features_validation)

test = np.load('bottleneck_features_test.npy')

predictions = model.predict(test, verbose=1)

sulaimanvesal commented Dec 8, 2016 edited

@MayorPain - I have created as follow the generator and send through the convolutional layer as follow:

generator = datagen.flow_from_directory(
test_data_dir,
batch_size=32,
target_size=(img_width, img_height),
class_mode=None,
shuffle=False)
test_data_features = model.predict_generator(generator, 12500)
np.save(open('test_data_features.npy', 'w'), test_data_features)

My question is that when this generator searches test-folder, it ask for a subfolder and count that subfolder as a class. I have run my program and everything is smooth but during the predication accuracy is really low. Although I have trained over entire Cat-&-Dog dataset with 50 Epochs and my validation accuracy for train data and validation data reaches to 99.2 %. But when I predict the test data it's really worse.

Would you please share what is wrong with my test-images or the generator provide some ambiguity problem.

Thanks for the tutorial , its nice one.
I need help to predict an image. I try everything and nothing works. :(

I get this error:
Exception: Error when checking : expected flatten_input_1 to have shape (None, 512, 4, 4) but got array with shape (1, 3, 150, 150)

@bluelight773
@xcmax
did you guys (or anyone who attempted applying the code for multiple classes) figure out how to make it work for multi-class? Thanks

austinchencym commented Dec 28, 2016 edited

I got

File "data/Classifier2.py", line 100, in train_top_model
train_data = np.load(open('bottleneck_features_train.npy'))

UnicodeDecodeError: 'charmap' codec can't decode byte 0x8d in position 142: character maps to undefined

did any one have the same issue and get solved?>

dzorlu commented Jan 7, 2017

For multiclass prediction you need to convert your labels via to_categorical. For more info see this:

http://stackoverflow.com/questions/36927025/how-to-use-keras-multi-layer-perceptron-for-multi-class-classification

shan4224 commented Jan 9, 2017 edited

I am getting some errors while predicting test data. How to predict the images for test data in Using the bottleneck features of a pre-trained network section.

I have tried the following :

test_data_dir = '/Dogs_Cats/data/test' ## There are 100 imagesin test data
datagen = ImageDataGenerator(rescale=1., featurewise_center=True)
datagen.mean=np.array([103.939, 116.779, 123.68],dtype=np.float32).reshape(3,1,1)
generator = datagen.flow_from_directory(
test_data_dir,
target_size=(img_width, img_height),
batch_size=32,
shuffle=False,
classes = None)

test_data_features = model.predict_generator(generator, 100)
np.save(open('/Dogs_Cats/data/test_data_features.npy','wb'), test_data_features)
test_data =np.load(open('/Dogs_Cats/data/test_data_features.npy','rb'))
predictions = model.predict_classes(test_data, verbose=1)

I am getting errors:
Exception: Error when checking : expected flatten_input_5 to have shape (None, 512, 4, 4) but got array with shape (100, 3, 150, 150)

@shan4224, @paho67

I am getting errors:
Exception: Error when checking : expected flatten_input_5 to have shape (None, 512, 4, 4) but got array with shape (100, 3, 150, 150)

that's why our little 2-layers network is not specified for getting pictures as input.
We need to add these two layers to the existing 13 ones and to compile ALL the 15-layers network.
If you look at the third part of the tutorial, you will understand how to do it

Hi Folks,

Anyone know the reason why I get loss : nan for some end epochs . I was trying this code with a multiclass classification

Epoch 44/50
3021/3021 [==============================] - 8s - loss: 0.0414 - acc: 0.9841 - val_loss: 0.5106 - val_acc: 0.9346
Epoch 45/50
3021/3021 [==============================] - 11s - loss: 0.0402 - acc: 0.9864 - val_loss: 0.5089 - val_acc: 0.9432
Epoch 46/50
3021/3021 [==============================] - 9s - loss: nan - acc: 0.7306 - val_loss: nan - val_acc: 0.4247
Epoch 47/50
3021/3021 [==============================] - 10s - loss: nan - acc: 0.4551 - val_loss: nan - val_acc: 0.4247
Epoch 48/50
3021/3021 [==============================] - 9s - loss: nan - acc: 0.4551 - val_loss: nan - val_acc: 0.4247
Epoch 49/50
3021/3021 [==============================] - 11s - loss: nan - acc: 0.4551 - val_loss: nan - val_acc: 0.4247
Epoch 50/50
3021/3021 [==============================] - 11s - loss: nan - acc: 0.4551 - val_loss: nan - val_acc: 0.4247

Thanks,
Biswa

al3xsh commented Jan 26, 2017 edited

Hi,

First of all, thanks for a great tutorial! I have only just discovered keras and this example has shown me how simple and powerful development using the keras framework can be.

I have followed the tutorial (and made the modification to use the published vgg16 image preprocessing means suggested by @yurkor) but i seem to get significantly different results using the theano backend and the tensorflow backend - does this happen to anybody else?

Using the vgg16 image preprocessing means and the theano backend I get a validation accuracy of ~95% after 50 epochs, but using the tensorflow backend the validation accuracy is between 88 - 90% after 50 epochs.

Without the vgg16 image preprocessing means the accuracy is around 90% for the theano backend and 86% for tensorflow.

Any ideas why theano seems to outperform tensorflow? I thought that they would be almost identical.

Regards,

Alex

al3xsh commented Jan 26, 2017

Follow up:

If I use the vgg16 example with imagenet weights provided with keras:

from keras.applications.vgg16 import VGG16
model = VGG16(weights='imagenet', include_top=False)

to do the feature extraction and then build the classifier on top of that (as in this tutorial), then I get the same accuracy using the tensorflow backend as with using the theano backend. I assume I'm messing up the conversion of the vgg 16 weights from theano format to tensorflow format. At the moment I am doing:

        # the saved weights are in theano format and we are using the 
        # tensorflow backend so we need to manually reorder them because 
        # tensorflow implements convolution differently ...
        if layer.__class__.__name__ in ['Convolution1D', 
                                        'Convolution2D', 
                                        'Convolution3D', 
                                        'AtrousConvolution2D']:
            weights[0] = np.transpose(weights[0], (2, 3, 1, 0))

which came from:
https://gist.github.com/baraldilorenzo/07d7802847aaad0a35d3#gistcomment-1905315

I want to use something like convert_kernel - but am having difficulty going from the h5 file to the numpy array.

Any ideas how I should be doing it?

Regards,

Alex

rk-ka commented Jan 29, 2017

TypeError: can't multiply sequence by non-int of type 'float'

TypeError Traceback (most recent call last)
in ()
169
170 save_bottlebeck_features()
--> 171 train_top_model()

in train_top_model()
149 def train_top_model():
150 train_data = np.load('C:/toolkits/data/model/bottleneck/bottleneck_features_train.npy')
--> 151 train_labels = np.array([0] * (nb_train_samples / 2) + [1] * (nb_train_samples / 2))
152
153 validation_data = np.load('C:/toolkits/data/model/bottleneck/bottleneck_features_validation.npy')

TypeError: can't multiply sequence by non-int of type 'float'


Help please

@rk-ka Are you running this under Python 3.5? I had this exact issue when I ran it under Python 3.5, however I was able to find a solution to the problem.

Change the following line:
train_labels = np.array([0] * (nb_train_samples / 2) + [1] * (nb_train_samples / 2))

To:
train_labels = np.array([0] * (nb_train_samples // 2) + [1] * (nb_train_samples // 2))

And also change the following line:
validation_labels = np.array([0] * (nb_validation_samples / 2) + [1] * (nb_validation_samples / 2))

To:
validation_labels = np.array([0] * (nb_validation_samples // 2) + [1] * (nb_validation_samples // 2))

tovalinder commented Feb 9, 2017 edited

@srikar2097 I had the same problem. My problem was not the number of images in the folders, however removing the file type in the filename solved the problem, see example below.

np.save(open('bottleneck_features_train.npy', 'wb'), bottleneck_features_train)
changed into
np.save(open('bottleneck_features_train', 'wb'), bottleneck_features_train)

Loading the bottleneck features with train_data = np.load(open('bottleneck_features_train', 'rb'))

@rk-ka Changing the second argument to open() into 'wb' for writing and 'rb' for reading (as in the example above) also solves the TypeError.

Here is my code for building the model

import h5py
import numpy as np
from keras.preprocessing.image import ImageDataGenerator
from keras.models import Sequential
from keras.layers import Convolution2D, MaxPooling2D, ZeroPadding2D
from keras.layers import Activation, Dropout, Flatten, Dense
from keras.applications.vgg16 import VGG16
from keras.models import model_from_json

def save_bottleneck_features():
datagen= ImageDataGenerator(rescale= 1./255)

base_model= VGG16(weights='imagenet', include_top=False)
print("model loaded.")

generator = datagen.flow_from_directory(
	'data/train',
	target_size=(150, 150),
	batch_size=32,
	class_mode=None,
	shuffle=False

)
bottleneck_features_train= base_model.predict_generator(generator,2000)

generator = datagen.flow_from_directory(
	'data/valid',
	target_size=(150, 150),
	batch_size=32,
	class_mode=None,
	shuffle=False)


bottleneck_features_validation= base_model.predict_generator(generator,800)

h5f= h5py.File('catsVSdogs_bottleNeck1.h5', 'w')
h5f.create_dataset('bottleneck_features_train', data=bottleneck_features_train)
h5f.create_dataset('bottleneck_features_validation', data=bottleneck_features_validation)
h5f.close()

def train_top_model():

h5f= h5py.File('catsVSdogs_bottleNeck1.h5', 'r')
train_data= h5f['bottleneck_features_train'][:]
train_labels= np.array([0]*1000+ [1]*1000)

validation_data= h5f['bottleneck_features_validation'][:]
validation_labels= np.array([0]*400+ [1]*400)
h5f.close()

model= Sequential()
model.add(Flatten(input_shape= train_data.shape[1:]))
model.add(Dense(256, activation= 'relu'))
model.add(Dropout(0.5))
model.add(Dense(1, activation= 'sigmoid'))

model.compile(loss='binary_crossentropy',
				optimizer= 'rmsprop',
				metrics= ['accuracy'])

model.fit( train_data,train_labels,
		nb_epoch= 30,
		batch_size= 50,
		validation_data=(validation_data, validation_labels))
model.save_weights('bottleneck_fc_model.h5')
model_json = model.to_json()
with open("model_catsVSdogs_bottleNeck1.json", "w") as json_file:
	json_file.write(model_json)
	print("saved model to disk")

#save_bottleneck_features()
train_top_model()

I used h5 format to save the weights.

for testing, I created a test directory with 20 images

import h5py
import numpy as np
from keras.preprocessing.image import ImageDataGenerator
from keras.models import Sequential
from keras.layers import Activation, Dropout, Flatten, Dense
from keras.applications.vgg16 import VGG16
from keras.models import model_from_json

datagen= ImageDataGenerator(rescale= 1./255)
base_model= VGG16(weights='imagenet', include_top=False)
print("model loaded.")
generator = datagen.flow_from_directory(
'data/test',
target_size=(150, 150),
batch_size=20,
class_mode=None,
shuffle= True)

bottleneck_features_test= base_model.predict_generator(generator,20)
test_labels= np.array([0]*10+ [1]*10)

json_file = open('model_catsVSdogs_bottleNeck1.json', 'r')
model_json= json_file.read()
json_file.close()
model = model_from_json(model_json)

model.compile(loss='binary_crossentropy', optimizer='rmsprop', metrics=['accuracy'])
model.load_weights('bottleneck_fc_model.h5')
score = model.predict_classes(bottleneck_features_test).reshape((1,20))
print(score)
print(len(bottleneck_features_test))
print(test_labels)

Wheele9 commented Mar 1, 2017 edited

I am getting an Error, when trying to runk the save_bottlebeck_features() function:

Model loaded.
Found 2000 images belonging to 2 classes.

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-9-1e8bd729719b> in <module>()
      1 
----> 2 save_bottlebeck_features()

<ipython-input-6-5c4d92c63b13> in save_bottlebeck_features()
     64             shuffle=False)
     65     bottleneck_features_train = model.predict_generator(generator, nb_train_samples)
---> 66     np.save(open('bottleneck_features_train.npy', 'w'), bottleneck_features_train)
     67 
     68     generator = datagen.flow_from_directory(

/home/matyi/miniconda3/envs/tf1/lib/python3.5/site-packages/numpy/lib/npyio.py in save(file, arr, allow_pickle, fix_imports)
    489         arr = np.asanyarray(arr)
    490         format.write_array(fid, arr, allow_pickle=allow_pickle,
--> 491                            pickle_kwargs=pickle_kwargs)
    492     finally:
    493         if own_fid:

/home/matyi/miniconda3/envs/tf1/lib/python3.5/site-packages/numpy/lib/format.py in write_array(fp, array, version, allow_pickle, pickle_kwargs)
    553     _check_version(version)
    554     used_ver = _write_array_header(fp, header_data_from_array_1_0(array),
--> 555                                    version)
    556     # this warning can be removed when 1.9 has aged enough
    557     if version != (2, 0) and used_ver == (2, 0):

/home/matyi/miniconda3/envs/tf1/lib/python3.5/site-packages/numpy/lib/format.py in _write_array_header(fp, d, version)
    326         raise ValueError(msg)
    327 
--> 328     fp.write(header_prefix)
    329     fp.write(header)
    330     return version


TypeError: write() argument must be str, not bytes

I am using python 3.5 with anaconda on ubuntu 16.04
tensorflow-gpu version 1.0.0
keras : 1.2.2
Theano version 0.8.2 , my keras.json file :
"backend": "tensorflow",
"epsilon": 1e-07,
"floatx": "float32",
"image_dim_ordering": "th"

Can someone help me with that?

@Wheele9 It seems numpy changed the save/load methods to use a filename. You can change each np.save and np.load (2 of each) to something like

np.save('bottleneck_features_train.npy', bottleneck_features_train)
and
train_data = np.load('bottleneck_features_train.npy')

I've got an error running the program, can someone help why this happened?
i'm using keras 1.2.2, numpy1.12.0, tensorflow1.0.0

python classifier_from_little_data_script_2.py
Using TensorFlow backend.
Traceback (most recent call last):
  File "classifier_from_little_data_script_2.py", line 163, in <module>
    save_bottlebeck_features()
  File "classifier_from_little_data_script_2.py", line 81, in save_bottlebeck_features
    model.add(MaxPooling2D((2, 2), strides=(2, 2)))
  File "/path_to_python_virtualenv/lib/python2.7/site-packages/keras/models.py", line 332, in add
    output_tensor = layer(self.outputs[0])
  File "/path_to_python_virtualenv/lib/python2.7/site-packages/keras/engine/topology.py", line 572, in __call__
    self.add_inbound_node(inbound_layers, node_indices, tensor_indices)
  File "/path_to_python_virtualenv/lib/python2.7/site-packages/keras/engine/topology.py", line 635, in add_inbound_node
    Node.create_node(self, inbound_layers, node_indices, tensor_indices)
  File "/path_to_python_virtualenv/lib/python2.7/site-packages/keras/engine/topology.py", line 166, in create_node
    output_tensors = to_list(outbound_layer.call(input_tensors[0], mask=input_masks[0]))
  File "/path_to_python_virtualenv/lib/python2.7/site-packages/keras/layers/pooling.py", line 160, in call
    dim_ordering=self.dim_ordering)
  File "/path_to_python_virtualenv/lib/python2.7/site-packages/keras/layers/pooling.py", line 210, in _pooling_function
    pool_mode='max')
  File "/path_to_python_virtualenv/lib/python2.7/site-packages/keras/backend/tensorflow_backend.py", line 2866, in pool2d
    x = tf.nn.max_pool(x, pool_size, strides, padding=padding)
  File "/path_to_python_virtualenv/lib/python2.7/site-packages/tensorflow/python/ops/nn_ops.py", line 1793, in max_pool
    name=name)
  File "/path_to_python_virtualenv/lib/python2.7/site-packages/tensorflow/python/ops/gen_nn_ops.py", line 1598, in _max_pool
    data_format=data_format, name=name)
  File "/path_to_python_virtualenv/lib/python2.7/site-packages/tensorflow/python/framework/op_def_library.py", line 763, in apply_op
    op_def=op_def)
  File "/path_to_python_virtualenv/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 2329, in create_op
    set_shapes_for_outputs(ret)
  File "/path_to_python_virtualenv/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1717, in set_shapes_for_outputs
    shapes = shape_func(op)
  File "/path_to_python_virtualenv/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1667, in call_with_requiring
    return call_cpp_shape_fn(op, require_shape_fn=True)
  File "/path_to_python_virtualenv/lib/python2.7/site-packages/tensorflow/python/framework/common_shapes.py", line 610, in call_cpp_shape_fn
    debug_python_shape_fn, require_shape_fn)
  File "/path_to_python_virtualenv/lib/python2.7/site-packages/tensorflow/python/framework/common_shapes.py", line 675, in _call_cpp_shape_fn_impl
    raise ValueError(err.message)
ValueError: Negative dimension size caused by subtracting 2 from 1 for 'MaxPool_1' (op: 'MaxPool') with input shapes: [?,1,75,128].

@galactica147 You need to change input_shape=(3, img_width, img_height) to input_shape=(img_width, img_height,3)

@galactica147 You need to change input_shape=(3, img_width, img_height) to input_shape=(img_width, img_height,3)

@xiaokaisq, thank you for the tip, but now i ran into another error:

File "classifier_from_little_data_script_2.py", line 119, in save_bottlebeck_features
    model.layers[k].set_weights(weights)
ValueError: Layer weight shape (3, 3, 3, 64) not compatible with provided weight shape (64, 3, 3, 3)

Any ideas how to further fix this? (new to keras)

ok, I fixed the crash by this tip: fchollet/keras#3945

suissemaxx commented Mar 12, 2017 edited

@yurkor @daveselinger Thanks for your hints! Worked like a charm.

@yurkor I had to change the .reshape for Tensorflow to:

datagen.mean=np.array([103.939, 116.779, 123.68],dtype=np.float32).reshape(1,1,3)

Could improve the accuracy like you suggested.

Maybe I'm to stupid, but I'm not able to test my model.

The training runs fine, but how should I test my model after that? Always running into some exceptions while trying to predict my own classes.

what i do to predict is like this , just put the image into the VGG16 again and use the output bottleneck feature to predict....
so i need to load the model twice,i donot konw whether this is good. sorry for my terrible english! see:

if name=='main':
model=applications.VGG16(include_top=False,weights='imagenet')
img=load('/home/lx/Desktop/cat1.jpg',target_size=(150,150))
img=img_to_array(img)
img=img.reshape((1,)+x.shape)
img=img/255.
feature_img=model.predict(img) #then the shape is (1,512,4,4)
model=load_model('xxx_model.h5') #my own model which the top is FC layers
classes=model.predict_classes(feature_img)
prob=model.predict_proba(feature_img)
print(classes,prob)

BoldinovaNatalya commented Apr 4, 2017 edited

Guys please help me. I followed this example and when i try to predict on model from "train_top_model" with the following code:
inputShape = (150, 150)
image = load_img("92.jpg", target_size=inputShape)
image = img_to_array(image)
preprocess = imagenet_utils.preprocess_input
image = np.expand_dims(image, axis=0)
image = preprocess(image)
preds = model.predict(image)

I got ValueError: Error when checking : expected flatten_2_input to have shape (None, 4, 4, 512) but got array with shape (1, 150, 150, 3)
Have you any ideas how to fix it? Should i add top_model to base model and call predict on this model?

JamesKeras commented Apr 5, 2017 edited

I got the first tutorial running fine, but I'm having trouble with this second one.

The script runs, I get these messages on the command prompt: https://image.ibb.co/c729GF/cmd_script2.png The script just keeps running, taking up 100% CPU and doing nothing after these messages appear.

I tried running in python 2.7 on Ubuntu and python 3.5 on Windows. Both times the same thing happened. How do I fix this?

@SoftwareentwicklungHell
This tutorial explains how to classification with keras, it might be helpful for you http://www.pyimagesearch.com/2016/08/10/imagenet-classification-with-python-and-keras/

@BoldinovaNatalya @liangxiao05
So you got the original script working fine and your problems are with edits? What versions of the python modules are you using?

@JamesKeras. I'm having a similar problem to yours. The script keeps running and I don't get the desired output. Were you able to find a solution to this issue?

Thanks.

First of all thanks for a great tutorial!

I ran into some problems while running the code.
It turned out that one of the images in my dataset (images from ImageNet) caused a lot of problems.
Whenever I ran the code I received the following error message.

AssertionError: Can't store in size_t for the bytes requested 1535065520309053200 * 4 Apply node that caused the error: GpuAlloc{memset_0=True}(CudaNdarrayConstant{0.0}, Shape_i{0}.0, Elemwise{add,no_inplace}.0, Elemwise{add,no_inplace}.0, Shape_i{3}.0) Toposort index: 762 Inputs types: [CudaNdarrayType(float32, scalar), TensorType(int64, scalar), TensorType(int64, scalar), TensorType(int64, scalar), TensorType(int64, scalar)] Inputs shapes: [(), (), (), (), ()] Inputs strides: [(), (), (), (), ()] Inputs values: [CudaNdarray(0.0), array(278L, dtype=int64), array(506083110L, dtype=int64), array(6L, dtype=int64), array(505538020L, dtype=int64)] Outputs clients: [[GpuIncSubtensor{InplaceSet;::, int64:int64:, int64:int64:, ::}(GpuAlloc{memset_0=True}.0, GpuFromHost.0, Constant{3}, ScalarFromTensor.0, Constant{3}, ScalarFromTensor.0)]]

It turned out though, that it has nothing to do with a memory error instead, the error was caused by the

img = img.resize(wh_tuple)

in the def load_img found in the image.py file. Running this block of code separately resulted in the following error message.

Traceback (most recent call last): File "C:\Toolkits\anaconda2-4.2.0\lib\threading.py", line 801, in __bootstrap_inner self.run() File "C:\Toolkits\anaconda2-4.2.0\lib\threading.py", line 754, in run self.__target(*self.__args, **self.__kwargs) File "C:\Toolkits\anaconda2-4.2.0\lib\site-packages\keras\engine\training.py", line 606, in data_generator_task generator_output = next(self._generator) File "C:\Toolkits\anaconda2-4.2.0\lib\site-packages\keras\preprocessing\image.py", line 965, in next target_size=self.target_size) File "C:\Toolkits\anaconda2-4.2.0\lib\site-packages\keras\preprocessing\image.py", line 331, in load_img img = img.resize(wh_tuple) File "C:\Toolkits\anaconda2-4.2.0\lib\site-packages\PIL\Image.py", line 1541, in resize self.load() File "C:\Toolkits\anaconda2-4.2.0\lib\site-packages\PIL\ImageFile.py", line 244, in load raise_ioerror(err_code) File "C:\Toolkits\anaconda2-4.2.0\lib\site-packages\PIL\ImageFile.py", line 59, in raise_ioerror raise IOError(message + " when reading image file") IOError: broken data stream when reading image file

I haven't figured out why this particular image makes the function fail since it too comes in JPEG format. I can get the code to work by adding a try and except in the def load_img and in the class DirectoryIterator(Iterator), but I guess that some of you might have a better solution. Also, some of you might know how to include a fix in the original code, such that other users of Keras don't run into the same problem as me.

I have GTX1080 GPU, but predict_generator is dramatically slow.
Any suggestion to make it quick?

  bottleneck_features_train = model.predict_generator(
       generator, nb_train_samples // batch_size, verbose=1)
Using TensorFlow backend.
W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE3 instructions, but these are available on your machine and could speed up CPU computations.
W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available on your machine and could speed up CPU computations.
W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations.
Found 2000 images belonging to 2 classes.
  1/125 [..............................] - ETA: 263s
  2/125 [..............................] - ETA: 252s
  3/125 [..............................] - ETA: 248s
  4/125 [..............................] - ETA: 244s
  5/125 [>.............................] - ETA: 241s
  6/125 [>.............................] - ETA: 238s
  7/125 [>.............................] - ETA: 236s
  8/125 [>.............................] - ETA: 233s
  9/125 [=>............................] - ETA: 231s
 10/125 [=>............................] - ETA: 229s
 11/125 [=>............................] - ETA: 227s
 12/125 [=>............................] - ETA: 225s

@biswatherockstar I faced the same problem. I solved it by adding 'wb' and 'rb' in the save and load functions respectively.
np.save(open('bottleneck_features_validation.npy', 'wb')
validation_data = np.load(open('bottleneck_features_validation.npy', 'rb'))

It has something to do with Windows and binary files.
Check out
https://docs.python.org/2/tutorial/inputoutput.html#reading-and-writing-files

KamalOthman commented Apr 28, 2017 edited

Hi @fchollet and everyone
I followed the program in this post for binary-classification problem. Everything worked well. The only extra thing I added in the code is saving the top model as json file with saving weights as shown in the code

Now, I wrote the below code for predicting new image. The problem is model.predict() or model.predict_proba() ormodel.predict_on_batch() in the head_model()function do not return any percentage of prbability. They return the same values of model.predict_classes(), which are either 0 or 1.

Is there any mistake in merging the two models? I know model.predict_proba() should return values from 0 to 1

Any suggestion?
Here is my code:

import numpy as np
import scipy.misc
from keras.applications.vgg16 import VGG16
from keras.preprocessing import image
from keras.applications.vgg16 import preprocess_input
from keras import backend as K
K.set_image_dim_ordering('th')

def vgg16(img_path):
    print('Loading VGG16')
    model = VGG16(weights='imagenet', include_top=False )
    img = image.load_img(img_path, target_size=(150, 150))
    x = image.img_to_array(img)
    print(x.shape)
    x = np.expand_dims(x, axis=0)
    x = preprocess_input(x)

    features = model.predict(x)
    print(features)
    return features

def load_model(model_def_fname, model_weight_fname):
    model = model_from_json(open(model_def_fname).read())
    model.load_weights(model_weight_fname)
    return model
 
def head_model(bottom_features):
    print('loading the top model')
    model = load_model(classifier_path, weights_path)
    prediction = model.predict_classes(bottom_features,batch_size=32, verbose=1)
    probability = model.predict(bottom_features,batch_size=32, verbose=1)
    return prediction, probability
       
if __name__ == '__main__':

    bottom_features = vgg16(image_path)
    
    prediction, probability = head_model(bottom_features)
    
    print(prediction, probability)
    
    print('the program is done')

I really love this simple example. But I got a problem now. Can anyone tell me how to make prediction? Here is what I did.
I use a picture of cat. Input it to the VGG model (bottle neck one). I got a feature. Then load the feature to the trained model (the added one). But got wrong result. The prediction is always 0 (the first one).

Hi @michelyang
I am wondering if you solved this problem or not yet. I am facing the same problem. I posted my code in the previous comment. Hope I can get your help.

Thanks
Kamal

KamalOthman commented May 1, 2017 edited

Hi Everyone and @michelyang
I did it:

('prediction of the class of many different classifiers ', array([0, 1, 1, 1, 1]))
('probability of the class of many different classifiers ', array([ 3.08696326e-04, 9.55853641e-01, 9.71532643e-01,
9.68130291e-01, 9.96707559e-01]))

I just normalized the image values

x = image.img_to_array(img)
x = x/255

and I removed:
x = preprocess_input(x)

Thanks

@abderhassan

I abandoned keras. I used tensorflow alone to create a deep learning object detector. I adapted this tutorial: https://www.tensorflow.org/tutorials/image_retraining

Keras was too hard to get working.

LaetiM commented May 3, 2017

Hi everyone ! I need your help :)
Here is my code :

_img_width, img_height = 150, 150

top_model_weights_path = 'bottleneck_fc_model.h5'
train_data_dir = 'data_requins_v2/train'
validation_data_dir = 'data_requins_v2/validation'
nb_train_samples = 1240
nb_validation_samples = 532
epochs = 50
batch_size = 16

def save_bottlebeck_features():
datagen = ImageDataGenerator(rescale=1. / 255)

# build the VGG16 network
model = applications.VGG16(include_top=False, weights='imagenet')

generator = datagen.flow_from_directory(
    'data_requins_v2/train',
    target_size=(150, 150),
    batch_size=16,
    class_mode=None,
    shuffle=False)
bottleneck_features_train = model.predict_generator(
    generator, 1240 // 16)
np.save(open('bottleneck_features_train', 'w'),
        bottleneck_features_train)

generator = datagen.flow_from_directory(
    'data_requins_v2/validation',
    target_size=(150, 150),
    batch_size=16,
    class_mode=None,
    shuffle=False)
bottleneck_features_validation = model.predict_generator(
    generator, 532 // 16)
np.save(open('bottleneck_features_validation', 'w'),
        bottleneck_features_validation)

def train_top_model():
train_data = np.load(open('bottleneck_features_train'))
train_labels = np.array(
[0] * (1240 // 2) + [1] * (1240 // 2))

validation_data = np.load(open('bottleneck_features_validation'))
validation_labels = np.array(
    [0] * (532 // 2) + [1] * (532 // 2))

model = Sequential()
model.add(Flatten(input_shape=train_data.shape[1:]))
model.add(Dense(256, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(1, activation='sigmoid'))

model.compile(optimizer='rmsprop',
              loss='binary_crossentropy', metrics=['accuracy'])

model.fit(train_data, train_labels,
          epochs=50,
          batch_size=16,
          validation_data=(validation_data, validation_labels))
model.save_weights(top_model_weights_path)

save_bottlebeck_features()
train_top_model()_

I am getting different Error messages :

  • TypeError : write() argument must be str, not bytes
  • AttributeError : 'NoneType' object has no attribute 'TF_DeleteStatus'
  • ValueError : input arrays should have the same number of samples as target arrays

I tried with and without '.npy', with and without 'wb' and 'rb', with and without 'import gc; gc.collect()'

But nothing works...
Can anyone help me ?

3born commented May 9, 2017 edited

@Wheele9
Opening the file in binary mode had to be specified on my system, i.e. using 'wb' mode instead of 'w' when opening the file. See http://stackoverflow.com/questions/5512811/builtins-typeerror-must-be-str-not-bytes

The approach proposed by @BrianMowrey obviously avoids the problem altogether.

I think there's an error with the predict_generator function. It should be:

bottleneck_features_train = vgg16.predict_generator(generator, 2000 // batch_size)

Otherwise you're creating around 32,000 samples.

Is the function name 'save_bottlebeck_features' a mistake? Do you mean 'bottleneck'?

@LaetiM I get the same error.

Hi, I have problem and need your help!!
I have problem about

"ValueError: Input arrays should have the same number of samples as target arrays. Found 180 input samples and 200 target samples."

Im using:
win10
tensorflow 1.2.0rc1
keras 2.0.4
python 3.5.3

my code

nb_train_samples = 180
nb_validation_samples = 20
epochs = 20
batch_size = 16

def save_bottleneck_features():
    datagen = ImageDataGenerator(
        rotation_range=40,
        width_shift_range=0.2,
        height_shift_range=0.2,
        shear_range=0.2,
        zoom_range=0.2,
        horizontal_flip=True,
        fill_mode='nearest')

    model = VGG16(include_top=False, weights='imagenet')

    generator = datagen.flow_from_directory(
        train_data_dir,
        target_size=(img_w, img_h),
        batch_size=batch_size,
        classes=classes,
        class_mode=None)
    bottleneck_features_train = model.predict_generator(
        generator, nb_train_samples // batch_size)
    np.save(open('bottleneck_features_train.npy', 'wb'),
            bottleneck_features_train)

    generator = datagen.flow_from_directory(
        validation_data_dir,
        target_size=(img_w, img_h),
        batch_size=batch_size,
        classes=classes,
        class_mode=None)
    bottleneck_feature_validation = model.predict_generator(
        generator, nb_validation_samples // batch_size)
    np.save(open('bottleneck_features_validation.npy', 'wb'),
            bottleneck_feature_validation)

then I got this Error

Traceback (most recent call last):
  File "train.py", line 108, in <module>
    train_top_model()
  File "train.py", line 101, in train_top_model
    validation_data=(validation_data, validation_labels))
  File "C:\Users\username\Anaconda3\envs\py35\lib\site-packages\keras\models.py", line 856, in fit
    initial_epoch=initial_epoch)
  File "C:\Users\username\Anaconda3\envs\py35\lib\site-packages\keras\engine\training.py", line 1429, in fit
    batch_size=batch_size)
  File "C:\Users\usernamei\Anaconda3\envs\py35\lib\site-packages\keras\engine\training.py", line 1317, in _standardize_user_data
    _check_array_lengths(x, y, sample_weights)
  File "C:\Users\username\Anaconda3\envs\py35\lib\site-packages\keras\engine\training.py", line 235, in _check_array_lengths
    'and ' + str(list(set_y)[0]) + ' target samples.')
ValueError: Input arrays should have the same number of samples as target arrays. Found 180 input samples and 200 target samples.

I think this error cause predict_generater() , but I have no idea.
Can anyone help?

ragvri commented Jun 22, 2017 edited

This is my code. It even predicts the class for a new image

Hi All,

I got a problem while using this example. I am using the same code. Can anyone help?

keras 2.0.4
python 3.5.3
Theano as backend

Error is:

ValueError: Input dimension mis-match. (input[1].shape[1] = 3, input[2].shape[1] = 64)
Apply node that caused the error: Elemwise{Composite{(i0 * ((i1 + i2) + Abs((i1 + i2))))}}[(0, 1)](TensorConstant{(1, 1, 1, 1) of 0.5}, CorrMM{half, (1, 1), (1, 1)}.0, InplaceDimShuffle{x,0,x,x}.0)
Toposort index: 40
Inputs types: [TensorType(float32, (True, True, True, True)), TensorType(float32, 4D), TensorType(float32, (True, False, True, True))]
Inputs shapes: [(1, 1, 1, 1), (20, 3, 224, 225), (1, 64, 1, 1)]
Inputs strides: [(4, 4, 4, 4), (604800, 201600, 900, 4), (256, 4, 4, 4)]
Inputs values: [array([[[[ 0.5]]]], dtype=float32), 'not shown', 'not shown']
Outputs clients: [[CorrMM{half, (1, 1), (1, 1)}(Elemwise{Composite{(i0 * ((i1 + i2) + Abs((i1 + i2))))}}[(0, 1)].0, Subtensor{::, ::, ::int64, ::int64}.0)]]

HINT: Re-running with most Theano optimization disabled could give you a back-trace of when this node was created. This can be done with by setting the Theano flag 'optimizer=fast_compile'. If that does not work, Theano optimizations can be disabled with 'optimizer=None'.
HINT: Use the Theano flag 'exception_verbosity=high' for a debugprint and storage map footprint of this apply node.

VinVinVinVin commented Jun 26, 2017 edited

Hello,
i've got a problem adapting the code. In my programm, I'm not able to store the pictures in an directory, but I become them as a numpy array directly. Therefore I am not able to use the flow_from_directory function but only the flow function for generating the training data. This seems to screw up the process. With the original code, I get an acc-rating of about 0.9. With my modified code it is barely above 0.5.
This is my function to create the bottleneck-feature-extraction:
`
self.modelVGG16=applications.VGG16(include_top=False,weights='imagenet')
nb_train_samples=self.training_data.shape[0]
nb_validation_samples=self.validation_data.shape[0]

    train_datagen = ImageDataGenerator(
        rescale=1. / 255,
        shear_range=0.2,
        zoom_range=0.2,
        horizontal_flip=True
        )
    test_datagen = ImageDataGenerator(rescale=1. / 255)
    
    train_generator = train_datagen.flow(
            self.training_data,
            self.training_label,
            batch_size=self.batch_size,
            )
    validation_generator = test_datagen.flow(
            self.validation_data,
            self.validation_label,
            batch_size=self.batch_size_val,
            )
    self.bottleneck_features_train=self.modelVGG16.predict_generator(
            train_generator, nb_train_samples // self.batch_size)
    self.bottleneck_features_validation = self.modelVGG16.predict_generator(
            validation_generator, nb_validation_samples // self.batch_size)

`
I then continue to train my top-network with the bottleneck_features_train/validation arrays. If instead I use the stored npy-Arrays from the original code, everything works fine. So I guess the problem lies somewhere in the above code. training_data/label and validation_data/label are numpy vstacks with the whole training/validation pictures stored.

Anybody have an idea why this isn't working?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment