Create a gist now

Instantly share code, notes, and snippets.

What would you like to do?
Fine-tuning a Keras model. Updated to the Keras 2.0 API.
'''This script goes along the blog post
"Building powerful image classification models using very little data"
from blog.keras.io.
It uses data that can be downloaded at:
https://www.kaggle.com/c/dogs-vs-cats/data
In our setup, we:
- created a data/ folder
- created train/ and validation/ subfolders inside data/
- created cats/ and dogs/ subfolders inside train/ and validation/
- put the cat pictures index 0-999 in data/train/cats
- put the cat pictures index 1000-1400 in data/validation/cats
- put the dogs pictures index 12500-13499 in data/train/dogs
- put the dog pictures index 13500-13900 in data/validation/dogs
So that we have 1000 training examples for each class, and 400 validation examples for each class.
In summary, this is our directory structure:
```
data/
train/
dogs/
dog001.jpg
dog002.jpg
...
cats/
cat001.jpg
cat002.jpg
...
validation/
dogs/
dog001.jpg
dog002.jpg
...
cats/
cat001.jpg
cat002.jpg
...
```
'''
from keras import applications
from keras.preprocessing.image import ImageDataGenerator
from keras import optimizers
from keras.models import Sequential
from keras.layers import Dropout, Flatten, Dense
# path to the model weights files.
weights_path = '../keras/examples/vgg16_weights.h5'
top_model_weights_path = 'fc_model.h5'
# dimensions of our images.
img_width, img_height = 150, 150
train_data_dir = 'cats_and_dogs_small/train'
validation_data_dir = 'cats_and_dogs_small/validation'
nb_train_samples = 2000
nb_validation_samples = 800
epochs = 50
batch_size = 16
# build the VGG16 network
model = applications.VGG16(weights='imagenet', include_top=False)
print('Model loaded.')
# build a classifier model to put on top of the convolutional model
top_model = Sequential()
top_model.add(Flatten(input_shape=model.output_shape[1:]))
top_model.add(Dense(256, activation='relu'))
top_model.add(Dropout(0.5))
top_model.add(Dense(1, activation='sigmoid'))
# note that it is necessary to start with a fully-trained
# classifier, including the top classifier,
# in order to successfully do fine-tuning
top_model.load_weights(top_model_weights_path)
# add the model on top of the convolutional base
model.add(top_model)
# set the first 25 layers (up to the last conv block)
# to non-trainable (weights will not be updated)
for layer in model.layers[:25]:
layer.trainable = False
# compile the model with a SGD/momentum optimizer
# and a very slow learning rate.
model.compile(loss='binary_crossentropy',
optimizer=optimizers.SGD(lr=1e-4, momentum=0.9),
metrics=['accuracy'])
# prepare data augmentation configuration
train_datagen = ImageDataGenerator(
rescale=1. / 255,
shear_range=0.2,
zoom_range=0.2,
horizontal_flip=True)
test_datagen = ImageDataGenerator(rescale=1. / 255)
train_generator = train_datagen.flow_from_directory(
train_data_dir,
target_size=(img_height, img_width),
batch_size=batch_size,
class_mode='binary')
validation_generator = test_datagen.flow_from_directory(
validation_data_dir,
target_size=(img_height, img_width),
batch_size=batch_size,
class_mode='binary')
# fine-tune the model
model.fit_generator(
train_generator,
samples_per_epoch=nb_train_samples,
epochs=epochs,
validation_data=validation_generator,
nb_val_samples=nb_validation_samples)

Great example, thanks for sharing Francois!! Aside from the comment from yurkor regarding the proper VGG image pre-processing (which I agree with), I have a few questions about the general procedure:

  1. Would it not be better to train the top model with different samples than you will later on use to fine tune your model? In the cat/dog example that would mean put aside 10-20% as validation set and then use 50% of the remainder for creating the top model and the other 50% for fine tuning. Any views?
  2. Do you have any insights regarding the best size of the top model when your final goal is a segmentation into 10 classes rather than 2? I tried to go back to the original VGG16 architecture, experimenting with two fully connected layers of 1024 and 512, but it appears there still is substantial overfitting. Any advice?

@SchumErik

  1. I think this approach need more data
  2. again more data if you train large network. You can use regularization to defeat overfitting

By the way, if you have data size like ImageNet, you even can train VGG16 from scratch, not use fine tune at all.

Arsey commented Sep 11, 2016 edited

@SchumErik, about the second question, my experiments shows that the top model from the original VGG16 works better than model with decreased dimension. Here I classify 102 classes https://github.com/Arsey/keras-transfer-learning-for-oxford102

Hi Francois, first of all, thanks for all this great material.
Quick question... is there a place from which we can download the fc_model.h5 ?
Many thanks,
AL.

@alinagithub - You have to train this model before fine tuning. There is one more tutorial from Francois about how to generate that model.

If we want to categorize 10 or 20 classes, is it necessary just to fine tune the same code over 10 or 20 classes by adding them as sub-folders into the train and validation data?

@vishsangale - many thanks!

I implemented the above model and everything worked fine. Now I have a sample of 1000 images in my test_data named folder which I want to predict using the model.predict() method. I first generated the test features as we generated the train_data above. This is shown below:
generator = datagen.flow_from_directory(test_dir, batch_size = 32, target_size =(img_rows, img_columns), classes = None, shuffle =False)

test_data_features = model.predict_generator(generator, 1000)
np.save(open('test_data_features.npy','wb'), test_data_features)
test_data = np.load(open('test_data_features.npy', 'rb'))

Now when I make predictions like model.predict(test_data), I get a numpy array of predictions. How can I test what each prediction is represting? The predictions are like this:
array([[ 9.99770999e-01], [ 7.65574304e-03], [ 2.06944350e-07], [ 9.96615469e-01], [ 4.59789817e-07], [ 9.93980138e-05], [ 5.27667798e-05],..........)

How can I compare my predictions to my images?

you can try model.predict_classes(test_data)

In addition to my yesterday's question, I would like to know that if we want to classify an object which is not belong to any of the categories in ImageNet what I have to do?

Arsey commented Oct 3, 2016 edited

I'm trying to adopt this approach for multiclass classification. And the first step when we're training the top model from bottlenecks works fine and I have 76% accuracy for 102 classes. But when I proceed to fine-tune step, the model with each next epoch decreases in accuracy. So if at the end of 1st epoch of fine-tune I have about 45% of accuracy, on the 2nd epoch I'm getting 40%, etc. Here's repo with the code https://github.com/Arsey/keras-oxford102. Can anyone help? I'm trying to find the issue for a few weeks but no results :(

apapiu commented Oct 5, 2016

Great Post, I am learning a lot. I do have a question about the top model. If I understand correctly I need to have the weights properly tuned already before sticking it to the convolutional layers. But how do I compute the weights to begin with?

To be more specific: In this line: top_model_weights_path = 'fc_model.h5' how did you compute the weights for the top model?

@alinagithub Did you find the tutorial about how to train fc_model.h5 ? I can't find it.

Golly commented Oct 8, 2016

@Arsey Same problem with fine tuning. Im trying now and model decreased in accuracy from 87% to 50%.

Euphemiasama commented Oct 9, 2016 edited

@apapiu @CharlesNord You can compute the weights to get the 'fc_model.h5' file by running part 2 of this tutorial entitled 'Using the bottleneck features of a pre-trained network: 90% accuracy in a minute' it creates a file named 'bottleneck_fc_model.h5' you rename it to 'fc_model.h5' and run the code.

@fchollet
In line 156 and 162, target_size=(img_height, img_width)
which should go first 'img_height' or 'img_width'?
In classifier_from_little_data_script_2.py, it is in reverse order. I am confused here.

@Euphemiasama Thank you very much. Another question, has any one reached 95% accuracy by using the fine tuning method? Before fine tuning, my accuracy is about 90%, and after fine tuning, it reaches 92% and I cannot make the result better anymore

How do I predict_classes on a new a Image(which is not in data base),
well I am trying the code below to load model and load a new Image pass through the predict_classes but I am getting an error:

import keras
from keras.models import load_model
from keras.models import Sequential
import cv2
import numpy as np 
from keras.preprocessing.image import ImageDataGenerator, array_to_img, img_to_array, load_img

model = Sequential()

model =load_model('firstmodel.h5')
model.compile(loss='binary_crossentropy',
              optimizer='rmsprop',
              metrics=['accuracy'])

img = cv2.imread('cat.jpg',0).astype(np.float32) / 255
img = cv2.resize(img, (150, 150))
#img = prep_data(img)
#img = img.flatten()
img = np.expand_dims(img, axis=0)
# img1 = np.expand_dims(img, axis=0)
# img = cv2.imread('cat.jpg')
# img = cv2.resize(img,(150,150))
# x = img_to_array(img)  # this is a Numpy array with shape (3, 150, 150)
classes = model.predict_classes(img)
print classes

I am getting error:

using Theano backend.
Traceback (most recent call last):
  File "detect.py", line 28, in <module>
    classes = model.predict_classes(img)
  File "/usr/local/lib/python2.7/dist-packages/keras/models.py", line 780, in predict_classes
    proba = self.predict(x, batch_size=batch_size, verbose=verbose)
  File "/usr/local/lib/python2.7/dist-packages/keras/models.py", line 672, in predict
    return self.model.predict(x, batch_size=batch_size, verbose=verbose)
  File "/usr/local/lib/python2.7/dist-packages/keras/engine/training.py", line 1174, in predict
    check_batch_dim=False)
  File "/usr/local/lib/python2.7/dist-packages/keras/engine/training.py", line 100, in standardize_input_data
    str(array.shape))
Exception: Error when checking : expected convolution2d_input_1 to have 4 dimensions, but got array with shape (1, 150, 150)

nathmo commented Dec 14, 2016 edited

hi, i made a derrived script from your code and get the following error :
i'm running tensor flow 0.12.0-rc1 With python 2.7
i have installed the latest version of keras from git.
What i have done wrong ?

Source code : http://pastebin.com/JEqcZe5x
Full Error : http://pastebin.com/yEFVHWEK

"ValueError: Error when checking model input: expected convolution2d_input_1 to have shape (None, 3, 32, 32) but got array with shape (32, 32, 32, 3)"

Thank in advance

Nathann

Abusnina commented Jan 5, 2017 edited

How can we use the modified pre-trained model to predict test data? I compiled the model successfully, but when I try to predict my test data i get the following error:
Error when checking : expected flatten_input_1 to have shape (None, 4, 4, 512) but got array with shape (1, 3, 150, 150)
Any advice on this?

@Golly
@Arsey

Same problem, before fine-tuning my model for 5 classes reached 98% accuracy but the first epoch of fine-tuning dropped to 20%. Did you or does anyone work it out for multi-class problem? I guess we need more train data to feed our model

Is the slice of :25 correct for setting the non-trainable parameter? If you do model.layers[:26] you see that the last layer is also a Conv layer.

Hi everyone,
I am following this example on GRAYSCALE images, which I converted them from color scale using PIL convert method. I expected the input_sahpe in the model should be (1, w, h). but I get this error:
Exception: Error when checking model input: expected convolution2d_input_1 to have shape (None, 1, 150, 150) but got array with shape (32, 3, 150, 150)
When I checked the image shape, I got only (w,h)
However, It still works with input_shape=(3, w, h). Isn't it strange?

My goal to compare the learning on gray images to color images. But I do not know how to expand the dimension of gray images within flow_from_directory.

Anyone can help please?

Thanks
Kamal

cobir commented Jan 30, 2017

Hi,

The input shape of the model is (3, img_width, img_height). Does this mean that we can only work with theano as backend when working with this pre-trained VGG16 model?

Thanks,
OC

@cobir

I used Tensorflow backend it works fine.

I get the following error when fine tuning a 8 class multi class classification. Any idea anyone please help

nb_val_samples=nb_validation_samples)

File "C:\Users\bgsingh\Anaconda2\lib\site-packages\keras\models.py", line 935, in fit_generator
initial_epoch=initial_epoch)
File "C:\Users\bgsingh\Anaconda2\lib\site-packages\keras\engine\training.py", line 1553, in fit_generator
class_weight=class_weight)
File "C:\Users\bgsingh\Anaconda2\lib\site-packages\keras\engine\training.py", line 1310, in train_on_batch
check_batch_axis=True)
File "C:\Users\bgsingh\Anaconda2\lib\site-packages\keras\engine\training.py", line 1034, in _standardize_user_data
exception_prefix='model target')
File "C:\Users\bgsingh\Anaconda2\lib\site-packages\keras\engine\training.py", line 124, in standardize_input_data
str(array.shape))
ValueError: Error when checking model target: expected sequential_2 to have shape (None, 1) but got array with shape (32L, 8L)

Thanks,
Biswa

a-ozbek commented Feb 5, 2017 edited

Excuse me if this issue was brought up before about this script. I couldn't find a resolution to this in the comments.

In this script, during the fine-tuning, the "train_generator" and "validation_generator" do not seem to do VGG16 pre-processing which is

      # 'RGB'->'BGR'  
        x = x[:, :, :, ::-1]  
        # Zero-center by mean pixel  
        x[:, :, :, 0] -= 103.939  
        x[:, :, :, 1] -= 116.779  
        x[:, :, :, 2] -= 123.68  

Isn't it wrong to do fine-tuning of VGG16 without this pre-processing step?

aidiary commented Feb 16, 2017

@a-ozbek

I have the same question.
I have experimented with and without this pre-processing and get the slightly better result in the case of without this pre-processing...

without pre-processing => val_acc = 93.5%@ epoch 50
with pre-processing      => val_acc = 92.8% @ epoch 50

I think this pre-processing is essential to use keras.applications.vgg16..
Does anyone know why?

# with pre-precessing version
# Use keras 1.2.2 for preprocessing_function

# x = 3D tensor version
def preprocess_input(x):
    # 'RGB'->'BGR'
    x = x[:, :, ::-1]
    # Zero-center by mean pixel
    x[:, :, 0] -= 103.939
    x[:, :, 1] -= 116.779
    x[:, :, 2] -= 123.68
    return x

train_datagen = ImageDataGenerator(
    preprocessing_function=preprocess_input,
    shear_range=0.2,
    zoom_range=0.2,
    horizontal_flip=True)

test_datagen = ImageDataGenerator(
    preprocessing_function=preprocess_input)

@CharlesNord: same here, at best 91.75%. Also tried the VGG16 net included in keras\applications, but that gave similar results.

embanner commented Mar 5, 2017

See this notebook for an example of fine-tuning a keras.applications.vgg16.VGG16 that hooks together keras.preprocessing.image.ImageDataGenerator withkeras.applications.vgg16.preprocess_input() for image preprocessing.

Note, I'm using the theano backend.

Original gist at https://gist.github.com/embanner/6149bba89c174af3bfd69537b72bca74.

cswwp commented Mar 7, 2017

@biswatherockstar I also have the same problem, do you solove it ?

Irtza commented Mar 15, 2017

The applications.VGG16 model is defined using the Functional API .. when I'm trying to model.'add' the instance returned by VGG16 base class on imagenet weights. I get Attribute error: that metaclass keras.engine.training.Model has no arrtibute add? has there been a change ? or am I missing something?

sampathweb commented Mar 15, 2017 edited

@Irtza Yes, keras.applications.vgg16 uses Functional API. You can only use the "add" method to a Sequential API. Functional API is actually more flexible and you can build out a graph. For an example of how to add your own layers on top, checkout this notebook posted by @embanner couple of posts above here.

I am trying to finetune the VGG16 for catsVSdogs datasets by exactly replicating the gist above. I am using tensorflow as my backend.
But I am getting the error at line:
top_model.add(Flatten(input_shape=model.output_shape[1:]))

The error is:
ValueError: The shape of the input to "Flatten" is not fully defined (got (None, None, 512). Make sure to pass a complete "input_shape" or "batch_input_shape" argument to the first layer in your model.

I am already up to date with the Keras 2 API. Please help me out here.

While trying above code, I am getting this error when adding top layers.
AttributeError: 'Model' object has no attribute 'add'

telesphore commented Mar 18, 2017 edited

@fwahhab89

I ran into a similar issue. I think you may need to specify the input shape for your VGG16 model. I'm not doing "cats and dogs" here but for my case I did.

model = applications.VGG16(weights='imagenet', include_top=False, input_shape=(224, 224, 3))

Remember tensorflow vs theano order

kevinpatricksmith commented Mar 19, 2017 edited

I am working through this set of three tutorials with tensorflow 1.0 on GPU and keras 2

When I try to create the top_model and as a first add step call Flatten, I get the following error:

ValueError: The shape of the input to "Flatten" is not fully defined (got (None, None, 512). Make sure to pass a complete "input_shape" or "batch_input_shape" argument to the first layer in your model

When fix as per @fwahhab89 with input-shape=(150,150,3), I then get the same error as rikenmehta03:
model.add(top_model)
AttributeError: 'Model' object has no attribute 'add'

Sequential model has an add function, but VGG16 is based on Model.

Got error like this:

 base_model.add(top_model)
AttributeError: 'Model' object has no attribute 'add'

kevinpatricksmith commented Mar 19, 2017 edited

This "AttributeError: 'Model' object has no attribute 'add'" issue seems to be related to this issue: fchollet/keras#3465

Their approach would work if we weren't trying to reload the top model weights from part 2 of the tutorial

Should it perhaps be something like:
model = Model(input=model.input, output=top_model)

Omegamon commented Mar 20, 2017 edited

@kevinpatricksmith
Thanks!
Fixed by this :

  input_tensor = Input(shape=(150,150,3))
  base_model = VGG16(weights='imagenet',include_top= False,input_tensor=input_tensor)
  top_model = Sequential()
  top_model.add(Flatten(input_shape=base_model.output_shape[1:]))
  top_model.add(Dense(256, activation='relu'))
  top_model.add(Dropout(0.5))
  top_model.add(Dense(1, activation='sigmoid'))
  top_model.load_weights('bootlneck_fc_model.h5')
  model = Model(input= base_model.input, output= top_model(base_model.output))

I guess this part needs to be updated for Keras 2 API.

# fine-tune the model
model.fit_generator(
    train_generator,
    steps_per_epoch=nb_train_samples // batch_size,
    epochs=epochs,
    validation_data=validation_generator,
    validation_steps=nb_validation_samples // batch_size)

kevinpatricksmith commented Mar 21, 2017 edited

@Omegamon and @hiroyachiba
Thanks.
I used both your suggestions and can now get training to complete.

I am now getting accuracy of 92% and validation set accuracy of almost 90%. However it does not really improve much from epoch to epoch.

When I use model.summary() to print out the model, I am seeing only one additional layer after the last set of VGG16 convolution/pooling. It is called Sequential_1. I don't know if the top_model was added correctly:

model.summary()

Layer (type)                 Output Shape              Param #   
=================================================================
input_1 (InputLayer)         (None, 150, 150, 3)       0         
_________________________________________________________________
block1_conv1 (Conv2D)        (None, 150, 150, 64)      1792      
_________________________________________________________________
block1_conv2 (Conv2D)        (None, 150, 150, 64)      36928     
_________________________________________________________________
block1_pool (MaxPooling2D)   (None, 75, 75, 64)        0         
_________________________________________________________________
block2_conv1 (Conv2D)        (None, 75, 75, 128)       73856     
_________________________________________________________________
block2_conv2 (Conv2D)        (None, 75, 75, 128)       147584    
_________________________________________________________________
block2_pool (MaxPooling2D)   (None, 37, 37, 128)       0         
_________________________________________________________________
block3_conv1 (Conv2D)        (None, 37, 37, 256)       295168    
_________________________________________________________________
block3_conv2 (Conv2D)        (None, 37, 37, 256)       590080    
_________________________________________________________________
block3_conv3 (Conv2D)        (None, 37, 37, 256)       590080    
_________________________________________________________________
block3_pool (MaxPooling2D)   (None, 18, 18, 256)       0         
_________________________________________________________________
block4_conv1 (Conv2D)        (None, 18, 18, 512)       1180160   
_________________________________________________________________
block4_conv2 (Conv2D)        (None, 18, 18, 512)       2359808   
_________________________________________________________________
block4_conv3 (Conv2D)        (None, 18, 18, 512)       2359808   
_________________________________________________________________
block4_pool (MaxPooling2D)   (None, 9, 9, 512)         0         
_________________________________________________________________
block5_conv1 (Conv2D)        (None, 9, 9, 512)         2359808   
_________________________________________________________________
block5_conv2 (Conv2D)        (None, 9, 9, 512)         2359808   
_________________________________________________________________
block5_conv3 (Conv2D)        (None, 9, 9, 512)         2359808   
_________________________________________________________________
block5_pool (MaxPooling2D)   (None, 4, 4, 512)         0         
_________________________________________________________________
sequential_1 (Sequential)    (None, 1)                 2097665   
=================================================================

This code and https://blog.keras.io/building-powerful-image-classification-models-using-very-little-data.html says to freeze the first 25 layers, but applications.VGG16 doesn't have that many! Pretty sure you need to freeze the first 15, not 25: here's my model.layers, including the top dense sequential model):

[<keras.engine.topology.InputLayer at 0x11aa49290>,
 <keras.layers.convolutional.Conv2D at 0x11aa49610>,
 <keras.layers.convolutional.Conv2D at 0x11a710490>,
 <keras.layers.pooling.MaxPooling2D at 0x11aa6c5d0>,
 <keras.layers.convolutional.Conv2D at 0x11aa49310>,
 <keras.layers.convolutional.Conv2D at 0x11aa78310>,
 <keras.layers.pooling.MaxPooling2D at 0x119c18fd0>,
 <keras.layers.convolutional.Conv2D at 0x119ca7fd0>,
 <keras.layers.convolutional.Conv2D at 0x11aa13510>,
 <keras.layers.convolutional.Conv2D at 0x11ac1da90>,
 <keras.layers.pooling.MaxPooling2D at 0x11b4b6f90>,
 <keras.layers.convolutional.Conv2D at 0x11b4c2c90>,
 <keras.layers.convolutional.Conv2D at 0x11b535990>,
 <keras.layers.convolutional.Conv2D at 0x11b518cd0>,
 <keras.layers.pooling.MaxPooling2D at 0x11b57dd50>, # want to freeze up to and including this layer
 <keras.layers.convolutional.Conv2D at 0x11b5a9950>,
 <keras.layers.convolutional.Conv2D at 0x11b84f990>,
 <keras.layers.convolutional.Conv2D at 0x11b58af90>,
 <keras.layers.pooling.MaxPooling2D at 0x11bb59bd0>,
 <keras.models.Sequential at 0x1179c58d0>]

@kevinpatricksmith
My model structure is the same as yours. I froze the first 15 layers by:

for layer in model.layers[:15]:
    layer.trainable = False

After 100 epoch, I got accuracy of 99.8% and validation set accuracy of 93.75%. After 50 epoch, the validation set accuracy
stabilized in the range of 93% ~ 94%

codemukul95 commented Mar 25, 2017 edited

Hi! If we look at the VGG16 source code (https://github.com/fchollet/keras/blob/master/keras/applications/vgg16.py) number of layers till last convolutional block are 18. they why learning rate of first 25 layers is set to false in the above code.

for layer in model.layers[:25]:
 layer.trainable = False 

Please clarify it ASAP.
Thanks in anticipation.

liangxiao05 commented Apr 1, 2017 edited

for layer in model.layers[:25]
[:25] shoule be [:15]

block5_conv1 (Convolution2D) (None, 512, 9, 9) 2359808 block4_pool[0][0]


block5_conv2 (Convolution2D) (None, 512, 9, 9) 2359808 block5_conv1[0][0]


block5_conv3 (Convolution2D) (None, 512, 9, 9) 2359808 block5_conv2[0][0]


block5_pool (MaxPooling2D) (None, 512, 4, 4) 0 block5_conv3[0][0]


sequential_1 (Sequential) (None, 1) 2097665 block5_pool[0][0]


Total params: 16,812,353
Trainable params: 9,177,089 ----------------#9177089=2359808*3+2097665
Non-trainable params: 7,635,264

@Abusnina How did you solve this problem?

What if we want to use multi-gpu?

CP121 commented Apr 8, 2017

What is the Keras API 2.0 version of this:

model = Model(input=base_model.input, output=top_model(base_model.output))

This line produces the user warning:

UserWarning: Update yourModelcall to the Keras 2 API:Model(outputs=Tensor("se..., inputs=Tensor("in...)

lusob commented Apr 16, 2017

@jwgs1 Use plurals in the args:

model = Model(inputs=base_model.input, outputs=top_model(base_model.output))

It was quite a trouble running this. Finally did it with your inputs @Omegamon @hiroyachiba @liangxiao05 :)

saulthu commented Apr 19, 2017 edited

Working code (below) was updated as per comments to run with Keras 2.

from keras import applications
from keras.preprocessing.image import ImageDataGenerator
from keras import optimizers
from keras.models import Sequential
from keras.models import Model
from keras.layers import Dropout, Flatten, Dense

# path to the model weights files.
weights_path = '../keras/examples/vgg16_weights.h5'
top_model_weights_path = 'bottleneck_fc_model.h5'
# dimensions of our images.
img_width, img_height = 150, 150

train_data_dir = 'data/train'
validation_data_dir = 'data/validation'
nb_train_samples = 2000
nb_validation_samples = 800
epochs = 50
batch_size = 16

# build the VGG16 network
base_model = applications.VGG16(weights='imagenet', include_top=False, input_shape=(150,150,3))
print('Model loaded.')

# build a classifier model to put on top of the convolutional model
top_model = Sequential()
top_model.add(Flatten(input_shape=base_model.output_shape[1:]))
top_model.add(Dense(256, activation='relu'))
top_model.add(Dropout(0.5))
top_model.add(Dense(1, activation='sigmoid'))

# note that it is necessary to start with a fully-trained
# classifier, including the top classifier,
# in order to successfully do fine-tuning
top_model.load_weights(top_model_weights_path)

# add the model on top of the convolutional base
# model.add(top_model)
model = Model(inputs=base_model.input, outputs=top_model(base_model.output))

# set the first 25 layers (up to the last conv block)
# to non-trainable (weights will not be updated)
for layer in model.layers[:15]:
    layer.trainable = False

# compile the model with a SGD/momentum optimizer
# and a very slow learning rate.
model.compile(loss='binary_crossentropy',
              optimizer=optimizers.SGD(lr=1e-4, momentum=0.9),
              metrics=['accuracy'])

# prepare data augmentation configuration
train_datagen = ImageDataGenerator(
    rescale=1. / 255,
    shear_range=0.2,
    zoom_range=0.2,
    horizontal_flip=True)

test_datagen = ImageDataGenerator(rescale=1. / 255)

train_generator = train_datagen.flow_from_directory(
    train_data_dir,
    target_size=(img_height, img_width),
    batch_size=batch_size,
    class_mode='binary')

validation_generator = test_datagen.flow_from_directory(
    validation_data_dir,
    target_size=(img_height, img_width),
    batch_size=batch_size,
    class_mode='binary')

model.summary()

# fine-tune the model
model.fit_generator(
    train_generator,
    steps_per_epoch=nb_train_samples // batch_size,
    epochs=epochs,
    validation_data=validation_generator,
    validation_steps=nb_validation_samples // batch_size,
    verbose=2)

Can any one tell me what is the "predict_class" equivalent in keras 2.0.3?
model.predict(x) gives me array of probas

Thank you.

@bis-carbon Believe it's predict_classes now. If you use anaconda, you can take a look at models.py under ~/anaconda2/lib/python2.7/site-packages/keras/

raaju-shiv commented Apr 27, 2017 edited

Hi, I have a dimension mismatch error on running the code:
`import numpy as np
from keras.preprocessing.image import ImageDataGenerator
from keras.models import Sequential
from keras.layers import Dropout, Flatten, Dense
from keras import applications
from keras import backend as K
K.set_image_dim_ordering('tf')
from keras import optimizers
from keras.models import Model

path to the model weights files.

weights_path = '../keras/examples/vgg16_weights.h5'
top_model_weights_path = 'bottleneck_fc_model.h5'
img_width, img_height = 150, 150
train_data_dir = 'data4/train'
validation_data_dir = 'data4/validation'
nb_train_samples = 20
nb_validation_samples = 20
epochs = 5
batch_size = 1
base_model = applications.VGG16(weights='imagenet', include_top=False, input_shape=(150,150,3))
print('Model loaded.')
top_model = Sequential()
top_model.add(Flatten(input_shape=base_model.output_shape[1:]))
top_model.add(Dense(128, activation='relu'))
top_model.add(Dropout(0.5))
top_model.add(Dense(1, activation='sigmoid'))
top_model.load_weights('bottleneck_fc_model.h5')
model = Model(input= base_model.input, output= top_model(base_model.output))
for layer in model.layers[:15]:
layer.trainable = False
model.compile(loss='binary_crossentropy',
optimizer=optimizers.SGD(lr=1e-4, momentum=0.9),
metrics=['accuracy'])

prepare data augmentation configuration

train_datagen = ImageDataGenerator(
rescale=1. / 255,
shear_range=0.2,
zoom_range=0.2,
horizontal_flip=True)
test_datagen = ImageDataGenerator(rescale=1. / 255)
train_generator = train_datagen.flow_from_directory(
train_data_dir,
target_size=(img_height, img_width),
batch_size=batch_size,
class_mode='binary')
validation_generator = test_datagen.flow_from_directory(
validation_data_dir,
target_size=(img_height, img_width),
batch_size=batch_size,
class_mode='binary')
model.summary()

fine-tune the model

model.fit_generator(
train_generator,
steps_per_epoch=nb_train_samples // batch_size,
epochs=epochs,
validation_data=validation_generator,
validation_steps=nb_validation_samples // batch_size,
verbose=2)`

The error is: Dimension 0 in both shapes must be equal, but are 8192 and 115200 for 'Assign_734' (op: 'Assign') with input shapes: [8192,128], [115200,128].
The error occurs when I try to execute the comment:
top_model.load_weights('bottleneck_fc_model.h5').

Kindly help.

skoch9 commented Apr 28, 2017

I run a slightly modified version of this example which only fine tunes the top layers (with Keras 2.0.3/Tensorflow on Ubuntu). This looks like the following:

img_width, img_height = 150, 150
train_data_dir = 'data/train_s'
validation_data_dir = 'data/val_s'
nb_train_samples = 2000
nb_validation_samples = 800
epochs = 10
batch_size = 16

base_model = applications.VGG16(weights='imagenet', include_top=False, input_shape=(img_width, img_height, 3))

top_model = Sequential()
top_model.add(Flatten(input_shape=base_model.output_shape[1:]))
top_model.add(Dense(256, activation='relu'))
top_model.add(Dense(1, activation='sigmoid'))

model = Model(inputs=base_model.input, outputs=top_model(base_model.output))
model.compile(loss='binary_crossentropy', optimizer=optimizers.SGD(lr=1e-4, momentum=0.9),
              metrics=['accuracy'])

train_datagen = ImageDataGenerator(
    rescale=1. / 255,
    shear_range=0.2,
    zoom_range=0.2,
    horizontal_flip=True)

test_datagen = ImageDataGenerator(rescale=1. / 255)

train_generator = train_datagen.flow_from_directory(
    train_data_dir,
    target_size=(img_height, img_width),
    batch_size=batch_size,
    class_mode='binary')

validation_generator = test_datagen.flow_from_directory(
    validation_data_dir,
    target_size=(img_height, img_width),
    batch_size=batch_size,
    class_mode='binary', shuffle=False)

model.fit_generator(
    train_generator,
    steps_per_epoch=nb_train_samples // batch_size,
    epochs=epochs,
    validation_data=validation_generator,
    validation_steps=nb_validation_samples // batch_size,
    verbose=2, workers=12)

score = model.evaluate_generator(validation_generator, nb_validation_samples/batch_size, workers=12)

scores = model.predict_generator(validation_generator, nb_validation_samples/batch_size, workers=12)

correct = 0
for i, n in enumerate(validation_generator.filenames):
    if n.startswith("cats") and scores[i][0] <= 0.5:
        correct += 1
    if n.startswith("dogs") and scores[i][0] > 0.5:
        correct += 1

print("Correct:", correct, " Total: ", len(validation_generator.filenames))
print("Loss: ", score[0], "Accuracy: ", score[1])

With this, I get arbitrary validation accuracy results. For example, predict_generator predicts 640 out of 800 (80%) classes correctly whereas evaluate_generator produces an accuracy score of 95%. Someone in #3477 suggests to remove the rescale=1. / 255 parameter from the validation generator, then I get results of 365/800=45% and 89% from evaluate_generator.

Is there something wrong with my evaluation or is this due to a bug? There are many similar issues (e.g. #3849, #6245) where the stated accuracy (during training and afterwards) doesn't match the actual predictions. Could someone experienced maybe shine some light onto this problem?

@Golly
@Arsey
@austinchencym

Same problem, before fine-tuning my model for 10 classes reached 68% accuracy but the first epoch of fine-tuning dropped to 30%. Did you or does anyone work out for multi-class problem? Thanks!

sakvaua commented May 3, 2017

Why do you guys not de-mean images but rescale=1. / 255, while pretrained VGG models require only de-meaning?

chairath commented May 5, 2017

after I finetuned already,but How can I extract feature from the added layer (sequentail_1 > dense)

what is the equivalent of predict_classes in keras 2.0

Hello there :)

1- I would like to know (please) what changes to do in order to make this work on python 3.5.

2- Also, i would like to know why in the code of tutorial 2 (for the bottleneck pre-trained features), we use put input shape as 'train_data.shape' inside the "flatten" layer, whereas in this code we choose the input shape to be 'model.output_shape' (see the line below for the particular 2 pieces of codes that i am talking about):

"model.add(Flatten(input_shape=train_data.shape[1:]))" versus " top_model.add(Flatten(input_shape=model.output_shape[1:]))"

Thank you for your time :)

@saulthu thank you for sharing your working code :) u da man!!
and yeah i don't understand why in the initial posted code all 24 layers were set to non trainable when we wanted to keep the last Conv Block free.

@saulthu
@akaashagarwal
Hi Guys,
What do you feel is the best way to fine-tune the Keras Xception network?

# set the first 100 layers 
for layer in model.layers[:100]:
layer.trainable = False

Can I do like this or do I have to begin the fine-tuning specifically from where the exit_flow starts and not in the entry and middle_flow?
Looking forward...

@raaju-shiv , i ran into the same error, "Dimension 0 in both shapes must be equal, but are 25088 and 8192 for 'Assign_26'"
do you have solved it?

jroberayalas commented May 20, 2017 edited

Quick questions:

  • Once you trained your model and you want to try it with new images, is there a fast way to convert it to the corresponding input dimensions, i.e. (150, 150, 3)? Can we use the image generator somehow to achieve this?
  • model.predict(x) computes the class for a given input x. Is there a way to get the probability of belonging to such class?

kai06046 commented Jun 4, 2017

Hi there, can anyone tell me why start with a fully-trained classifier is necessary?

top_model.load_weights(top_model_weights_path)

oxydron commented Jun 6, 2017

Line 75 doesn't work for application ResNet50. Any tips?

How do you add auc computation to this code?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment