'''This script goes along the blog post | |
"Building powerful image classification models using very little data" | |
from blog.keras.io. | |
It uses data that can be downloaded at: | |
https://www.kaggle.com/c/dogs-vs-cats/data | |
In our setup, we: | |
- created a data/ folder | |
- created train/ and validation/ subfolders inside data/ | |
- created cats/ and dogs/ subfolders inside train/ and validation/ | |
- put the cat pictures index 0-999 in data/train/cats | |
- put the cat pictures index 1000-1400 in data/validation/cats | |
- put the dogs pictures index 12500-13499 in data/train/dogs | |
- put the dog pictures index 13500-13900 in data/validation/dogs | |
So that we have 1000 training examples for each class, and 400 validation examples for each class. | |
In summary, this is our directory structure: | |
``` | |
data/ | |
train/ | |
dogs/ | |
dog001.jpg | |
dog002.jpg | |
... | |
cats/ | |
cat001.jpg | |
cat002.jpg | |
... | |
validation/ | |
dogs/ | |
dog001.jpg | |
dog002.jpg | |
... | |
cats/ | |
cat001.jpg | |
cat002.jpg | |
... | |
``` | |
''' | |
from keras.preprocessing.image import ImageDataGenerator | |
from keras.models import Sequential | |
from keras.layers import Conv2D, MaxPooling2D | |
from keras.layers import Activation, Dropout, Flatten, Dense | |
from keras import backend as K | |
# dimensions of our images. | |
img_width, img_height = 150, 150 | |
train_data_dir = 'data/train' | |
validation_data_dir = 'data/validation' | |
nb_train_samples = 2000 | |
nb_validation_samples = 800 | |
epochs = 50 | |
batch_size = 16 | |
if K.image_data_format() == 'channels_first': | |
input_shape = (3, img_width, img_height) | |
else: | |
input_shape = (img_width, img_height, 3) | |
model = Sequential() | |
model.add(Conv2D(32, (3, 3), input_shape=input_shape)) | |
model.add(Activation('relu')) | |
model.add(MaxPooling2D(pool_size=(2, 2))) | |
model.add(Conv2D(32, (3, 3))) | |
model.add(Activation('relu')) | |
model.add(MaxPooling2D(pool_size=(2, 2))) | |
model.add(Conv2D(64, (3, 3))) | |
model.add(Activation('relu')) | |
model.add(MaxPooling2D(pool_size=(2, 2))) | |
model.add(Flatten()) | |
model.add(Dense(64)) | |
model.add(Activation('relu')) | |
model.add(Dropout(0.5)) | |
model.add(Dense(1)) | |
model.add(Activation('sigmoid')) | |
model.compile(loss='binary_crossentropy', | |
optimizer='rmsprop', | |
metrics=['accuracy']) | |
# this is the augmentation configuration we will use for training | |
train_datagen = ImageDataGenerator( | |
rescale=1. / 255, | |
shear_range=0.2, | |
zoom_range=0.2, | |
horizontal_flip=True) | |
# this is the augmentation configuration we will use for testing: | |
# only rescaling | |
test_datagen = ImageDataGenerator(rescale=1. / 255) | |
train_generator = train_datagen.flow_from_directory( | |
train_data_dir, | |
target_size=(img_width, img_height), | |
batch_size=batch_size, | |
class_mode='binary') | |
validation_generator = test_datagen.flow_from_directory( | |
validation_data_dir, | |
target_size=(img_width, img_height), | |
batch_size=batch_size, | |
class_mode='binary') | |
model.fit_generator( | |
train_generator, | |
steps_per_epoch=nb_train_samples // batch_size, | |
epochs=epochs, | |
validation_data=validation_generator, | |
validation_steps=nb_validation_samples // batch_size) | |
model.save_weights('first_try.h5') |
This comment has been minimized.
This comment has been minimized.
@utonesm: indeed. The code in the blog is correct. I agree, excellent tutorial! |
This comment has been minimized.
This comment has been minimized.
Good learning example. How does one add : |
This comment has been minimized.
This comment has been minimized.
Answering my own, Thx to Jason Brownlee Checkpointing is as follows filepath="weights-improvement-{epoch:02d}-{accuracy:.2f}.hdf5" model.fit_generator( |
This comment has been minimized.
This comment has been minimized.
Thanks for this Gist, using it, I got a strange behaviour, every epoch has a loss set to NAN and an accuracy of 0.0. I suppose this is wrong but I don't find where the bug is. I have to run it once again, I did not notice this was a save_weights and not a load_weights as a consequence it crashed at the end... |
This comment has been minimized.
This comment has been minimized.
In case somebody encounters the same issue than me (see previous comment), I found the reason: I did not copy the correct number of files in the directories, for unknown reason it bugs on accuracy and loss |
This comment has been minimized.
This comment has been minimized.
Thank you for your codes, I think in the last line, it should be model.save_weights('first_try.h5') rather than model.load_weights('first_try.h5') |
This comment has been minimized.
This comment has been minimized.
I just used the model and my model magically increased its accuracy from 96 to 99 . I have no idea what happened ! But this is a very good tutorial . |
This comment has been minimized.
This comment has been minimized.
@fchollet The main changes I made in the tutorial's code to fit my problem are as follow:
I really appreciate your help. |
This comment has been minimized.
This comment has been minimized.
Hi, Thanks |
This comment has been minimized.
This comment has been minimized.
@KamalOthman,Thank You very much! |
This comment has been minimized.
This comment has been minimized.
Hello Everyone, When i used my own dataset. I am getting below error. OverflowError: Range exceeds valid bounds Can some one please advise. Thanks, |
This comment has been minimized.
This comment has been minimized.
@prapo550: can you specify where you encountered the error? The traceback? Also can you paste your |
This comment has been minimized.
This comment has been minimized.
@uom3: Thanks for the response. I am getting below error Error: Here is my ~/.keras/keras.json |
This comment has been minimized.
This comment has been minimized.
Now i am getting this error. I am using .tif images as input. |
This comment has been minimized.
This comment has been minimized.
I carefully followed the given procedure given both at blog and here. Unfortunately, I am getting the following error which I couldn't find a way to solve: |
This comment has been minimized.
This comment has been minimized.
@fchollet first of all thank you very much for the tutorial! My doubt is:
Please help |
This comment has been minimized.
This comment has been minimized.
I am getting an error:
How can I load the model for the prediction of new image. |
This comment has been minimized.
This comment has been minimized.
@aquibjaved |
This comment has been minimized.
This comment has been minimized.
Thanks for the great tutorial. I am getting the following error while running the model:
There are 2000 images in training folder and 800 in validation folder.
|
This comment has been minimized.
This comment has been minimized.
Hello,
EDIT : Solved by replacing "tf" by "th" in keras.json ! |
This comment has been minimized.
This comment has been minimized.
I get this error: The model needs to be compiled before being used. " " |
This comment has been minimized.
This comment has been minimized.
how can i use this to show a confusion matrix of the validation generator thanks for the help |
This comment has been minimized.
This comment has been minimized.
Hey, Is there a way to make the data generators process and provide the images faster? I suspect that every epoch the program re-loads the images and has to resize and process them because it has already "forgotten" that it has processed them before (because for a large image set you wouldn't have enough RAM memory to contain the resized images indefinitely). This is slowing my network training down to a very, very slow pace. I have an NVIDIA graphics card I use to train networks and usually it is very fast, however when executing this code the training is at least 10x slower than using an already formatted dataset like MNIST data. Any help is appreciated! Thanks! |
This comment has been minimized.
This comment has been minimized.
Hi @fchollet, First of all thanks to you, for all great contributions to the AI. I have been learning a lot from here. Yesterday I was working with |
This comment has been minimized.
This comment has been minimized.
I've save the model firstmodel.h5
I am missing some pre-processing any one please help me out of this? |
This comment has been minimized.
This comment has been minimized.
@fchollet, I followed your example of this code, but my training and validation loss values are stuck. I arranged the files into train and validation folders, each contains subfolders for cat and dog images. The code I have is shown below. I've tried the newest version of keras (1.1.2) and the same thing happened. So I tried another version because some other kagglers said it worked for them (didn't work for me). Any suggestions would be appreciated. `In [1]: model.add(Convolution2D(32, 3, 3)) model.add(Convolution2D(64, 3, 3)) the model so far outputs 3D feature maps (height, width, features)model.add(Flatten()) # this converts our 3D feature maps to 1D feature vectors model.compile(loss='binary_crossentropy', In [19]: this is the augmentation configuration we will use for trainingtrain_datagen = ImageDataGenerator( this is a generator that will read pictures found insubfolers of 'data/train', and indefinitely generatebatches of augmented image datatrain_generator = train_datagen.flow_from_directory( this is a similar generator, for validation datavalidation_generator = test_datagen.flow_from_directory( |
This comment has been minimized.
This comment has been minimized.
running this code without changes. Using TensorFlow backend. Traceback (most recent call last): |
This comment has been minimized.
This comment has been minimized.
Does anyone knows how to do k fold cross-validation using the code sample given above? |
This comment has been minimized.
This comment has been minimized.
tiff image cannot load! |
This comment has been minimized.
This comment has been minimized.
Question: Want to do a binary classification.I do have a dataset for it.But my classes are not among the 1000 classes that the Imagenet model has.Will the Transfer learning and fine-tuning still works ?? |
This comment has been minimized.
This comment has been minimized.
@UTTAM2311 features learned from training a deep network on ImageNet are relevant to other unsean images/classes so tranfer learning and fine tuning would still work on new datasets |
This comment has been minimized.
This comment has been minimized.
@lucastanziano Have you tried |
This comment has been minimized.
This comment has been minimized.
How to give custom input and validate |
This comment has been minimized.
This comment has been minimized.
OverflowError: Range exceeds valid bound with Dense layer to solve this error put this in the beginning of the code
|
This comment has been minimized.
This comment has been minimized.
@KamalOthman |
This comment has been minimized.
This comment has been minimized.
@austinchencym |
This comment has been minimized.
This comment has been minimized.
Hi Everyone When I checked the shape of the image, I got (w, h), Strangely, I do not get error when I keep the input_shape (3, w, h). Is not it strange? How can I expand the images dimension within flow_from_directory? Anyone can help please? Thanks |
This comment has been minimized.
This comment has been minimized.
Thanks for the excellent tutorial. One issue: For the data downloaded from Kaggle directly, the code won't run through. Also, the validation sets should include cats labeled from 1000 to 1399, and dogs similarly, if the size is set to 400. |
This comment has been minimized.
This comment has been minimized.
Hi Thanks |
This comment has been minimized.
This comment has been minimized.
I executed the classifier_from_little_data_script_1.py program. However, I don't know how to make a prediction after we have build and trained the model.. How to do it? |
This comment has been minimized.
This comment has been minimized.
tks for sharing ur experience :) |
This comment has been minimized.
This comment has been minimized.
@ynngliu Thanks! That worked. |
This comment has been minimized.
This comment has been minimized.
Hi, Now how can I load the model and test it on the test images ? If anyone can help me with some code snippets it will be helpful. Thanks |
This comment has been minimized.
This comment has been minimized.
@ynngliu Thanks, that works. |
This comment has been minimized.
This comment has been minimized.
See this notebook for an example of fine-tuning a Note, I'm using the You can find the original gist at https://gist.github.com/embanner/6149bba89c174af3bfd69537b72bca74. |
This comment has been minimized.
This comment has been minimized.
Is target_size a necessary parameter? I'm using SPP layer thus there is no need to resize input shape, but it doesn't seem to work with flow_from_directory. Can somebody help? |
This comment has been minimized.
This comment has been minimized.
I was getting an ''TypeError: sigmoid_cross_entropy_with_logits() got an unexpected keyword argument" error at the model.compile(...) line. Turns out you need to be using Tensorflow 1.0 or later. Problem disappeared when I upgraded TF. |
This comment has been minimized.
This comment has been minimized.
Hello, I copied this code to convnet.py and I added the images to the data folder, but when I run by saying 'python convnet.py' i get this error What can I do to fix this? |
This comment has been minimized.
This comment has been minimized.
Data prep helper to go from Kaggle to the layout he has:
Just set different values for each run:
PS: Script can be made simpler! |
This comment has been minimized.
This comment has been minimized.
Hi,
|
This comment has been minimized.
This comment has been minimized.
Hello, `### import keras dimensions of our images.img_width, img_height = 150, 150 train_data_dir = 'all_ball_database/train/' model = Sequential() model.add(Convolution2D(32, 3, 3)) model.add(Convolution2D(64, 3, 3)) model.add(Flatten()) model.summary() model.compile(loss='categorical_crossentropy', ######## this is the augmentation configuration we will use for trainingtrain_datagen = ImageDataGenerator( test_datagen = ImageDataGenerator(rescale=1./255) train_generator = train_datagen.flow_from_directory( validation_generator = test_datagen.flow_from_directory( model.fit_generator( model.save_weights('first_try_all_ball_test1.h5') Result obtainedFound 9000 images belonging to 6 classes. |
This comment has been minimized.
This comment has been minimized.
@PareshKamble |
This comment has been minimized.
This comment has been minimized.
I have a dataset of 10000 shoeprint images, 5000 left and 5000 right. The images are grayscale and of different sizes. I tried running this code but on every epoch, the accuracy and loss remains approximately the same, 50% and 0.69 respectively. The only difference in the code that I have is that I do not have labelled validation data so I removed that option from model.fit_generator. Can you suggest what I may be doing wrong? |
This comment has been minimized.
This comment has been minimized.
@durgeshm
I will try with 'softmax' as well. |
This comment has been minimized.
This comment has been minimized.
Excuse me, what is rescale, shear_range, zoom_range and forizontal_filp actually doing respectively? I am a DNN newbie. |
This comment has been minimized.
This comment has been minimized.
Why Also is there any example of L1\L2 regularization? |
This comment has been minimized.
This comment has been minimized.
Hello there @fchollet and everyone else :) Lovely tutorial!! I am pretty new to Keras and to the deep learning realm. I have several questions to ask about the script written above: 1- I have taken a look at the different images within the provided dataset and realized that they are not all the same size. So the question is "does the code resizes them when we specify the input_size?" 2- Does this program actually generate new images from the ones we have saved in the data folder? (because after running the script i did not get anything new except the 'first_try.h5' weights file. 3_ Is the sequential model in the first part of the code made to classify or just to generate the new images? Or in another way, could i use a VGG_Net or GoogleNet instead of it? Thank you for your time. |
This comment has been minimized.
This comment has been minimized.
@mrgloom i think sigmoid was used because we only have 2 classes and using a binary classification :) |
This comment has been minimized.
This comment has been minimized.
Thanks a lot for this nice tutorial. |
This comment has been minimized.
This comment has been minimized.
For doing predictions this worked for me:
|
This comment has been minimized.
This comment has been minimized.
if it is not the directory structure and all the images are in a same directory, how to deal with this? |
This comment has been minimized.
This comment has been minimized.
Thanks @AurelianTactics prediction worked for me too! |
This comment has been minimized.
This comment has been minimized.
@AurelianTactics, I've tried your code but my error message is : "No model found in config file. Could anyone, help me please ? :) _from keras.models import load_model test_model = load_model('my_model_name.h5') |
This comment has been minimized.
This comment has been minimized.
Hi @AurelianTactics, your code it works 1/1 [==============================] - 0s 1/1 [==============================] - 0s How do I determine if it is a dog or a cat? |
This comment has been minimized.
This comment has been minimized.
Confirming The output Here is my update for this script and for prediction. For this first script, I have 20 test images, 10 cats, 10 dogs (not part of the training or validation set). Predicted results: 6 images out of 20 were predicted incorrectly. |
This comment has been minimized.
This comment has been minimized.
How do I make a difference if it is not a dog or a cat? |
This comment has been minimized.
This comment has been minimized.
Is anyone able to explain @vutsalsinghal and @Asgu95's question? I am getting the same result when making predictions. Thank you.
I have 3 different folders in |
This comment has been minimized.
This comment has been minimized.
Figured it out. @vutsalsinghal and @Asgu95. I believe the result is of type |
This comment has been minimized.
This comment has been minimized.
Hello everyone! can we use the training data from '.npy' file instead of jpeg/png for ImageDataGenerator? |
This comment has been minimized.
This comment has been minimized.
Hello Everyone, I tried to follow the tutorial as it is with my own datasets of Cats and Dogs on CPU. I am getting error " Segmnentation fault after I train the model". Enclosed here is the output. For testing I have used only 5 epochs with a batch size of 4. $ python classifier_from_little_data_script_1.py Can someone help me with this? Thank you. |
This comment has been minimized.
This comment has been minimized.
Hello guys :) I would like to know how to set-up a callback function in order to evaluate my testing set of data at each epoch after the training is being done between the Training set and validation set. I got 90% validation accuracy while performing the code above (modified to 5 classes), but when i also try to evaluate the testing data, i get only 30% accuracy. Here is how i coded the evaluation at the end of all the training: #def test_the_model(): bottleneck_features_testing = np.load(open('bottleneck_features_testing_5_classes.npy', "rb"))model2 = load_model('my_model_5_classes.h5')model2.compile(optimizer='adam',loss='categorical_crossentropy', metrics=['accuracy'])testing_labels = np.array([1,0,0,0,0] * 1523 + [0,1,0,0,0] * 1343 + [0,0,1,0,0] * 1557 + [0,0,0,1,0] * 1365 + [0,0,0,0,1] * 5492)testing_labels = testing_labels.reshape(11280,5)bottleneck_features_testing_ev = model2.evaluate(bottleneck_features_testing, testing_labels, verbose=0)print(bottleneck_features_testing_ev)Hence, i would like to check the evaluation accuracy (testing accuracy) at each epoch in order to visualize at which epoch the over-fitting starts to take place. You may find below my code (which is definitely not perfected yet :) ): NOTE: here the TestCallback() has been taken from keras-team/keras#2548, but here we need to adapt it i guess, because the input to the model.evaluate() function is not the raw image. import h5py dimensions of our images.img_width, img_height = 150, 150 weights_path = 'vgg16_weights.h5' train_data_dir = 'training2' nb_train_samples = 50000 epochs = 50 class TestCallback(Callback):
def save_bottlebeck_features():
#generating the training data
THEN BCA_A, THEN BDD, THEN NONE; the predict_generator method returns the output of a model, givena generator that yields batches of numpy data
def train_top_model():
Callbacks definition: 1st saves weights at the end of every epoch; 2nd creates the TensorBoard data
!#! If you want to visualize the files created during training, run in your terminal:tensorboard --logdir path_to_current_dir/Graph
|
This comment has been minimized.
This comment has been minimized.
This tutorial literally saved me days of researches, really hope there's more like this! |
This comment has been minimized.
This comment has been minimized.
For this code:
I am getting
This makes sense to me, because the first dimension is initially Am I getting something wrong? |
This comment has been minimized.
This comment has been minimized.
Hi,I am trying to trained 100 classes,but I got following things when I train it. I changed like this: nb_train_samples = 8153 always got negative number and it did not trands to 0,how can I do?Thanks! |
This comment has been minimized.
This comment has been minimized.
I was getting a similar error about negative dimensions on the MaxPool layer. After looking at the cat ~/.keras/keras.json file, I see that there is a parameter "image_data_format": "channels_last". The code we are given has the channels (3) first, so change the input shape to be (150,150,3). |
This comment has been minimized.
This comment has been minimized.
dumb question but is there no targets? like the pics are dogs and cats, but what tells it which image is which during the classification |
This comment has been minimized.
This comment has been minimized.
While running this script (thanks to all the feedback and suggestions posted above) I got two warnings: UserWarning: Update your UserWarning: The semantics of the Keras 2 argument UserWarning: Update your But otherwise the code finished without problem and a model is generated. Should I be concerned about the above warnings? |
This comment has been minimized.
This comment has been minimized.
@TannerS I think that is determined by your train data folder structure. |
This comment has been minimized.
This comment has been minimized.
Anyone down for a tutorial, or more specifically teach me step by step how to solve this problem? I am willing to compensate ($$). I am having all sort of issues trying to run this code. Thank you |
This comment has been minimized.
This comment has been minimized.
can you give me how to run this code for dcgan as feature extractor? |
This comment has been minimized.
This comment has been minimized.
Hi Everyone, _Epoch 1/20 Traceback (most recent call last): slurmstepd: error: Exceeded step memory limit at some point. |
This comment has been minimized.
This comment has been minimized.
You get target_size=(img_width, img_height) incorrecly. and input_shape should be
In this example it does not matter, but i got some trouble with rectangular images. |
This comment has been minimized.
This comment has been minimized.
Hi, `Using TensorFlow backend. ..... Epoch 8/50 |
This comment has been minimized.
This comment has been minimized.
I ran this exact code, but the accuracy stays 0.5. What could be the reason? My keras version is 2.0.9. This seems to be close to what @yinniyu gets. Did you @yinniyu figure out whay? Found 2000 images belonging to 2 classes. |
This comment has been minimized.
This comment has been minimized.
I'm also seeing the val_acc suddenly drop after 10 or so epochs. Anyone have an explanation to this? Learning rate too high? |
This comment has been minimized.
This comment has been minimized.
@yinniyu |
This comment has been minimized.
This comment has been minimized.
@leopsidom, @Kolous |
This comment has been minimized.
This comment has been minimized.
Using TensorFlow backend. |
This comment has been minimized.
This comment has been minimized.
I used the same code for a small data set but after running this piece of code model.fit_generator( model.save_weights('first_try.h5') I am getting error - Epoch 1/7UnboundLocalError Traceback (most recent call last) /Users/madhuri/conda/lib/python3.6/site-packages/keras/legacy/interfaces.py in wrapper(*args, **kwargs) /Users/madhuri/conda/lib/python3.6/site-packages/keras/models.py in fit_generator(self, generator, steps_per_epoch, epochs, verbose, callbacks, validation_data, validation_steps, class_weight, max_queue_size, workers, use_multiprocessing, shuffle, initial_epoch) /Users/madhuri/conda/lib/python3.6/site-packages/keras/legacy/interfaces.py in wrapper(*args, **kwargs) /Users/madhuri/conda/lib/python3.6/site-packages/keras/engine/training.py in fit_generator(self, generator, steps_per_epoch, epochs, verbose, callbacks, validation_data, validation_steps, class_weight, max_queue_size, workers, use_multiprocessing, shuffle, initial_epoch) /Users/madhuri/conda/lib/python3.6/site-packages/keras/legacy/interfaces.py in wrapper(*args, **kwargs) /Users/madhuri/conda/lib/python3.6/site-packages/keras/engine/training.py in evaluate_generator(self, generator, steps, max_queue_size, workers, use_multiprocessing) UnboundLocalError: local variable 'outs' referenced before assignment |
This comment has been minimized.
This comment has been minimized.
Hello, great tutorial. classe 1 with 80 samples for training and 35 for validation code modified : #model.compile(loss='binary_crossentropy', optimizer='rmsprop', metrics=['accuracy']) #train_generator = train_datagen.flow_from_directory( train_data_dir, target_size=(img_width, img_height), batch_size=batch_size, class_mode='binary') #validation_generator = test_datagen.flow_from_directory( validation_data_dir, target_size=(img_width, img_height), batch_size=batch_size, class_mode='binary') `
Thanks for your help |
This comment has been minimized.
This comment has been minimized.
filenotfound: [winerror31] the system cannot find the path specified: 'data/train.............. |
This comment has been minimized.
This comment has been minimized.
i am getting this error Epoch 1/50 File "", line 6, in File "C:\Anaconda3\envs\tensorflowww\lib\site-packages\keras\legacy\interfaces.py", line 91, in wrapper File "C:\Anaconda3\envs\tensorflowww\lib\site-packages\keras\models.py", line 1256, in fit_generator File "C:\Anaconda3\envs\tensorflowww\lib\site-packages\keras\legacy\interfaces.py", line 91, in wrapper File "C:\Anaconda3\envs\tensorflowww\lib\site-packages\keras\engine\training.py", line 2145, in fit_generator File "C:\Anaconda3\envs\tensorflowww\lib\site-packages\keras\utils\data_utils.py", line 561, in get File "", line 3, in raise_from StopIteration: Could not import PIL.Image. The use of model.fit_generator( File "", line 6, in File "C:\Anaconda3\envs\tensorflowww\lib\site-packages\keras\legacy\interfaces.py", line 91, in wrapper File "C:\Anaconda3\envs\tensorflowww\lib\site-packages\keras\models.py", line 1256, in fit_generator File "C:\Anaconda3\envs\tensorflowww\lib\site-packages\keras\legacy\interfaces.py", line 91, in wrapper File "C:\Anaconda3\envs\tensorflowww\lib\site-packages\keras\engine\training.py", line 2145, in fit_generator File "C:\Anaconda3\envs\tensorflowww\lib\site-packages\keras\utils\data_utils.py", line 561, in get File "", line 3, in raise_from StopIteration: Could not import PIL.Image. The use of |
This comment has been minimized.
This comment has been minimized.
@Ajithkumar011297 you put your dataset in
also since you get @Anxit you need to install PIL correctly. This usually requires a binary installation in addition to the pip installation. See https://wp.stolaf.edu/it/installing-pil-pillow-cimage-on-windows-and-mac/ |
This comment has been minimized.
This comment has been minimized.
Hello everyone, everything good? I have a dataset 224x224x3. With 60000 images. 44000 pictures of training and 16000 of tests. Transformed to float32, the computer does not accept. The "memory explodes". So I decided to use the above example of keras. I believe the above example of keras is using all my data in the form of batch size. I believe he is not generating more images. Then my code looks like this: train_datagen = ImageDataGenerator (rescale = 1.255) train_generator = train_datagen.flow_from_directory ( test_generator = test_datagen.flow_from_directory ( validation_generator = validation_datagen.flow_from_directory ( for data_batch, labels_batch in train_generator: historyTrain = model.fit_generator ( But I have some doubts.
I thank you for your attention, |
This comment has been minimized.
This comment has been minimized.
Hello, I am new using Keras deep learning, I would Like to know how can I fix a image folder for use model.predict_classes(). This example did not include that part. |
This comment has been minimized.
This comment has been minimized.
Hi, tried this with the full cat and dog set (20000 train and 5000 val pictures) and got to 83% val accuracy. (I changed it from binary to categorical classifier and used softmax on the last layer) Sounds fine, but upon checking with LIME I see that the network very often thinks that the floor or background is part of a cat or dog, or thinks cat's shouldn't have pointy ears, etc., meaning the network has not learned the real difference between a cat and a dog! Any ideas how to improve this without transfer learning? I thought 25000 pictures should be enough, but apperently not? Network also seems big enough (even added a conv and dense layer)? What else can I try? PS: Thank you for putting up those tutorials, very helpful! |
This comment has been minimized.
This comment has been minimized.
Found 8005 images belonging to 2 classes. ImportError Traceback (most recent call last) c:\users\admin\appdata\local\programs\python\python36\lib\multiprocessing\pool.py in get(self, timeout) c:\users\admin\appdata\local\programs\python\python36\lib\multiprocessing\pool.py in worker(inqueue, outqueue, initializer, initargs, maxtasks, wrap_exception) c:\users\admin\appdata\local\programs\python\python36\lib\site-packages\keras\utils\data_utils.py in get_index(uid, i) c:\users\admin\appdata\local\programs\python\python36\lib\site-packages\keras\preprocessing\image.py in getitem(self, idx) c:\users\admin\appdata\local\programs\python\python36\lib\site-packages\keras\preprocessing\image.py in _get_batches_of_transformed_samples(self, index_array) c:\users\admin\appdata\local\programs\python\python36\lib\site-packages\keras\preprocessing\image.py in load_img(path, grayscale, target_size, interpolation) ImportError: Could not import PIL.Image. The use of The above exception was the direct cause of the following exception: StopIteration Traceback (most recent call last) c:\users\admin\appdata\local\programs\python\python36\lib\site-packages\keras\legacy\interfaces.py in wrapper(*args, **kwargs) c:\users\admin\appdata\local\programs\python\python36\lib\site-packages\keras\models.py in fit_generator(self, generator, steps_per_epoch, epochs, verbose, callbacks, validation_data, validation_steps, class_weight, max_queue_size, workers, use_multiprocessing, shuffle, initial_epoch) c:\users\admin\appdata\local\programs\python\python36\lib\site-packages\keras\legacy\interfaces.py in wrapper(*args, **kwargs) c:\users\admin\appdata\local\programs\python\python36\lib\site-packages\keras\engine\training.py in fit_generator(self, generator, steps_per_epoch, epochs, verbose, callbacks, validation_data, validation_steps, class_weight, max_queue_size, workers, use_multiprocessing, shuffle, initial_epoch) c:\users\admin\appdata\local\programs\python\python36\lib\site-packages\keras\utils\data_utils.py in get(self) c:\users\admin\appdata\local\programs\python\python36\lib\site-packages\six.py in raise_from(value, from_value) StopIteration: Could not import PIL.Image. The use of I HAVE INSTALLED PILLOW,still its not working |
This comment has been minimized.
This comment has been minimized.
I have finished training and validation, How can i test my classifier? |
This comment has been minimized.
This comment has been minimized.
How do you provide labels for training? |
This comment has been minimized.
This comment has been minimized.
thanks for this script i tested it for flowers datasets it was work very well ... |
This comment has been minimized.
This comment has been minimized.
Hi, @randhawp I have seen your post and want to know how can I apply K fold cross validation on the given code |
This comment has been minimized.
This comment has been minimized.
I tried to use gpu : +-----------------------------------------------------------------------------+ Why is the usage rate only 10%? |
This comment has been minimized.
This comment has been minimized.
I have the same problem. |
This comment has been minimized.
This comment has been minimized.
Accuracy not rising over 0.5000 |
This comment has been minimized.
This comment has been minimized.
I wonder, don't we need the 'label' ? There is no such vector "y_train" being used in the code? |
This comment has been minimized.
This comment has been minimized.
dimensions of our images.img_width, img_height = 1700, 2200 train_data_dir = 'data/train' This is the error i am getting Kindly anyone help me, Thank you |
This comment has been minimized.
This comment has been minimized.
@leopsidom, @Kolous, @FlamesoFF, @yinniyu, @Dugudaibo Accuracy not rising over 0.5000 probably because your dataset is composed of 50% images belonging to a class and 50% belonging to another class: the real problem is that for some reason your model always predicts the same class for any images. |
This comment has been minimized.
This comment has been minimized.
Hi, how many classes do you have? 2 or 11? model.add(Dense(1, activation='sigmoid')) and for a single-input model with 11 classes (categorical classification) you should use : model.add(Dense(11, activation='softmax')) |
This comment has been minimized.
This comment has been minimized.
I'm on Google Colab and I'm finding it extremely difficult to replicate the folder structure. I'm using kaggle api and unzipping them, but the file names are not very conducive to the folder structure. What is everyone else doing? |
This comment has been minimized.
This comment has been minimized.
How to test this model. Actually I am a new learner of this thing. |
This comment has been minimized.
This comment has been minimized.
your kernel size is larger than the size of the image - either at input or even as you move through the CNN. |
This comment has been minimized.
This comment has been minimized.
how to use it to train digits I have 10 folders under train and validation code ran successfully but in result I got "[array([[1]], dtype=int32), array([[1.]], dtype=float32)]" |
This comment has been minimized.
This comment has been minimized.
i am using this fit_generator method -- classifier.fit_generator(training_set, and getting this error -- File "", line 5, in File "C:\Users\dell\Anaconda3\lib\site-packages\keras\legacy\interfaces.py", line 91, in wrapper File "C:\Users\dell\Anaconda3\lib\site-packages\keras\engine\training.py", line 1418, in fit_generator File "C:\Users\dell\Anaconda3\lib\site-packages\keras\engine\training_generator.py", line 40, in fit_generator File "C:\Users\dell\Anaconda3\lib\site-packages\keras\engine\training.py", line 519, in _make_train_function File "C:\Users\dell\Anaconda3\lib\site-packages\keras\backend\tensorflow_backend.py", line 2744, in function File "C:\Users\dell\Anaconda3\lib\site-packages\keras\backend\tensorflow_backend.py", line 2575, in init ValueError: ('Some keys in session_kwargs are not supported at this time: %s', dict_keys(['matris'])) can anybody help? |
This comment has been minimized.
This comment has been minimized.
How to check model accuracy? |
This comment has been minimized.
This comment has been minimized.
@marco-zorzi thank you! |
This comment has been minimized.
This comment has been minimized.
Hi,@fchollet and anyone |
This comment has been minimized.
This comment has been minimized.
its model.save_weights('first_try.h5'). |
This comment has been minimized.
This comment has been minimized.
I am still confused how does one add : a) K-fold cross validation |
This comment has been minimized.
This comment has been minimized.
@fchollet hope u are doing well....
Kindly check the code....... |
This comment has been minimized.
This comment has been minimized.
I would like go implement a hierarchical resnet architecture. However, I could not find any solution for this. For example, my data structure is like: class A I know how to compile the model and do fit_generator, but do not get the intuition behind the model creation.
|
This comment has been minimized.
This comment has been minimized.
What is validation data? Is it the same with train data? |
This comment has been minimized.
This comment has been minimized.
My validation data is the same as train data. I use ImageDataGenerator for splitting.
I was thinking to train first a model on N age groups (0-10,11-18,...) and then train N models for each age group (model 1 (0-10), model 2 (11-18), model 3 (..)) and then combine those 5 models into 1 model, however I do not know how to combine those N models as 1 model. Because if 1 model has 10 prediction layers and another model 15 etc.. The final model must have 25 prediction output, so the predictions outputs are also combined into 1 large output (100 prediction size output for the combination of all the model). |
This comment has been minimized.
This comment has been minimized.
I see some samples , not use validation data. |
This comment has been minimized.
This comment has been minimized.
I see number image of valid data is smaller than train data. |
This comment has been minimized.
This comment has been minimized.
Has anyone written small prediction script for testing the data under test folder (if given one?) |
This comment has been minimized.
This comment has been minimized.
Hello everyone, |
This comment has been minimized.
This comment has been minimized.
Hello How do you use .load_weights('example.h5') do you have an example ? Thanks ! |
This comment has been minimized.
This comment has been minimized.
hello! I wonder if the file type must be .h5 in the sentence "model.save_weights('first_try.h5')" ? Thanks !! |
This comment has been minimized.
This comment has been minimized.
Hi, I generated the confusion matrix on prediction result and that is [[2500 0] [2500 0] ]. I think it's not up to the mark. I'm training dogs and cats , training images are 20000 and validation images are 5000. |
This comment has been minimized.
This comment has been minimized.
Can you tell me how you generated the confusion matrix? |
This comment has been minimized.
This comment has been minimized.
Hello.did you implement the one hot encoding in your code?if so then will you please help me out and is it a must to use one hot encoding in multi class image classification?? Thank you |
This comment has been minimized.
This comment has been minimized.
I have this same question.can anyone please clarify? Thanks |
This comment has been minimized.
This comment has been minimized.
I met the same problem! Did you solve it. |
This comment has been minimized.
This comment has been minimized.
Hi, I am doing Python and classification both the first time. Can anyone tell me is it necessary to keep the index as part of the image name or as long as I have a unique image name it will work? |
This comment has been minimized.
thanks for great tutorial! I think the last line of code should be model_save_weights('first_try.h5').