This is the C3D model used with a fork of Caffe to the Sports1M dataset migrated to Keras. Details about the network architecture can be found in the following arXiv paper:
Tran, Du, et al. "Learning Spatiotemporal Features With 3D Convolutional Networks." Proceedings of the IEEE International Conference on Computer Vision. 2015.
Download: weights
I ported this into Keras+Tensorflow (not Theano-backend) for those who are interested: https://github.com/chuckcho/c3d-keras
^ The only difference in my implementation with Tensorflow is the mean subtraction, which led to even better results:
Top 5 probabilities and labels:
basketball: 0.71422
streetball: 0.10293
volleyball: 0.04900
greco-roman wrestling: 0.02638
freestyle wrestling: 0.02408
@chuckcho Hi, Thank you for you sharing. I downloaded your code and tried to run it. It seems that the downloaded file caffe.proto cannot be compiled properly. The warning message "[libprotobuf WARNING google/protobuf/compiler/parser.cc:546] No syntax specified for the proto file: caffe.proto. Please use 'syntax = "proto2";' or 'syntax = "proto3";' to specify a syntax version. (Defaulted to proto2 syntax.)". I added ""syntax = "proto2";" to fix this. I wonder whether there is any difference between them?
Hi Alberto,
Great job ! Thanks for sharing !
How do you know that ZeroPadding3D (zeropadding3d) (None, 512, 2, 9, 9) 0 layer is used ? I was not able to find it in the original Caffe model ... Could you please point the place where it is ?
@yzexeter sorry for the late reply. yes, the caffe.proto is in v2 format (and to be compiled as such), and it seems v3 has non-backward-compatible features. http://stackoverflow.com/questions/33204321/upgrading-protobuf-from-version-2-to-3-incompatible-with-protobuf-default-valu
Can you share your code about converting sport1m caffe model to its tensorflow version? I tried to use this code https://github.com/ethereon/caffe-tensorflow, which did not support the video data layer. Thanks. @chuckcho
Hi Alberto,
Could you tell whether you convert the images to RGB as by default openCV cap reads them in GBR format.
Do you know on which of them conv3D was trained?
Thanks
Hi,
I have loaded your transfered model using bellow command:
model = model_from_json(open('./sports_1M.json', 'r').read())
and I have received this error:
ValueError: Improper config format.
Could you please tell are there any problems with .json file?
Hi, I have your same problem @MOHAMMAD-PY, did you found a solution for that?
is it possible to retrain the network again?
@smarinka Sorry for the delay, but the original model was trained in GBR format.
@albertomontesg: Thank you for your sharing! I am studying the Keras, I used the UCF101 dataset to run and get a result, now I want to use SVM to classify but I don't know how to prepare input file for C3D model in Keras. Could you please help me. Thank you so much!
Hi,
Could you specify the version of your python, your Keras, and Theano? I am running it here on June 14 2018. Now the latest version cannot be compatible with the old one, so when I read your model json file and h5 file, it shows error. It also shows error when I define the model structure by myself and load h5 file.
@ jian @jiang2764 Hi, have u fixed this problem? I have tried to convert caffe's model of 3D convolution by using https://github.com/ethereon/caffe-tensorflow , I also failed.
looking forward to any replies.
@albertomontesg Hi, Thank you for your great job. I have a problem loading the weights in my model. I thought it might be because I use tensorflow backend for my keras, so I decided to convert the weights, but I still get the same error:
ValueError: You are trying to load a weight file containing 0 layers into a model with 11 layers.
Could you please help where is the problem?
Hi,
Could you specify the version of your python, your Keras, and Theano? I am running it here on June 14 2018. Now the latest version cannot be compatible with the old one, so when I read your model json file and h5 file, it shows error. It also shows error when I define the model structure by myself and load h5 file.
hi, did you solve that?
Hi,
The links on weights and model to keras model you gave have some issues. Or my versions should be missmatched. Here are my libraries and versions
When using json model here is the error
ValueError: Improper config format:
When using previously created model (by adding layers manually) to lead h5 file weights here is the error
ValueError: You are trying to load a weight file containing 0 layers into a model with 11 layers.
Looks like the weights files cant be decoded properly. Any Idea on what to do?
@supun-kanda Based on this project: https://github.com/axon-research/c3d-keras
I created an updated version of the model (for keras 2.2.4): https://github.com/aslucki/C3D_Sport1M_keras
and weights.
@albertomontesg
Hi, I am facing this error. Did anyone else face this issue?
AttributeError Traceback (most recent call last)
in
22 weights_b = np.array(layer.blobs[1].data, dtype=np.float32)
23 weights_p = np.array(layer.blobs[0].data, dtype=np.float32).reshape(
---> 24 layer.blobs[0].num, layer.blobs[0].channels, layer.blobs[0].length,
25 layer.blobs[0].height, layer.blobs[0].width
26 )
AttributeError: length
I am new to caffe and getting attributes from blob using hpp files is sort of not understandable to me here.
Thanks in advance.
@albertomontesg
Is there any training code for the sports-1m dataset?
how can we use the C3D features to model violence detection on videos?
Hi @mzolfaghari,
Actually I don't know any tool to do that. If you want to give a try, there is a repo that ports from Caffe to Keras, so digging into the code, you may find the way to do it on the inverse way.
Best,