Skip to content

Instantly share code, notes, and snippets.

@albertomontesg
Last active October 15, 2024 20:40
Show Gist options
  • Save albertomontesg/d8b21a179c1e6cca0480ebdf292c34d2 to your computer and use it in GitHub Desktop.
Save albertomontesg/d8b21a179c1e6cca0480ebdf292c34d2 to your computer and use it in GitHub Desktop.
C3D Model for Keras

C3D Model for Keras

This is the C3D model used with a fork of Caffe to the Sports1M dataset migrated to Keras. Details about the network architecture can be found in the following arXiv paper:

Tran, Du, et al. "Learning Spatiotemporal Features With 3D Convolutional Networks." Proceedings of the IEEE International Conference on Computer Vision. 2015.

Content

Download: weights

Display the source blob
Display the rendered blob
Raw
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@albertomontesg
Copy link
Author

albertomontesg commented Apr 25, 2016

@raingo I hadn't it converted at that moment, but if you are intereseted I have published all of this in this repository which includes weights, model and mean:
keras-model-zoo C3D model

@mzolfaghari
Copy link

Hi,
Is there any way to convert Keras model to Caffe model?

Best,

@albertomontesg
Copy link
Author

albertomontesg commented Jun 12, 2016

Hi @mzolfaghari,
Actually I don't know any tool to do that. If you want to give a try, there is a repo that ports from Caffe to Keras, so digging into the code, you may find the way to do it on the inverse way.

Best,

@chuckcho
Copy link

chuckcho commented Feb 15, 2017

I ported this into Keras+Tensorflow (not Theano-backend) for those who are interested: https://github.com/chuckcho/c3d-keras

@chuckcho
Copy link

^ The only difference in my implementation with Tensorflow is the mean subtraction, which led to even better results:

Top 5 probabilities and labels:
basketball: 0.71422
streetball: 0.10293
volleyball: 0.04900
greco-roman wrestling: 0.02638
freestyle wrestling: 0.02408

@yzexeter
Copy link

yzexeter commented Mar 5, 2017

@chuckcho Hi, Thank you for you sharing. I downloaded your code and tried to run it. It seems that the downloaded file caffe.proto cannot be compiled properly. The warning message "[libprotobuf WARNING google/protobuf/compiler/parser.cc:546] No syntax specified for the proto file: caffe.proto. Please use 'syntax = "proto2";' or 'syntax = "proto3";' to specify a syntax version. (Defaulted to proto2 syntax.)". I added ""syntax = "proto2";" to fix this. I wonder whether there is any difference between them?

@yuijim
Copy link

yuijim commented Mar 17, 2017

Hi Alberto,
Great job ! Thanks for sharing !
How do you know that ZeroPadding3D (zeropadding3d) (None, 512, 2, 9, 9) 0 layer is used ? I was not able to find it in the original Caffe model ... Could you please point the place where it is ?

@chuckcho
Copy link

@yzexeter sorry for the late reply. yes, the caffe.proto is in v2 format (and to be compiled as such), and it seems v3 has non-backward-compatible features. http://stackoverflow.com/questions/33204321/upgrading-protobuf-from-version-2-to-3-incompatible-with-protobuf-default-valu

@zhuolinumd
Copy link

Can you share your code about converting sport1m caffe model to its tensorflow version? I tried to use this code https://github.com/ethereon/caffe-tensorflow, which did not support the video data layer. Thanks. @chuckcho

@smarinka
Copy link

smarinka commented Jun 5, 2017

Hi Alberto,
Could you tell whether you convert the images to RGB as by default openCV cap reads them in GBR format.
Do you know on which of them conv3D was trained?
Thanks

@mmderakhshani
Copy link

Hi,
I have loaded your transfered model using bellow command:
model = model_from_json(open('./sports_1M.json', 'r').read())
and I have received this error:
ValueError: Improper config format.
Could you please tell are there any problems with .json file?

@dataintensiveapplication

Hi, I have your same problem @MOHAMMAD-PY, did you found a solution for that?

@Zumbalamambo
Copy link

is it possible to retrain the network again?

@albertomontesg
Copy link
Author

@smarinka Sorry for the delay, but the original model was trained in GBR format.

@TrungHieu-Le
Copy link

@albertomontesg: Thank you for your sharing! I am studying the Keras, I used the UCF101 dataset to run and get a result, now I want to use SVM to classify but I don't know how to prepare input file for C3D model in Keras. Could you please help me. Thank you so much!

@bochen1106
Copy link

Hi,

Could you specify the version of your python, your Keras, and Theano? I am running it here on June 14 2018. Now the latest version cannot be compatible with the old one, so when I read your model json file and h5 file, it shows error. It also shows error when I define the model structure by myself and load h5 file.

@Ai-is-light
Copy link

@ jian @jiang2764 Hi, have u fixed this problem? I have tried to convert caffe's model of 3D convolution by using https://github.com/ethereon/caffe-tensorflow , I also failed.
looking forward to any replies.

@lDark-Moonl
Copy link

lDark-Moonl commented Nov 17, 2018

@albertomontesg Hi, Thank you for your great job. I have a problem loading the weights in my model. I thought it might be because I use tensorflow backend for my keras, so I decided to convert the weights, but I still get the same error:

ValueError: You are trying to load a weight file containing 0 layers into a model with 11 layers.

Could you please help where is the problem?

@EMCL
Copy link

EMCL commented Dec 10, 2018

Hi,

Could you specify the version of your python, your Keras, and Theano? I am running it here on June 14 2018. Now the latest version cannot be compatible with the old one, so when I read your model json file and h5 file, it shows error. It also shows error when I define the model structure by myself and load h5 file.

hi, did you solve that?

@supun-kanda
Copy link

supun-kanda commented Jan 23, 2019

Hi,
The links on weights and model to keras model you gave have some issues. Or my versions should be missmatched. Here are my libraries and versions

  • caffe-gpu=1.0=py36h51fbcb3_4
  • cudatoolkit=9.2=0
  • cudnn=7.2.1=cuda9.2_0
  • cupti=9.2.148=0
  • ffmpeg=4.0=hcdf2ecd_0
  • freeglut=3.0.0=hf484d3e_5
  • gast=0.2.0=py36_0
  • gflags=2.2.2=he6710b0_0
  • jasper=2.0.14=h07fcdf6_1
  • keras-applications=1.0.6=py36_0
  • keras-base=2.2.4=py36_0
  • keras-gpu=2.2.4=0
  • keras-preprocessing=1.0.5=py36_0
  • libboost=1.67.0=h46d08c1_4
  • libglu=9.0.0=hf484d3e_1
  • libopencv=3.4.2=hb342d67_1
  • libopus=1.3=h7b6447c_0
  • libprotobuf=3.6.1=hd408876_0
  • lmdb=0.9.22=hf484d3e_1
  • markdown=3.0.1=py36_0
  • protobuf=3.6.1=py36he6710b0_0
  • py-boost=1.67.0=py36h04863e7_4
  • py-opencv=3.4.2=py36hb342d67_1
  • python-gflags=3.1.2=py36_0
  • python-leveldb=0.194=py36_1
  • tensorboard=1.12.0=py36hf484d3e_0
  • tensorflow=1.12.0=gpu_py36he74679b_0
  • tensorflow-base=1.12.0=gpu_py36had579c0_0
  • tensorflow-gpu=1.12.0=h0d30ee6_0
  • termcolor=1.1.0=py36_1
  • conda=4.5.12=py36_0
  • conda-build=3.17.6=py36_0
  • conda-env=2.6.0=1
  • jupyter=1.0.0=py36_7
  • jupyter_client=5.2.4=py36_0
  • jupyter_console=6.0.0=py36_0
  • jupyter_core=4.4.0=py36_0
  • jupyterlab=0.35.3=py36_0
  • jupyterlab_server=0.2.0=py36_0
  • numpy=1.15.4=py36h7e9f1db_0
  • numpy-base=1.15.4=py36hde5b4d6_0
  • pillow=5.3.0=py36h34e0f95_0
  • pip=18.1=py36_0
  • python=3.6.7=h0371630_0
  • scikit-image=0.14.1=py36he6710b0_0
  • scikit-learn=0.20.1=py36hd81dba3_0
  • scipy=1.1.0=py36h7c811a0_2
  • pip:
    • keras==2.2.4

When using json model here is the error
ValueError: Improper config format:

When using previously created model (by adding layers manually) to lead h5 file weights here is the error
ValueError: You are trying to load a weight file containing 0 layers into a model with 11 layers.

Looks like the weights files cant be decoded properly. Any Idea on what to do?

@aslucki
Copy link

aslucki commented Feb 5, 2019

@supun-kanda Based on this project: https://github.com/axon-research/c3d-keras
I created an updated version of the model (for keras 2.2.4): https://github.com/aslucki/C3D_Sport1M_keras
and weights.

@iriyagupta
Copy link

iriyagupta commented Sep 10, 2019

@albertomontesg
Hi, I am facing this error. Did anyone else face this issue?


AttributeError Traceback (most recent call last)
in
22 weights_b = np.array(layer.blobs[1].data, dtype=np.float32)
23 weights_p = np.array(layer.blobs[0].data, dtype=np.float32).reshape(
---> 24 layer.blobs[0].num, layer.blobs[0].channels, layer.blobs[0].length,
25 layer.blobs[0].height, layer.blobs[0].width
26 )

AttributeError: length

I am new to caffe and getting attributes from blob using hpp files is sort of not understandable to me here.
Thanks in advance.

@parasmaharjan
Copy link

@albertomontesg
Is there any training code for the sports-1m dataset?

@masouduut94
Copy link

how can we use the C3D features to model violence detection on videos?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment