Skip to content

Instantly share code, notes, and snippets.

@rkfg
Last active February 5, 2017 13:32
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save rkfg/73b8bffbd8a91f50624a6a83bb7456f1 to your computer and use it in GitHub Desktop.
Save rkfg/73b8bffbd8a91f50624a6a83bb7456f1 to your computer and use it in GitHub Desktop.
LSTM error
NeuralNetConfiguration.Builder builder = new NeuralNetConfiguration.Builder();
builder.iterations(1).learningRate(LEARNING_RATE).rmsDecay(RMS_DECAY)
.optimizationAlgo(OptimizationAlgorithm.STOCHASTIC_GRADIENT_DESCENT).seed(123).miniBatch(true).updater(Updater.RMSPROP)
.weightInit(WeightInit.XAVIER).regularization(true).l2(L2)
.gradientNormalization(GradientNormalization.RenormalizeL2PerLayer);
GraphBuilder graphBuilder = builder.graphBuilder().pretrain(false).backprop(true).backpropType(BackpropType.TruncatedBPTT)
.tBPTTBackwardLength(TBPTT_SIZE).tBPTTForwardLength(TBPTT_SIZE);
graphBuilder.addInputs("firstLine").setInputTypes(InputType.recurrent(dict.size()))
.addLayer("encoder", new GravesLSTM.Builder().nIn(dict.size()).nOut(HIDDEN_LAYER_WIDTH).activation(Activation.TANH).build(),
"firstLine")
.addVertex("thoughtVector", new LastTimeStepVertex("firstLine"), "encoder")
.addLayer("decoder",
new GravesLSTM.Builder().nIn(HIDDEN_LAYER_WIDTH).nOut(HIDDEN_LAYER_WIDTH).activation(Activation.TANH)
.build(),
"thoughtVector")
.addLayer("output", new RnnOutputLayer.Builder().nIn(HIDDEN_LAYER_WIDTH).nOut(dict.size()).activation(Activation.SOFTMAX)
.lossFunction(LossFunctions.LossFunction.MCXENT).build(), "decoder")
.setOutputs("output");
ComputationGraphConfiguration conf = graphBuilder.build();
Exception in thread "main" org.deeplearning4j.exception.DL4JInvalidInputException: Received input with size(1) = 1024 (input array shape = [17, 1024]); input.size(1) must match layer nIn size (nIn = 11059)
at org.deeplearning4j.nn.layers.recurrent.LSTMHelpers.activateHelper(LSTMHelpers.java:142)
at org.deeplearning4j.nn.layers.recurrent.GravesLSTM.activateHelper(GravesLSTM.java:150)
at org.deeplearning4j.nn.layers.recurrent.GravesLSTM.rnnActivateUsingStoredState(GravesLSTM.java:222)
at org.deeplearning4j.nn.graph.ComputationGraph.rnnActivateUsingStoredState(ComputationGraph.java:2081)
at org.deeplearning4j.nn.graph.ComputationGraph.computeGradientAndScore(ComputationGraph.java:955)
at org.deeplearning4j.optimize.solvers.BaseOptimizer.gradientAndScore(BaseOptimizer.java:151)
at org.deeplearning4j.optimize.solvers.StochasticGradientDescent.optimize(StochasticGradientDescent.java:54)
at org.deeplearning4j.optimize.Solver.optimize(Solver.java:51)
at org.deeplearning4j.nn.graph.ComputationGraph.doTruncatedBPTT(ComputationGraph.java:2032)
at org.deeplearning4j.nn.graph.ComputationGraph.fit(ComputationGraph.java:824)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment