Skip to content

Instantly share code, notes, and snippets.

@liweigu
Created July 11, 2018 03:25
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save liweigu/9e1b9e2917b5d06d1faf9493701c8d54 to your computer and use it in GitHub Desktop.
Save liweigu/9e1b9e2917b5d06d1faf9493701c8d54 to your computer and use it in GitHub Desktop.
ND4JWorkspaceException-dl4j
Error Log:
java.lang.IllegalStateException: Backprop: array (ACTIVATION_GRAD) workspace validation failed (vertex lstm1 - class: GravesLSTM) - array is defined in incorrect workspace
at org.deeplearning4j.nn.graph.ComputationGraph.validateArrayWorkspaces(ComputationGraph.java:1708)
at org.deeplearning4j.nn.graph.ComputationGraph.calcBackpropGradients(ComputationGraph.java:2435)
at org.deeplearning4j.nn.graph.ComputationGraph.computeGradientAndScore(ComputationGraph.java:1319)
at org.deeplearning4j.nn.graph.ComputationGraph.computeGradientAndScore(ComputationGraph.java:1280)
at org.deeplearning4j.optimize.solvers.BaseOptimizer.gradientAndScore(BaseOptimizer.java:178)
at org.deeplearning4j.optimize.solvers.StochasticGradientDescent.optimize(StochasticGradientDescent.java:60)
at org.deeplearning4j.optimize.Solver.optimize(Solver.java:54)
at org.deeplearning4j.nn.graph.ComputationGraph.fit(ComputationGraph.java:1104)
at org.deeplearning4j.nn.graph.ComputationGraph.fit(ComputationGraph.java:1050)
at org.deeplearning4j.nn.graph.ComputationGraph.fit(ComputationGraph.java:944)
at com.liweigu.dls.competition.srad.RadLSTM.run(RadLSTM.java:102)
at com.liweigu.dls.competition.srad.RadLSTMTest.testRun(RadLSTMTest.java:10)
Caused by: org.nd4j.linalg.workspace.ND4JWorkspaceException: Array workspace validation failed: Array of type ACTIVATION_GRAD should be in workspace "WS_LAYER_ACT_1" but is in workspace "WS_LAYER_WORKING_MEM"
at org.nd4j.linalg.workspace.BaseWorkspaceMgr.validateArrayLocation(BaseWorkspaceMgr.java:221)
at org.deeplearning4j.nn.workspace.LayerWorkspaceMgr.validateArrayLocation(LayerWorkspaceMgr.java:66)
at org.deeplearning4j.nn.graph.ComputationGraph.validateArrayWorkspaces(ComputationGraph.java:1699)
... 33 more
Key Code:
int cnn1Stride = 4;
int lstmHiddenCount = 100;
LOGGER.info("lstmHiddenCount = " + lstmHiddenCount);
int channels = 1;
int kernelSize = 3; // 5
Map<String, InputPreProcessor> inputPreProcessors = new HashMap<String, InputPreProcessor>();
inputPreProcessors.put("cnn1", new RnnToCnnPreProcessor(501, 501, channels));
inputPreProcessors.put("lstm1", new CnnToRnnPreProcessor(120, 120, channels)); // 120 = Math.pow(14400, 0.5)
GraphBuilder graphBuilder = builder.graphBuilder().pretrain(false).backprop(true).backpropType(BackpropType.Standard)
.addInputs("inputs")
.addLayer("cnn1", new ConvolutionLayer.Builder(new int[] { kernelSize, kernelSize }, new int[] { cnn1Stride, cnn1Stride }, new int[] { 0, 0 })
.nIn(channels).nOut(50).activation(Activation.RELU).weightInit(WeightInit.RELU).build(), "inputs")
// Output: (501 - kernelSize + 0) / cnn1Stride + 1 = 125 --> 125 * 125 * 50 = 781250
.addLayer("cnn2", new SubsamplingLayer.Builder(new int[] { 2, 2 }, new int[] { 2, 2 }).build(), "cnn1")
// (125-2+0)/2+1 = 62
.addLayer("cnn3",
new ConvolutionLayer.Builder(new int[] { kernelSize, kernelSize }, new int[] { 5, 5 }, new int[] { 0, 0 })
.nIn(50).nOut(100).activation(Activation.RELU).weightInit(WeightInit.RELU).build(), "cnn2")
// Output: (62 - kernelSize + 0) / 5 + 1 = 12 --> 12 * 12 * 100 = 14400
.addLayer("lstm1", new GravesLSTM.Builder().activation(Activation.SOFTSIGN)
.nIn(14400).nOut(lstmHiddenCount).build(), "cnn3")
.addLayer("lstm2", new GravesLSTM.Builder().activation(Activation.SOFTSIGN)
.nIn(lstmHiddenCount).nOut(lstmHiddenCount).build(), "lstm1")
.addVertex("thoughtVector", new LastTimeStepVertex("inputs"), "lstm2")
.addLayer("output", new OutputLayer
.Builder(LossFunctions.LossFunction.MSE) // MSE, MEAN_ABSOLUTE_ERROR
.activation(Activation.IDENTITY)
.nIn(lstmHiddenCount)
.nOut(501 * 501).build(), "thoughtVector")
.setOutputs("output");
graphBuilder.setInputPreProcessors(inputPreProcessors);
ComputationGraph multiLayerNetwork = new ComputationGraph(graphBuilder.build());
multiLayerNetwork.init();
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment