gistfile1.txt

13:10:07,659 INFO  ~ Allocating [LOOP_ITER] workspace of 0 bytes...
13:10:07,821 INFO  ~ [LOOP_ITER] spilled DEVICE array of 1152 bytes, capacity of 288 elements
13:10:07,831 INFO  ~ [LOOP_ITER] spilled DEVICE array of 1152 bytes, capacity of 288 elements                   
13:10:07,840 INFO  ~ [LOOP_ITER] spilled DEVICE array of 1152 bytes, capacity of 288 elements      
13:10:07,844 WARN  ~ Workspace initialization OVER_TIME was selected, but number of cycles isn't positive value!
13:10:07,844 INFO  ~ Allocating [LOOP_FF] workspace of 0 bytes...                   
13:10:07,844 INFO  ~ Allocating [LOOP_FF] workspace of 0 bytes...                               
13:10:07,844 INFO  ~ [LOOP_FF] spilled DEVICE array of 1152 bytes, capacity of 288 elements      
CUDA error at /home/rkfg/soft/svn-soft/libnd4j/blas/cuda/NativeOps.cu:4801 code=11(cudaErrorInvalidValue) "result" 
Failed on [1110846756352] -> [1110844720640], size: [1152], direction: [2], result: [11]
13:10:07,846 INFO  ~ Allocating [LOOP_ITER] workspace of 6912 bytes...                                                           
[WARNING]                                                            
java.lang.reflect.InvocationTargetException                                                                     
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)                                                     
        at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:294)                        
        at java.lang.Thread.run(Thread.java:745)                                                   
Caused by: java.lang.IllegalStateException: MemcpyAsync failed: AllocationShape(offset=0, length=288, stride=1, elementSize=4, dataType=FLOAT)
        at org.nd4j.jita.flow.impl.SynchronousFlowController.synchronizeToHost(SynchronousFlowController.java:63)
        at org.nd4j.jita.flow.impl.GridFlowController.synchronizeToHost(GridFlowController.java:36)
        at org.nd4j.jita.handler.impl.CudaZeroHandler.synchronizeThreadDevice(CudaZeroHandler.java:1227)           
        at org.nd4j.jita.allocator.impl.AtomicAllocator.synchronizeHostData(AtomicAllocator.java:321)
        at org.nd4j.linalg.jcublas.buffer.BaseCudaDataBuffer.getFloat(BaseCudaDataBuffer.java:965)                               
        at org.nd4j.linalg.jcublas.buffer.CudaFloatDataBuffer.getDouble(CudaFloatDataBuffer.java:237)
        at org.nd4j.linalg.api.shape.Shape.getDouble(Shape.java:163)                                            
        at org.nd4j.linalg.api.ndarray.BaseNDArray.getDouble(BaseNDArray.java:1697)
        at org.nd4j.linalg.api.ndarray.BaseNDArray.getInt(BaseNDArray.java:1666)
        at org.deeplearning4j.nn.layers.feedforward.embedding.EmbeddingLayer.preOutput(EmbeddingLayer.java:93)
        at org.deeplearning4j.nn.layers.feedforward.embedding.EmbeddingLayer.activate(EmbeddingLayer.java:113)  
        at org.deeplearning4j.nn.graph.vertex.impl.LayerVertex.doForward(LayerVertex.java:103)     
        at org.deeplearning4j.nn.graph.ComputationGraph.feedForward(ComputationGraph.java:1182)    
        at org.deeplearning4j.nn.graph.ComputationGraph.computeGradientAndScore(ComputationGraph.java:1054)
        at org.deeplearning4j.optimize.solvers.BaseOptimizer.gradientAndScore(BaseOptimizer.java:160)
        at org.deeplearning4j.optimize.solvers.StochasticGradientDescent.optimize(StochasticGradientDescent.java:56)
        at org.deeplearning4j.optimize.Solver.optimize(Solver.java:59)                                             
        at org.deeplearning4j.nn.graph.ComputationGraph.fit(ComputationGraph.java:854)
        at dlchat.EncoderDecoderLSTM.train(EncoderDecoderLSTM.java:248)                                                          
        at dlchat.EncoderDecoderLSTM.run(EncoderDecoderLSTM.java:201)
        at dlchat.EncoderDecoderLSTM.main(EncoderDecoderLSTM.java:149)                                          
        ... 6 more