Skip to content

Instantly share code, notes, and snippets.

@neomatrix369
Created October 21, 2019 16:06
Show Gist options
  • Save neomatrix369/f444933bffbf4ec5481f9d61e85cd04f to your computer and use it in GitHub Desktop.
Save neomatrix369/f444933bffbf4ec5481f9d61e85cd04f to your computer and use it in GitHub Desktop.
Logs of the CPU version of dl4j-nlp failing due to OOME (dl4j-nlp-cuda)
19:34:12 openjdk version "1.8.0_232"
19:34:12 OpenJDK Runtime Environment (build 1.8.0_232-20191008104205.buildslave.jdk8u-src-tar--b07)
19:34:12 OpenJDK 64-Bit GraalVM CE 19.2.1 (build 25.232-b07-jvmci-19.2-b03, mixed mode)
19:34:12 Starting data download (80MB)...
19:34:23 Data (.tar.gz file) downloaded to /tmp/dl4j_w2vSentiment/aclImdb_v1.tar.gz
19:34:28 2019-10-20 18:34:28,210ate - 18:34:28,210 INFO ~ Loaded [CpuBackend] backend
19:34:29 2019-10-20 18:34:29,610ate - 18:34:29,610 INFO ~ Number of threads used for NativeOps: 4
19:34:30 2019-10-20 18:34:30,299ate - 18:34:30,299 INFO ~ Number of threads used for BLAS: 4
19:34:30 2019-10-20 18:34:30,305ate - 18:34:30,305 INFO ~ Backend used: [CPU]; OS: [Linux]
19:34:30 2019-10-20 18:34:30,305ate - 18:34:30,305 INFO ~ Cores: [8]; Memory: [3.2GB];
19:34:30 2019-10-20 18:34:30,305ate - 18:34:30,305 INFO ~ Blas vendor: [MKL]
19:34:30 2019-10-20 18:34:30,363ate - 18:34:30,363 INFO ~ Starting ComputationGraph with WorkspaceModes set to [training: ENABLED; inference: ENABLED], cacheMode set to [NONE]
19:34:30 Number of parameters by layer:
19:34:30 cnn3 90100
19:34:30 cnn4 120100
19:34:30 cnn5 150100
19:34:30 globalPool 0
19:34:30 out 602
19:34:30 Loading word vectors and creating DataSetIterators
19:36:35 Exception in thread "main" java.lang.OutOfMemoryError: Cannot allocate new BytePointer(1200): totalBytes = 2921M, physicalBytes = 6717M
19:36:35 at org.bytedeco.javacpp.BytePointer.<init>(BytePointer.java:106)
19:36:35 at org.nd4j.compression.impl.NoOp.compressPointer(NoOp.java:94)
19:36:35 at org.nd4j.compression.impl.AbstractCompressor.compress(AbstractCompressor.java:162)
19:36:35 at org.nd4j.compression.impl.AbstractCompressor.compress(AbstractCompressor.java:134)
19:36:35 at org.nd4j.storage.CompressedRamStorage.store(CompressedRamStorage.java:84)
19:36:35 at org.deeplearning4j.models.embeddings.loader.WordVectorSerializer.loadStaticModel(WordVectorSerializer.java:2785)
19:36:35 at org.deeplearning4j.examples.convolution.sentenceclassification.CnnSentenceClassificationExample.main(CnnSentenceClassificationExample.java:141)
19:36:35 Caused by: java.lang.OutOfMemoryError: Physical memory usage is too high: physicalBytes (6717M) > maxPhysicalBytes (6571M)
19:36:35 at org.bytedeco.javacpp.Pointer.deallocator(Pointer.java:585)
19:36:35 at org.bytedeco.javacpp.Pointer.init(Pointer.java:125)
19:36:35 at org.bytedeco.javacpp.BytePointer.allocateArray(Native Method)
19:36:35 at org.bytedeco.javacpp.BytePointer.<init>(BytePointer.java:98)
19:36:35 ... 6 more
19:36:35
19:36:35 real 2m22.848s
19:36:35 user 6m45.908s
19:36:35 sys 0m23.172s
19:36:35
19:36:35 real 2m22.913s
19:36:35 user 6m45.948s
19:36:35 sys 0m23.192s
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment