Skip to content

Instantly share code, notes, and snippets.

View lacic's full-sized avatar

Emanuel Lacic lacic

8 followers · 0 following
View GitHub Profile
@lacic
lacic / ParagraphVectorsSaveBug.java
Created November 21, 2018 15:00
package service.deeplearning;
import org.deeplearning4j.models.embeddings.loader.WordVectorSerializer;
import org.deeplearning4j.models.paragraphvectors.ParagraphVectors;
import org.deeplearning4j.models.word2vec.VocabWord;
import org.deeplearning4j.models.word2vec.wordstore.inmemory.AbstractCache;
import org.deeplearning4j.text.documentiterator.BasicLabelAwareIterator;
import org.deeplearning4j.text.sentenceiterator.CollectionSentenceIterator;
import org.deeplearning4j.text.tokenization.tokenizer.preprocessor.CommonPreprocessor;
import org.deeplearning4j.text.tokenization.tokenizerfactory.DefaultTokenizerFactory;
@lacic
lacic / comp_graph_error.log
Created June 21, 2018 05:24
o.n.l.f.Nd4jBackend - Loaded [CpuBackend] backend
o.n.n.NativeOpsHolder - Number of threads used for NativeOps: 4
o.n.n.Nd4jBlas - Number of threads used for BLAS: 4
o.n.l.a.o.e.DefaultOpExecutioner - Backend used: [CPU]; OS: [Linux]
o.n.l.a.o.e.DefaultOpExecutioner - Cores: [8]; Memory: [3.4GB];
o.n.l.a.o.e.DefaultOpExecutioner - Blas vendor: [OPENBLAS]
o.d.n.g.ComputationGraph - Starting ComputationGraph with WorkspaceModes set to [training: ENABLED; inference: ENABLED], cacheMode set to [NONE]
o.d.e.t.BaseEarlyStoppingTrainer - Starting early stopping training
o.d.e.t.BaseEarlyStoppingTrainer - Early stopping training terminated due to exception at epoch 0, iteration 0
java.lang.IllegalStateException: Feed forward (inference): array (ACTIVATIONS) workspace validation failed (vertex encoder - class: AutoEncoder) - array is defined in incorrect workspace
@lacic
lacic / nd4j_fail.log
Created June 14, 2018 23:36
#
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0x00007f549ae5b182, pid=16918, tid=140007211910912
#
# JRE version: Java(TM) SE Runtime Environment (8.0_91-b14) (build 1.8.0_91-b14)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.91-b14 mixed mode linux-amd64 )
# Problematic frame:
# C [libopenblas.so.0+0x1143182] sgemm_incopy_SANDYBRIDGE+0x8d2
#
@lacic
lacic / syn0.txt
Created October 6, 2017 13:08
22471 150 1668
B64:dW5k 0.0033117409329861403 0.0019191396422684193 0.00221459474414587 -0.0021077159326523542 8.493439527228475E-4 -0.0017041091341525316 -0.0025153029710054398 0.00208079582080245 4.312245000619441E-4 -0.0029814057052135468 0.002164070261642337 0.0012126270448789 5.760510684922338E-4 4.937883350066841E-4 -0.0014595423126593232 -0.00239103683270514 -0.001964881783351302 0.0023842724040150642 -0.001518659177236259 -2.8960962663404644E-4 0.0024277083575725555 -1.4996746904216707E-4 -0.0013514095917344093 -8.353352313861251E-5 -0.002643878571689129 0.0023342736531049013 0.0031588184647262096 -0.0029023464303463697 7.042980287224054E-4 0.0023141801357269287 -9.172326535917819E-4 8.497278031427413E-5 0.0032884525135159492 -0.002120635472238064 -0.0032929035369306803 0.002765442943200469 0.002038929844275117 9.461244189878926E-6 -0.0029955727513879538 -0.0027375707868486643 -3.519121673889458E-4 -3.000358701683581E-4 -0.0027778323274105787 0.001297975773923099 1.223723120347131E-5 -4.26717189839109
@lacic
lacic / train.java
Created October 6, 2017 12:59
DocAwareIterator jaIter = new DocAwareIterator(myDocData);
AbstractCache<VocabWord> cache = new AbstractCache<>();
TokenizerFactory t = new DefaultTokenizerFactory();
t.setTokenPreProcessor(new CommonPreprocessor());
ParagraphVectors.Builder vecBuilder = new ParagraphVectors.Builder()
.minWordFrequency(1)
.iterations(5)
@lacic
lacic / DocAwareIterator.java
Created October 6, 2017 12:25
public class DocAwareIterator implements LabelAwareIterator{
private AtomicInteger currPosition;
private List<String> labels;
private Map<String, String> docData;
private LabelsSource generator;
public DocAwareIterator(Map<String, String> docData) {
@lacic
lacic / mvn.txt
Created May 6, 2017 18:46
cuda-7.5-error-0.8.1-SNAPSHOT
Apache Maven 3.3.9
Maven home: /usr/share/maven
Java version: 1.8.0_131, vendor: Oracle Corporation
Java home: /usr/lib/jvm/java-8-oracle/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "4.10.9-041009-generic", arch: "amd64", family: "unix"
[DEBUG] Created new class realm maven.api
[DEBUG] Importing foreign packages into class realm maven.api
[DEBUG] Imported: javax.enterprise.inject.* < plexus.core
[DEBUG] Imported: javax.enterprise.util.* < plexus.core
@lacic
lacic / gpu-classification.java
Created April 8, 2017 12:41
public class ModelTrainer {
private String modelPath;
private String modelName;
private final static Integer WORKERS = 4;
private final static Integer PREFETCH_BUFFER = 24;
private final static Integer N_EPOCHS = 250;
private final static Integer BATCH_SIZE = 300;
private final static Integer CLASS_COUNT = 20;
@lacic
lacic / thread-state-exception.java
Created January 5, 2017 15:56
16:40:07.783 [com.issue2548.EvaluationExampleCorrect.main()] INFO org.reflections.Reflections - Reflections took 80 ms to scan 12 urls, producing 368 keys and 1424 values
16:40:08.167 [com.issue2548.EvaluationExampleCorrect.main()] INFO org.nd4j.nativeblas.Nd4jBlas - Number of threads used for BLAS: 4
Successfully tested the NN model
[WARNING] thread Thread[NativeRandomDeallocator thread 0,5,com.issue2548.EvaluationExampleCorrect] was interrupted but is still alive after waiting at least 15000msecs
[WARNING] thread Thread[NativeRandomDeallocator thread 0,5,com.issue2548.EvaluationExampleCorrect] will linger despite being asked to die via interruption
[WARNING] NOTE: 1 thread(s) did not finish despite being asked to via interruption. This is not a problem with exec:java, it is a problem with the running code. Although not serious, it should be remedied.
[WARNING] Couldn't destroy threadgroup org.codehaus.mojo.exec.ExecJavaMojo$IsolatedThreadGroup[name=com.issue2548.EvaluationExampleCorrect,maxpri=10]
java.
@lacic
lacic / error-dl4j-0-7-2.java
Created January 5, 2017 15:03
16:01:43.946 [main] INFO org.nd4j.linalg.factory.Nd4jBackend - Loaded [CpuBackend] backend
Exception in thread "main" java.lang.ExceptionInInitializerError
at org.nd4j.linalg.cpu.nativecpu.ops.NativeOpExecutioner.<init>(NativeOpExecutioner.java:41)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at java.lang.Class.newInstance(Class.java:442)
at org.nd4j.linalg.factory.Nd4j.initWithBackend(Nd4j.java:5646)
at org.nd4j.linalg.factory.Nd4j.initContext(Nd4j.java:5554)