Skip to content

Instantly share code, notes, and snippets.

View Habitats's full-sized avatar
🚀
What's happening?

Patrick Skjennum Habitats

🚀
What's happening?
30 followers · 34 following
View GitHub Profile
@Habitats
Habitats / gist:1fba454eb9028a6f767c
Created March 2, 2016 10:39
11:34:44.863 ERROR - Lost executor driver on localhost: Executor heartbeat timed out after 200691 ms
11:34:44.869 WARN - Killing executors is only supported in coarse-grained mode
11:34:44.934 WARN - Stage 4 contains a task of very large size (12814 KB). The maximum recommended task size is 100 KB.
11:34:45.422 INFO - Running distributed training: (averaging each iteration = true), (iterations =1), (num partions = 8)
11:34:45.422 INFO - Broadcasting initial parameters of length 965018
11:34:54.785 INFO - Ran iterative reduce... averaging parameters now.
11:35:06.025 WARN - Stage 5 contains a task of very large size (12814 KB). The maximum recommended task size is 100 KB.
[Stage 5:> (0 + 8) / 8]11:35:06.648 DEBUG - Training on 50 examples with data {0=3150.0, 1=12.0, 2=2.0, 4=1.0, 5=29.0, 10=1.0, 11=3.0, 12=1.0, 14=1.0}
11:35:06.683 DEBUG - Training on 50 examples with data {0=2700.0, 16=6.0, 17=1.0, 1=13.0, 2=2.0, 3=3.0, 4=2.0, 5=15.0, 1
@Habitats
Habitats / gist:a703837951c6823340cd
Created March 2, 2016 10:40
11:30:15.453 WARN - Stage 1 contains a task of very large size (430 KB). The maximum recommended task size is 100 KB.
11:30:50.981 WARN - Stage 3 contains a task of very large size (430 KB). The maximum recommended task size is 100 KB.
11:34:44.207 CORPUS - Starting training ...
11:34:44.332 WARN - Removing executor driver with no recent heartbeats: 200691 ms exceeds timeout 120000 ms
11:34:44.863 ERROR - Lost executor driver on localhost: Executor heartbeat timed out after 200691 ms
11:34:44.869 WARN - Killing executors is only supported in coarse-grained mode
11:34:44.934 WARN - Stage 4 contains a task of very large size (12814 KB). The maximum recommended task size is 100 KB.
11:34:45.422 INFO - Running distributed training: (averaging each iteration = true), (iterations =1), (num partions = 8)
11:34:45.422 INFO - Broadcasting initial parameters of length 965018
11:34:54.785 INFO - Ran iterative reduce... averaging parameters now.
@Habitats
Habitats / gist:a6de2cee8a82ceaf10ec
Created March 2, 2016 14:20
import java.util
import org.deeplearning4j.datasets.iterator.DataSetIterator
import org.nd4j.linalg.dataset.DataSet
import org.nd4j.linalg.dataset.api.DataSetPreProcessor
import org.nd4j.linalg.factory.Nd4j
import scala.collection.JavaConverters._
import scala.util.Random
@Habitats
Habitats / Main.scala
Last active March 2, 2016 18:13
import java.util
import org.deeplearning4j.datasets.iterator.DataSetIterator
import org.nd4j.linalg.dataset.DataSet
import org.nd4j.linalg.dataset.api.DataSetPreProcessor
import org.nd4j.linalg.factory.Nd4j
import scala.collection.JavaConverters._
import scala.util.Random
@Habitats
Habitats / gist:f0ac6068b4a23d647c0a
Last active March 4, 2016 12:14
For a binary classification with two outputs, eihter [1,0] or [0,1],
and the Evalution object, the following outputs are generated ...
The following confusion matrix:
Predicted Class
0 1 Total
Actual Class 0 1500 2 1502
1 68 131 199
Total 1568 133
@Habitats
Habitats / log4j.properties
Created March 4, 2016 13:35
log4j.appender.myConsoleAppender=org.apache.log4j.ConsoleAppender
log4j.appender.myConsoleAppender.layout=org.apache.log4j.PatternLayout
log4j.appender.myConsoleAppender.layout.ConversionPattern=%d [%t] %-5p %c - %m%n
log4j.appender.RollingAppender=org.apache.log4j.DailyRollingFileAppender
log4j.appender.RollingAppender.File=/var/log/spark.log
log4j.appender.RollingAppender.DatePattern='.'yyyy-MM-dd
log4j.appender.RollingAppender.layout=org.apache.log4j.PatternLayout
log4j.appender.RollingAppender.layout.ConversionPattern=[%p] %d %c %M - %m%n
@Habitats
Habitats / exception.txt
Created March 4, 2016 14:04
INFO [2016-03-04 14:03:57,816] org.eclipse.jetty.util.log: Logging initialized @4270ms
INFO [2016-03-04 14:03:57,903] io.dropwizard.assets.AssetsBundle: Registering AssetBundle with name: assets for path /assets/*
port: 0
port: 0
WARN [2016-03-04 14:03:57,977] org.glassfish.jersey.internal.Errors: The following warnings have been detected: WARNING: Cannot create new registration for component type class org.deeplearning4j.ui.exception.GenericExceptionMapper: Existing previous registration found for the type.
INFO [2016-03-04 14:03:57,986] io.dropwizard.server.ServerFactory: Starting UiServer
WARN [2016-03-04 14:03:58,017] org.glassfish.jersey.internal.Errors: The following warnings have been detected: WARNING: Cannot create new registration for component type class io.dropwizard.jersey.jackson.JsonProcessingExceptionMapper: Existing previous registration found for the type.
INFO [2016-03-04 14:03:58,071] org.eclipse.jetty.setuid.SetUIDListener: Opened application@2f79f192{HTTP/1.1}{0.0.0.0
@Habitats
Habitats / gist:3703e8adac820137f7d7
Created March 9, 2016 10:14
build.gradle:
plugins {
id 'idea'
}
subprojects{
apply plugin: 'scala'
repositories {
@Habitats
Habitats / Annotate these
Created December 1, 2017 11:14
# Norweigan
## Shopping
- [aHR0cDovL3d3dy5hdmlzYS1zdC5uby9ueWhldGVyLzIwMTcvMTEvMjIvTiVDMyVBNS1lci1kZXQtZiVDMyVBNS1kYWdlci10aWwtJUMzJUE1cmV0cy12aWxsZXN0ZS1oYW5kbGVkYWctMTU2Mzg5OTQuZWNl](http://omni.mito.ai/news/aHR0cDovL3d3dy5hdmlzYS1zdC5uby9ueWhldGVyLzIwMTcvMTEvMjIvTiVDMyVBNS1lci1kZXQtZiVDMyVBNS1kYWdlci10aWwtJUMzJUE1cmV0cy12aWxsZXN0ZS1oYW5kbGVkYWctMTU2Mzg5OTQuZWNl/annotate)
- [aHR0cHM6Ly93d3cuaWZpbm5tYXJrLm5vL2FsdGEvb3Nsby90cm9tc28vZGV0dGUtYmlsZGV0LWVyLXRhdHQtcGEtbm9yd2VnaWFucy1mbHktbWVsbG9tLWFsdGEtb2ctb3Nsby10aWwtdHJvbXNvLXZhci1kZXQtc21la2tmdWxsdC9vLzUtODEtNjM1MTg2](http://omni.mito.ai/news/aHR0cHM6Ly93d3cuaWZpbm5tYXJrLm5vL2FsdGEvb3Nsby90cm9tc28vZGV0dGUtYmlsZGV0LWVyLXRhdHQtcGEtbm9yd2VnaWFucy1mbHktbWVsbG9tLWFsdGEtb2ctb3Nsby10aWwtdHJvbXNvLXZhci1kZXQtc21la2tmdWxsdC9vLzUtODEtNjM1MTg2/annotate)
## News
- [aHR0cDovL3d3dy5hZHJlc3NhLm5vL255aGV0ZXIvdHJvbmRlbGFnLzIwMTcvMTEvMjQvQnJhbm4taS1iaWwtaS1Pcmthbmdlci1pLW5hdHQtMTU2NTAwNzUuZWNl](http://omni.mito.ai/news/aHR0cDovL3d3dy5hZHJlc3NhLm5vL255aGV0ZXIvdHJvbmRlbGFnLzIwMTc
@Habitats
Habitats / annotations.md
Created December 1, 2017 11:15
Annotate these