Skip to content

Instantly share code, notes, and snippets.

@Andy-P
Andy-P / StreamAnalytics.jl
Last active July 25, 2024 20:59
High Performance Streaming Analytics in Julia
# This code is part of a presentation on streaming analytics in Julia
# It was inspired by a number of individuals and makes use of some of their ideas
# 1. FastML.com got me thinking about inline processing after
# reading his great Vowpal Wabbit posts
# 2. John Lanford and his fantastic Vowpal Wabbit library.
# Check out his NYU video course to learn more (see below)
# 3. John Myles White's presentation on online SDG and his StreamStats.jl library
# Thank you all!
using Dates
# set_default_plot_size(20cm, 10cm)
# simple Integrate and Fire Synapse Mode
type Synapse
Ω::Real # membrane resistance [MΩ]
τ::Real # membrane time constant [ms]
rV::Real # resting membrane potential [mV]
thV::Real # spike threshold [mV]
sV::Real # spike voltage [mV]
function generate_batch(batch_size, num_skips, skip_window, data_index)
@assert batch_size % num_skips == 0
@assert num_skips <= 2 * skip_window
batch = zeros(Int32, (batch_size))
labels = zeros(Int32, (batch_size,1))
span = 2 * skip_window + 1
buffer = Int[]
for _ in 1:span
push!(buffer,data[data_index])
@Andy-P
Andy-P / pg-pong.py
Created August 7, 2016 05:12 — forked from karpathy/pg-pong.py
Training a Neural Network ATARI Pong agent with Policy Gradients from raw pixels
""" Trains an agent with (stochastic) Policy Gradients on Pong. Uses OpenAI Gym. """
import numpy as np
import cPickle as pickle
import gym
# hyperparameters
H = 200 # number of hidden layer neurons
batch_size = 10 # every how many episodes to do a param update?
learning_rate = 1e-4
gamma = 0.99 # discount factor for reward
@Andy-P
Andy-P / gist:0c676a5017cdd2d4525cb83ff7294b33
Created October 3, 2016 06:28
errors in POM.xml after adding nd4j dependency
Multiple annotations found at this line:
- Missing artifact concurrent:concurrent:jar:1.3.4
- Missing artifact jna:jna:jar:1.0
- Missing artifact commons-net:commons-net:jar:2.2
- Missing artifact com.github.lucarosellini.rJava:JRI:jar:0.9
- Missing artifact org.apache.commons:commons-lang3:jar:3.2
- Missing artifact poi-ooxml:poi-ooxml:jar:3.7
- Missing artifact log4j:log4j:jar:1.2.14
- Missing artifact simplex.galaxy.l3j:l3j:jar:1.1.80
- Missing artifact mail:mail:jar:1.0.0
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
CudaGridExecutioner
Build model....
Beginning training
Epoch = 0
Batch 000 [150, 3] [150, 3, 32, 16]
CUDA error at /projects/skymind/libnd4j/blas/cuda/NativeOps.cu:4969 code=77(<unknown>) "result"
@Andy-P
Andy-P / g2.2xlarge_GPU_output.txt
Last active November 28, 2016 07:49
g2.2xlarge GPU output
15:10:38.984 [main] INFO org.nd4j.nativeblas.NativeOps - Number of threads used for NativeOps: 32
15:10:48.878 [main] INFO org.nd4j.nativeblas.Nd4jBlas - Number of threads used for BLAS: 4
15:11:58.989 [main] DEBUG org.reflections.Reflections - going to scan these urls:
jar:file:/simplex/simulator/arcadia-dealing-simulator.jar!/
15:11:59.760 [main] INFO org.reflections.Reflections - Reflections took 766 ms to scan 1 urls, producing 29 keys and 172 values
15:12:05.823 [main] DEBUG o.n.j.a.c.impl.BasicContextPool - Creating new stream for thread: [1], device: [0]...
15:12:05.826 [main] DEBUG o.n.j.a.c.impl.BasicContextPool - Creating new stream for thread: [1], device: [0]...
15:12:05.826 [main] DEBUG o.n.j.a.c.impl.BasicContextPool - Creating new stream for thread: [1], device: [0]...
15:12:05.827 [main] DEBUG o.n.j.a.c.impl.BasicContextPool - Creating new stream for thread: [1], device: [0]...
15:12:05.827 [main] DEBUG o.n.j.a.c.impl.BasicContextPool - Creating new stream for thread: [1], device: [0]...