codingneo

Simulation Environment For RL

Motivation

To apply (deep) reinforcement learning to other real problems (e.g. energy management, traffic optimization etc.), you first need a simulation environment to learn/train autonomous agent.

Code

• The Arcade Learning Environment
• PyGame-Learning-Environment
• OpenAI Gym

codingneo

Enterprise Model Factory

codingneo

""" Trains an agent with (stochastic) Policy Gradients on Pong. Uses OpenAI Gym. """
import numpy as np
import cPickle as pickle
import gym

# hyperparameters
H = 200 # number of hidden layer neurons
batch_size = 10 # every how many episodes to do a param update?
learning_rate = 1e-4
gamma = 0.99 # discount factor for reward

codingneo

Smart Energy Using Machine Learning

Applications

Solar (PV)

• Solar Forecast
• Prediction of customers who will buy solar (reduce sale cost)
• Insurance/Finance for buying solar

Non-intrusive Load Monitoring (NILM)

codingneo

aaa

codingneo

ps server

import tensorflow as tf

cluster = tf.train.ClusterSpec({"ps": ['localhost:2222'], "worker": ['localhost:2224','localhost:2225']})
server = tf.train.Server(cluster.as_cluster_def(), job_name='ps', task_index=0)

server.join()

codingneo

---

Preliminary Updates and Installations

(http://markus.com/install-theano-on-aws/)

sudo apt-get update

sudo apt-get -y dist-upgrade  

codingneo

1. Launch the g2.2xlarge instance, select a security group with SSH rule
2. ssh to the instance

ssh -i [pem file] ubuntu@[Public DNS of the instance]

3. Update the locale

export LC_ALL="en_US.UTF-8"
export LC_CTYPE="en_US.UTF-8"
sudo dpkg-reconfigure locales

codingneo

Problem

Reference

Learning From Aggregates

1. Novi Quadrianto, Alex J. Smola, Tiberio S. Caetano and Quoc V. Le, Estimating Labels from Label Proportions, JMLR 2009.
2. Giorgio Patrini, Richard Nock, Paul Rivera and Tiberio Caetano, (Almost) No Label No Cry, NIPS 2014 [Supplement] [Code].
3. Richard Nock, Giorgio Patrini and Arik Friedman, Rademacher Observations, Private Data and Boosting, ICML 2015, [PDF] [Ariv Long Version].

Differential Privacy

codingneo

##Reference

• M A H Dempster and E A Medova, Planning for Retirement: Asset Liability Management for Individuals, Asset Liability Management 2010 Yearbook
• E. A. MEDOVA, J. K. MURPHY, A. P. OWEN and K. REHMAN, Individual Asset Liability Management Quantitative Finance, 2008