Skip to content

Instantly share code, notes, and snippets.

Avatar

Kashif Rasul kashif

View GitHub Profile
@kashif
kashif / cem.md
Last active May 30, 2021
Cross Entropy Method
View cem.md

Cross Entropy Method

How do we solve for the policy optimization problem which is to maximize the total reward given some parametrized policy?

Discounted future reward

To begin with, for an episode the total reward is the sum of all the rewards. If our environment is stochastic, we can never be sure if we will get the same rewards the next time we perform the same actions. Thus the more we go into the future the more the total future reward may diverge. So for that reason it is common to use the discounted future reward where the parameter discount is called the discount factor and is between 0 and 1.

A good strategy for an agent would be to always choose an action that maximizes the (discounted) future reward. In other words we want to maximize the expected reward per episode.

@kashif
kashif / cifar10_wide_resnet.py
Last active May 10, 2021
Keras Wide Residual Networks CIFAR-10
View cifar10_wide_resnet.py
from __future__ import print_function
from keras.datasets import cifar10
from keras.layers import merge, Input
from keras.layers.convolutional import Convolution2D, ZeroPadding2D, AveragePooling2D
from keras.layers.core import Dense, Activation, Flatten, Dropout
from keras.layers.normalization import BatchNormalization
from keras.models import Model
from keras.preprocessing.image import ImageDataGenerator
from keras.utils import np_utils
@kashif
kashif / cifar10_resnet.py
Last active Feb 3, 2021
Keras Pre-activation Residual Network for CIFAR-10
View cifar10_resnet.py
from __future__ import print_function
from keras.datasets import cifar10
from keras.layers import merge, Input
from keras.layers.convolutional import Convolution2D, ZeroPadding2D, AveragePooling2D
from keras.layers.core import Dense, Activation, Flatten
from keras.layers.normalization import BatchNormalization
from keras.models import Model
from keras.preprocessing.image import ImageDataGenerator
from keras.utils import np_utils
@kashif
kashif / fashion_mnist_cnn.py
Last active Nov 27, 2020
Fashion Mnist Benchmark
View fashion_mnist_cnn.py
'''Trains a simple convnet on the Zalando MNIST dataset.
Gets to 81.03% test accuracy after 30 epochs
(there is still a lot of margin for parameter tuning).
3 seconds per epoch on a GeForce GTX 980 GPU with CuDNN 5.
'''
from __future__ import print_function
import numpy as np
from mnist import MNIST
View batch_embedded_GBRT.py
from sklearn.datasets import load_boston
from sklearn.linear_model import (LinearRegression, Ridge,
Lasso, RandomizedLasso)
from sklearn.feature_selection import RFE, f_regression
from sklearn.preprocessing import MinMaxScaler
from sklearn.ensemble import RandomForestRegressor, GradientBoostingRegressor
import numpy as np
#from minepy import MINE
from sklearn.metrics import mean_squared_error
@kashif
kashif / batch_EN.py
Last active Sep 7, 2020
Batch ElasticNet
View batch_EN.py
from sklearn.datasets import load_boston
from sklearn.linear_model import (LinearRegression, Ridge, LassoCV, ElasticNetCV,
ElasticNet, Lasso, RandomizedLasso)
from sklearn.feature_selection import RFE, f_regression
from sklearn.preprocessing import MinMaxScaler
from sklearn.ensemble import RandomForestRegressor, GradientBoostingRegressor
import numpy as np
import pdb
#from minepy import MINE
@kashif
kashif / batch_SGDEN.py
Last active Sep 7, 2020
Batch SGD ElasticNet
View batch_SGDEN.py
from sklearn.datasets import load_boston
from sklearn.linear_model import (LinearRegression, Ridge, SGDRegressor,
Lasso, ElasticNetCV)
from sklearn.preprocessing import MinMaxScaler
import numpy as np
#from minepy import MINE
from sklearn.metrics import mean_squared_error
View evonorm2d.py
import torch
import torch.nn as nn
class EvoNorm2d(nn.Module):
__constants__ = ['num_features', 'eps', 'nonlinearity']
def __init__(self, num_features, eps=1e-5, nonlinearity=True):
super(EvoNorm2d, self).__init__()
@kashif
kashif / geodatabases.md
Created May 27, 2011
GeoDatabases - a discussion
View geodatabases.md

GeoDatabases

What is a GeoDatabase? and how is it diff from a regular db

  • Stores geometries
  • multi-dimensional
  • R-Tree indexing (Query planner uses it?)
  • projections
  • Supports Datums ?
  • Vector and raster support
@kashif
kashif / amsgrad.py
Last active May 13, 2019
Keras implementation of AMSGrad optimizer from "On the Convergence of Adam and Beyond" paper
View amsgrad.py
class AMSgrad(Optimizer):
"""AMSGrad optimizer.
Default parameters follow those provided in the Adam paper.
# Arguments
lr: float >= 0. Learning rate.
beta_1: float, 0 < beta < 1. Generally close to 1.
beta_2: float, 0 < beta < 1. Generally close to 1.
epsilon: float >= 0. Fuzz factor.