Skip to content

Instantly share code, notes, and snippets.

Avatar

David Rodriguez DavidRdgz

  • San Francisco State University - Graduate Student
  • Berkeley, CA
View GitHub Profile
@DavidRdgz
DavidRdgz / pymc-zero-inflated-normal.py
Last active Mar 27, 2019
Minimal, Complete, Verifiable example of Zero Inflated Normal Distribution with modified zero
View pymc-zero-inflated-normal.py
data = [7.650241800665554e-05,3.923801883327712e-05,3.0,3.0,3.0,7.283467537625357e-05,3.0,5.0,4.578335196970549e-05,8.864074492628704e-05,5.4930696397944266e-05,6.126587295338719e-05,6.870468675141216e-05,5.013033109090004e-05,1.5649705654594946e-05,6.665684055182832e-05,7.778241848184921e-05,6.152523259571738e-05,2.197341773568182e-05,6.514893533227304e-05,9.840354316222558e-05,2.0,5.118635504549967e-05,2.219241550856768e-05,8.707278988254699e-05,7.345254298994388e-05,1.038200304754831e-05,7.486178857899007e-05,4.400363543164666e-05,2.6109114103329375e-05,6.980217556269073e-05,6.497584542503085e-05,7.413016730642387e-05,8.622898038036913e-05,2.5108810275908995e-05,2.0,5.478937414861367e-05,3.416370076636977e-05,3.420693549749728e-05,5.289158824042955e-05,1.3250620985521934e-05,5.5149269115319916e-05,3.3351624592411055e-05,2.0,2.0,1.0,6.0963304813514516e-05,5.174888962660831e-05,3.234479792687862e-05,7.467867763461055e-05,4.008562585999516e-05,3.9392903445443475e-05,8.890909111618834e-05,3.979655341257482e-05
@DavidRdgz
DavidRdgz / fit.txt
Last active Mar 27, 2019
Minimal, Complete, and Verifiable Example Zipf on List[Int]
View fit.txt
7.96 |
| · · · · ·· ·· ·
| ·· ·· · · · · · ·· · ·
| · · ·· · · · ········· ······ ··· ·· ·
| · ·· · ·· ·· ····· ······················ ·
7.62 | · · · ········································ ··· ·
| · · · ·· ·····················∘·········∘············ ·· ·· ·
|· · ·· ·················∘·∘∘∘··∘·················· · ··· ·
| · ·· ···············∘·······∘·∘∘∘·○·∘··∘∘∘················· ·
| · · · ··· ·············∘·∘∘∘∘∘··∘∘∘∘∘∘∘·∘∘··∘∘············· ·
@DavidRdgz
DavidRdgz / annotations.py
Created Jan 7, 2019
Xgboost model on the Prudential life insurance dataset from Kaggle
View annotations.py
"""
A simple example of decluttering the settings for pandas so
that when developing the model and testing it, the dataframe
is a little cleaner and more readable.
"""
def pandas_defaults(defaults, pd):
def decorator(f):
def wrapper(*args, **kwargs):
@DavidRdgz
DavidRdgz / Numpy.scala
Created Jan 2, 2019
Creating numpy-like arrays in scala using implicit class conversion
View Numpy.scala
package numpy
trait NumpyWriter[A] {
def lessThan(list: List[A])(value: A): List[A]
def greaterThan(list: List[A])(value: A): List[A]
def multiply(list: List[A])(value: A): List[A]
def add(list: List[A])(value: A): List[A]
def subtract(list: List[A])(value: A): List[A]
}
@DavidRdgz
DavidRdgz / SparkRainer.scala
Last active Aug 23, 2018
[Rainier] Massive Bayesian Inference in Spark using Rainer
View SparkRainer.scala
import com.stripe.rainier.core.{Normal, Poisson}
import com.stripe.rainier.sampler.{RNG, ScalaRNG}
import org.apache.spark.{SparkConf, SparkContext}
object Driver {
implicit val rng: RNG = ScalaRNG(1527608515939L)
val DROP_BURN_IN = 100
/*
Refer to StackOverflow Q, about serializing methods/objects:
@DavidRdgz
DavidRdgz / Discrete.scala
Last active Aug 18, 2018
[Rainier] Example truncating combinator on base Discrete distributions.
View Discrete.scala
package com.stripe.rainier.core
import com.stripe.rainier.compute.{Evaluator, If, Real}
trait Discrete extends Distribution[Int] {
self: Discrete =>
val emptyEvaluator = new Evaluator(Map.empty)
def logDensity(v: Real): Real
@DavidRdgz
DavidRdgz / README.md
Last active Aug 5, 2018
Negative Binomial Approximation to Normal Threshold
View README.md

nb approximation to normal threshold

There's many ways to test if a negative binomial is approximately normal: e.g.

  • visualize the qq plot
  • normalize the nb sample and perform shapiro-wilkes test

Below is an image of the envelope where the negative binomial parameters create a distribution that is approximately normal.

@DavidRdgz
DavidRdgz / Discrete.scala
Last active Jul 30, 2018
A few discrete probability distributions for Rainier
View Discrete.scala
/**
* Bernoulli distribution with expectation `p`
*
* @param p The probability of success
*/
final case class Bernoulli(p: Real) extends Discrete {
val generator: Generator[Int] =
Generator.require(Set(p)) { (r, n) =>
val u = r.standardUniform
val l = n.toDouble(p)
@DavidRdgz
DavidRdgz / mixture_of_experts.py
Last active Jun 23, 2018
Another toy (binary) mixture of experts model with 4 experts and a gating network.
View mixture_of_experts.py
from keras.models import Model
from keras.layers import Input, Dense, concatenate, dot
from numpy.random import randint
import numpy as np
def my_model(n=20):
inputs = Input(shape=(n,))
m1 = Dense(1)(inputs)
@DavidRdgz
DavidRdgz / expert_mixtures.py
Last active Jun 23, 2018
Really small example (multi-class) mixture of experts model, almost. Technically, belief_per_model function needs to assign probabilities based on a function.
View expert_mixtures.py
from keras.models import Model
from keras.layers import Input, Lambda, Dense
from keras.utils import to_categorical
from numpy.random import randint
import numpy as np
def belief_per_model(x):
x1, x2, x3, x4 = x
return x1 * .2 + x2 * .3 + x3 * .4 + x4 * .1
You can’t perform that action at this time.