Skip to content

Instantly share code, notes, and snippets.

Gavin Gray gngdb

Block or report user

Report or block gngdb

Hide content and notifications from this user.

Learn more about blocking users

Contact Support about this user’s behavior.

Learn more about reporting abuse

Report abuse
View GitHub Profile
@thomwolf
thomwolf / top-k-top-p.py
Last active Mar 5, 2020
Sample the next token from a probability distribution using top-k and/or nucleus (top-p) sampling
View top-k-top-p.py
def top_k_top_p_filtering(logits, top_k=0, top_p=0.0, filter_value=-float('Inf')):
""" Filter a distribution of logits using top-k and/or nucleus (top-p) filtering
Args:
logits: logits distribution shape (vocabulary size)
top_k >0: keep only top k tokens with highest probability (top-k filtering).
top_p >0.0: keep the top tokens with cumulative probability >= top_p (nucleus filtering).
Nucleus filtering is described in Holtzman et al. (http://arxiv.org/abs/1904.09751)
"""
assert logits.dim() == 1 # batch size 1 for now - could be updated for more but the code would be less clear
top_k = min(top_k, logits.size(-1)) # Safety check
@eamartin
eamartin / notebook.ipynb
Last active Feb 4, 2020
Understanding & Visualizing Self-Normalizing Neural Networks
View notebook.ipynb
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@gngdb
gngdb / shortbold.md
Last active Aug 3, 2017
Jupyter shortbold markdown cell, just paste into a markdown cell to enjoy shortbold MathJax for the rest of the notebook!
View shortbold.md

$ \newcommand{\aB}{\mathbf{a}} \newcommand{\bB}{\mathbf{b}} \newcommand{\cB}{\mathbf{c}} \newcommand{\dB}{\mathbf{d}} \newcommand{\eB}{\mathbf{e}} \newcommand{\fB}{\mathbf{f}} \newcommand{\gB}{\mathbf{g}} \newcommand{\hB}{\mathbf{h}} \newcommand{\iB}{\mathbf{i}}

View Wikipedia_fastText_gender_names.csv
Name t
Jovan 0.143522377788
Wilford 0.171813290491
Newton 0.192343843426
Maurice 0.193607112432
Emmanuel 0.20571087052
Joseph 0.210762071958
Milton 0.21296788724
Ahmad 0.214983745995
Julius 0.218052193228
@simonkamronn
simonkamronn / hyperband.py
Created Nov 11, 2016
Hyperband for hyperparameter optimization
View hyperband.py
# https://people.eecs.berkeley.edu/~kjamieson/hyperband.html
# you need to write the following hooks for your custom problem
from problem import get_random_hyperparameter_configuration,run_then_return_val_loss
max_iter = 81 # maximum iterations/epochs per configuration
eta = 3 # defines downsampling rate (default=3)
logeta = lambda x: log(x)/log(eta)
s_max = int(logeta(max_iter)) # number of unique executions of Successive Halving (minus one)
B = (s_max+1)*max_iter # total number of iterations (without reuse) per execution of Succesive Halving (n,r)
View animate-matplotlib-python.ipynb
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
You can’t perform that action at this time.