Skip to content

Instantly share code, notes, and snippets.

View j-min's full-sized avatar

Jaemin Cho j-min

View GitHub Profile
@VikingPenguinYT
VikingPenguinYT / dropout_bayesian_approximation_tensorflow.py
Last active December 13, 2023 01:59
Implementing Dropout as a Bayesian Approximation in TensorFlow
import numpy as np
import tensorflow as tf
import matplotlib.pyplot as plt
from tensorflow.contrib.distributions import Bernoulli
class VariationalDense:
"""Variational Dense Layer Class"""
def __init__(self, n_in, n_out, model_prob, model_lam):
self.model_prob = model_prob
@thomwolf
thomwolf / top-k-top-p.py
Last active January 2, 2024 07:43
Sample the next token from a probability distribution using top-k and/or nucleus (top-p) sampling
def top_k_top_p_filtering(logits, top_k=0, top_p=0.0, filter_value=-float('Inf')):
""" Filter a distribution of logits using top-k and/or nucleus (top-p) filtering
Args:
logits: logits distribution shape (vocabulary size)
top_k >0: keep only top k tokens with highest probability (top-k filtering).
top_p >0.0: keep the top tokens with cumulative probability >= top_p (nucleus filtering).
Nucleus filtering is described in Holtzman et al. (http://arxiv.org/abs/1904.09751)
"""
assert logits.dim() == 1 # batch size 1 for now - could be updated for more but the code would be less clear
top_k = min(top_k, logits.size(-1)) # Safety check
@nikitakit
nikitakit / tf_beam_decoder.py
Last active January 6, 2024 08:48
Tensorflow Beam Search
"""
Beam decoder for tensorflow
Sample usage:
```
from tf_beam_decoder import beam_decoder
decoded_sparse, decoded_logprobs = beam_decoder(
cell=cell,
@mattbasta
mattbasta / codegen.py
Created January 22, 2011 18:16
A module to "unparse" a Python AST tree.
# -*- coding: utf-8 -*-
"""
codegen
~~~~~~~
Extension to ast that allow ast -> python code generation.
:copyright: Copyright 2008 by Armin Ronacher.
:license: BSD.
"""
@teamdandelion
teamdandelion / labels_1024.tsv
Last active February 6, 2024 08:33
TensorBoard: TF Dev Summit Tutorial
We can make this file beautiful and searchable if this error is corrected: No tabs found in this TSV file in line 0.
7
2
1
0
4
1
4
9
5
9
@application2000
application2000 / how-to-install-latest-gcc-on-ubuntu-lts.txt
Last active February 21, 2024 03:02
How to install latest gcc on Ubuntu LTS (12.04, 14.04, 16.04)
These commands are based on a askubuntu answer http://askubuntu.com/a/581497
To install gcc-6 (gcc-6.1.1), I had to do more stuff as shown below.
USE THOSE COMMANDS AT YOUR OWN RISK. I SHALL NOT BE RESPONSIBLE FOR ANYTHING.
ABSOLUTELY NO WARRANTY.
If you are still reading let's carry on with the code.
sudo apt-get update && \
sudo apt-get install build-essential software-properties-common -y && \
sudo add-apt-repository ppa:ubuntu-toolchain-r/test -y && \
@offchan42
offchan42 / Machine Learning Curriculum.md
Last active March 2, 2024 00:02
Machine learning resources and related artificial intelligence concepts.
@InnovArul
InnovArul / tied_linear.py
Last active April 11, 2024 11:01
tied linear layer experiment
import torch, torch.nn as nn, torch.nn.functional as F
import numpy as np
import torch.optim as optim
# tied autoencoder using off the shelf nn modules
class TiedAutoEncoderOffTheShelf(nn.Module):
def __init__(self, inp, out, weight):
super().__init__()
self.encoder = nn.Linear(inp, out, bias=False)
self.decoder = nn.Linear(out, inp, bias=False)
@wangruohui
wangruohui / Install NVIDIA Driver and CUDA.md
Last active April 23, 2024 02:03
Install NVIDIA Driver and CUDA on Ubuntu / CentOS / Fedora Linux OS