Skip to content

Instantly share code, notes, and snippets.

Avatar
🥱
Launching https://wellsaidlabs.com

Michael Petrochuk PetrochukM

🥱
Launching https://wellsaidlabs.com
View GitHub Profile
@PetrochukM
PetrochukM / hyperband.py
Last active Feb 6, 2021
Here we implement hyperband and successive halving adaptions. We found that the original hyperband implementation was messy and not tested. We also wanted to adapt it to include model reuse.
View hyperband.py
"""
We implement additional hyperparameter optimization methods not present in
https://scikit-optimize.github.io/.
Gist: https://gist.github.com/Deepblue129/2c5fae9daf0529ed589018c6353c9f7b
"""
import math
import logging
import random
@PetrochukM
PetrochukM / top_k_viterbi.py
Last active Apr 3, 2020
Implemented a Top K Viterbi Decoder algorithm in PyTorch. Useful for Conditional Random Fields (CRFs)-based probabilistic graphical modelling. Learn more here: https://nlp.stanford.edu/joberant/esslli_2016/kbest-ict.pdf
View top_k_viterbi.py
import torch
# Credits to AllenNLP for the base implementation and base tests:
# https://github.com/allenai/allennlp/blob/master/allennlp/nn/util.py#L174
# Modified AllenNLP `viterbi_decode` to support `top_k` sequences efficiently.
def viterbi_decode(tag_sequence: torch.Tensor, transition_matrix: torch.Tensor, top_k: int=5):
"""
Perform Viterbi decoding in log space over a sequence given a transition matrix
specifying pairwise (transition) potentials between tags and a matrix of shape
@PetrochukM
PetrochukM / optimizer_1.py
Last active Jul 26, 2019
PyTorch Optimizer_1 from `Neural Optimizer Search with Reinforcement Learning`
View optimizer_1.py
import torch
from torch.optim import Optimizer
class Optimizer_1(Optimizer):
"""Implements Optimizer_1 algorithm.
It was been proposed in `http://proceedings.mlr.press/v70/bello17a/bello17a.pdf`.
@PetrochukM
PetrochukM / subword_text_tokenizer.py
Last active Sep 18, 2020
Tensor2Tensor Subword Text Tokenizer.
View subword_text_tokenizer.py
# coding=utf-8
# Copyright 2017 The Tensor2Tensor Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software