Skip to content

Instantly share code, notes, and snippets.

View sgugger's full-sized avatar

Sylvain Gugger sgugger

View GitHub Profile
@sgugger
sgugger / WordFinder.py
Created January 30, 2018 16:26
A small program to find words on a grid.
"""
Created on Mon Jan 29 08:27:26 2018
On a random grid of letters, find the word that's worth the most of points.
A word can be drawn going in any direction (even diagonals), changing direction
at any time is allowed, the only thing that isn't is to use the same letter
twice.
The score of a word is the sum of the scores of the letters (depending on their
frequency) multiplied by its length minus two.
Example of use (have the dictionary in the same directory as the python file):
generate_grid()
import torch
import torch.nn as nn
import torch.optim as optim
import torch.nn.functional as F
class WeightDropout(nn.Module):
"A module that warps another layer in which some weights will be replaced by 0 during training."
def __init__(self, module, dropout, layer_names=['weight_hh_l0']):
super().__init__()
@sgugger
sgugger / Weight_drop.ipynb
Created September 5, 2018 01:54
Notebooks/Weight_drop.ipynb
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@sgugger
sgugger / Weight_drop.ipynb
Last active September 5, 2018 19:52
Notebooks/Weight_drop.ipynb
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
import TensorFlow
var xTrain = Tensor<Float>(randomNormal: [1024, 784])
var yTrain = Tensor<Int32>(repeating: 0, shape: [1024])
public struct MyModel: Layer {
public var layer1: Dense<Float>
public var layer2: Dense<Float>
public init(nIn: Int, nHid: Int, nOut: Int){
@sgugger
sgugger / numpynn
Created July 1, 2019 17:06
Numpy nn
# -*- coding: utf-8 -*-
import numpy as np
import _pickle as cPickle
import gzip
import json
import sys
import time
class Quadratic(object):
@staticmethod
@sgugger
sgugger / update_swift.sh
Created April 16, 2019 13:03
Updating S4TF
#!/usr/bin/env bash
pushd ~/swift/
rm -rf usr
popd
pushd ~/download/
rm swift-tensorflow-DEVELOPMENT-cuda10.0-cudnn7-ubuntu18.04.tar.gz
wget https://storage.googleapis.com/s4tf-kokoro-artifact-testing/latest/swift-tensorflow-DEVELOPMENT-cuda10.0-cudnn7-ubuntu18.04.tar.gz
tar -xf swift-tensorflow-DEVELOPMENT-cuda10.0-cudnn7-ubuntu18.04.tar.gz
mv usr/ ~/swift/
mv ~/swift/usr/lib/python3.6 ~/swift/usr/lib/python3.7
@sgugger
sgugger / automodel_for_multiple_choice.py
Last active June 15, 2020 12:41
Auto model for multiple choice
import torch
import torch.nn as nn
from transformers import PreTrainedModel, AutoConfig, AutoModel
class ModelWithMultipleChoiceHead(PreTrainedModel):
def __init__(self, config):
super().__init__(config)
base_model = AutoModel.from_config(config)
@sgugger
sgugger / convert_to_md.py
Created June 20, 2023 20:01
Convert all MDX to MD in the doc folder of a HF repo
import argparse
import re
import shutil
from pathlib import Path
from tqdm import tqdm
DISCLAIMER = """⚠️ Note that this file is in Markdown but contain specific syntax for our doc-builder (similar to MDX) that may not be
rendered properly in your Markdown viewer."""
@sgugger
sgugger / LARS.py
Last active November 27, 2023 04:39
from torch.optim.optimizer import Optimizer, required
class LARS(Optimizer):
def __init__(self, params, lr=required, momentum=0, dampening=0,
weight_decay=0, nesterov=False, eta=0.001):
if lr is not required and lr < 0.0:
raise ValueError("Invalid learning rate: {}".format(lr))
if momentum < 0.0:
raise ValueError("Invalid momentum value: {}".format(momentum))