neubig

import requests
import sys
import time

sleep_time = 20
def query_api(url, session):
  global sleep_time
  time.sleep(sleep_time / 1000.0)
  r = session.get(url)
  while r.status_code == 429:

mingfeima

(Internal Tranining Material)

Usually the first step in performance optimization is to do profiling, e.g. to identify performance hotspots of a workload. This gist tells basic knowledge of performance profiling on PyTorch, you will get:

• How to find the bottleneck operator?
• How to trace source file of a particular operator?
• How do I indentify threading issues? (oversubscription)
• How do I tell a specific operator is running efficiently or not?

This tutorial takes one of my recent projects - pssp-transformer as an example to guide you through path of PyTorch CPU peformance optimization. Focus will be on Part 1 & Part 2.

pommedeterresautee

# required by (\ SHELL COMMANDS \)
SHELL:=/bin/bash

VIRT_ENV_FOLDER = ~/.local/share/virtualenvs/xnli
SOURCE_VIRT_ENV = source $(VIRT_ENV_FOLDER)/bin/activate


.PHONY: train
train:
	( \

thomwolf

def top_k_top_p_filtering(logits, top_k=0, top_p=0.0, filter_value=-float('Inf')):
    """ Filter a distribution of logits using top-k and/or nucleus (top-p) filtering
        Args:
            logits: logits distribution shape (vocabulary size)
            top_k >0: keep only top k tokens with highest probability (top-k filtering).
            top_p >0.0: keep the top tokens with cumulative probability >= top_p (nucleus filtering).
                Nucleus filtering is described in Holtzman et al. (http://arxiv.org/abs/1904.09751)
    """
    assert logits.dim() == 1  # batch size 1 for now - could be updated for more but the code would be less clear
    top_k = min(top_k, logits.size(-1))  # Safety check

roeeaharoni

import torch
from pytorch_pretrained_bert import BertForMaskedLM, BertTokenizer
import random

# Requires pytorch_pretrained_bert: https://github.com/huggingface/pytorch-pretrained-BERT

# returns the probabilities over the vocabulary for the masked words in sent
def get_preds(sent):
    tokenized = bert_tokenizer.tokenize(sent)
    tokenized = ['[CLS]'] + ['[MASK]' if x == 'mask' else x for x in tokenized] + ['[SEP]']

kylemcdonald

W4ngatang

''' Script for downloading all GLUE data.

Note: for legal reasons, we are unable to host MRPC.
You can either use the version hosted by the SentEval team, which is already tokenized, 
or you can download the original data from (https://download.microsoft.com/download/D/4/6/D46FF87A-F6B9-4252-AA8B-3604ED519838/MSRParaphraseCorpus.msi) and extract the data from it manually.
For Windows users, you can run the .msi file. For Mac and Linux users, consider an external library such as 'cabextract' (see below for an example).
You should then rename and place specific files in a folder (see below for an example).

mkdir MRPC
cabextract MSRParaphraseCorpus.msi -d MRPC

matpalm

#!/usr/bin/env python3

from PIL import Image
import numpy as np
import tensorflow as tf
import tensorflow_hub as hub

# smooth values from point a to point b.
STEPS = 100
pt_a = np.random.normal(size=(512))

jeasinema

#!/usr/bin/env python
# -*- coding:UTF-8 -*-

import torch
import torch.nn as nn
import torch.nn.init as init


def weight_init(m):
    '''

kastnerkyle

import numpy as np
from scipy.misc import imread
from itertools import product


# https://www.mathworks.com/matlabcentral/fileexchange/36538-very-fast-mutual-information-betweentwo-images?focused=3869473&tab=function
def mutual_info(X, Y, levels=256):
    # example inputs are images, but it shouldn't matter
    X = X.astype("float32")
    Y = Y.astype("float32")