Skip to content

Instantly share code, notes, and snippets.

View JossWhittle's full-sized avatar
😸

Joss Whittle JossWhittle

😸
View GitHub Profile
def find_spans(sample, common_words):
spans = []
for word in common_words:
count = 0
s = re.finditer(f'(^|\s){re.escape(word)}[^a-zA-Z]', sample, flags=re.IGNORECASE)
for idx in s:
start, end = idx.span()
count += 1
spans.append([start, end])
@JossWhittle
JossWhittle / example-nucleus-sampling.md
Last active December 1, 2019 20:38
GPT-2 large (7.62 million param) with no fine-tuning, sampled using Nucleus Sampling https://arxiv.org/pdf/1904.09751.pdf with p=0.9.

Global Step: 0 Script: data/scripts/Friends S07E18.txt

TITLE: The One With Joey's Award

SCENE: [Scene: Central Perk, Phoebe, Monica, and Rachel are hanging out.]

Rachel: So Pheebs, you know I was just at this batchelorette party and they had a cake that in the shape of a man's....thing-a-ding-ding, and I was wondering, what you'd thought of that: offensive or amusing?

Phoebe: I am not sure. Monica what do you think. >

import transformers
from transformers import GPT2Tokenizer
from transformers import TFGPT2LMHeadModel
tokenizer = GPT2Tokenizer.from_pretrained('gpt2-large')
model = TFGPT2LMHeadModel.from_pretrained('gpt2-large')
print(sample_sequence('What does the fox say?', tokenizer, model, max_length=32, top_k=100, temperature=1.0, repetition_penalty=2.0))
#include <iostream>
int main(int argc, char* argv[]) {
int i = 2;
((i == 0) ? ([]() {
std::cout << "Branch 1" << std::endl;
})() : ([]() {
std::cout << "Branch 2" << std::endl;
@JossWhittle
JossWhittle / hello.cu
Last active February 12, 2019 10:40
Adds to arrays of ints together element-wise.
// Compile: nvcc hello.cu -o hello
// Run: ./hello
#include <iostream>
#include <assert.h>
// The GPU kernel
// Computes c[i] = a[i] + b[i] for all i
__global__
void hello(const int num_elements, int *in_a, int *in_b, int *out_c) {
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
import glob, os, re
output_file_name = 'output.csv'
print('Writing output to %s...' % (output_file_name))
with open(output_file_name, 'w') as output_file:
file_list = glob.glob('*.txt')
print('Found %d files to process...' % (len(file_list)))
# Perform 2d convolution using runtime-initialization constant evaluation
#
def dense_with_scaling(inputs, fan_out, activation=None, name='dense_with_scaling'):
with tf.variable_scope(name, reuse=tf.AUTO_REUSE):
# Features from previous tensor
fan_in = int(inputs.get_shape()[-1])
# Compute He initialization constant
C = np.sqrt(1.3 * 2.0 / fan_in)
W = tf.get_variable('W', shape=(fan_in, fan_out),
initializer=tf.initializers.truncated_normal()) #stddev=C
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.