Skip to content

Instantly share code, notes, and snippets.

@zitterbewegung
zitterbewegung / tldr-with-huggingface.ipynb
Created September 14, 2020 02:21
TLDR with Huggingface
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@zitterbewegung
zitterbewegung / data_cleaning.ipynb
Last active August 9, 2020 02:36
Dialog with a gpt-2
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@zitterbewegung
zitterbewegung / requirements.txt
Last active November 21, 2020 00:07
Voice interface to gpt-2
absl-py==0.9.0
appdirs==1.4.4
argcomplete==1.10.3
arrow==0.15.4
astor==0.8.1
beautifulsoup4==4.9.1
binaryornot==0.4.4
certifi==2017.7.27.1
chardet==3.0.4
clarifai==2.0.32
@zitterbewegung
zitterbewegung / sentiment140-fine-tuning-gpt-2-1-5b-using-tpus.ipynb
Created November 7, 2019 00:30
Sentiment140 Fine-tuning GPT-2 1.5B using TPUs.ipynb
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
import pandas as pd
data = pd.read_csv("/home/ec2-user/SageMaker/gpt-2-simple/gpt_2_simple/src/training.1600000.processed.noemoticon.csv",
encoding='latin-1',
names=["Label",
"tweet_id",
"Date", "Query",
'Username',
'tweet'])
data['tweet'] = '<|startoftext|>' + data['tweet'].astype(str) + "<|endoftext|>"
@zitterbewegung
zitterbewegung / gpt-2-twitter.ipynb
Last active October 24, 2019 02:54
Modified version of Train a GPT-2 Text-Generating Model w/ GPU For Free to work with AWS.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
{
"package_name": "prose",
"requirements": [
{"name" : "prose-resolver",
"UUID" : "ed721f86-c8ae-4360-af40-8c4d0b9485bb",
"hash" : "0c63a75b845e4f7d01107d852e4c2485c51a50aaaa94fc61995e71bbee983a2ac3713831264adb47fb6bd1e058d5f004",
"author" : {"name":"zitterbewegung",
"public_key": "asdf"
},
import spherogram
def generate_random_table(crossing_number, num_components, table_size):
"""
Requires spherogram library (pip install sphereogram).
"""
problem_set = [spherogram.random_link(crossing_number, num_components=num_components, initial_map_gives_link=True, alternating=True) for x in range(1,table_size)]
return problem_set
poly = [x.alexander_polynomial() for x in generate_random_table(10, 1, 10)]
poly
print(responsejson)
print("3rd Concept" +get_concept(responsejson, 2))
print("4rd Concept" +get_concept(responsejson, 3))
print("5th Concept" +get_concept(responsejson, 4))
resp = MessagingResponse()
resp.message("I think it might be one of the following: " +
get_concept(responsejson, 0) + " or " +
get_concept(responsejson, 1) + " or " +
get_concept(responsejson, 2) + " or " +
get_concept(responsejson, 3))