- The paper describes an unsupervised approach to train a generic, distributed sentence encoder.
- It also describes a vocabulary expansion method to encode words not seen at training time.
- Link to the paper
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
try: | |
page_response = requests.get(page_link, timeout=5) | |
if page_response.status_code == 200: | |
# extract | |
else: | |
print(page_response.status_code) | |
# notify, try again | |
except requests.Timeout as e: | |
print("It is time to timeout") | |
print(str(e)) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import tensorflow as tf | |
import numpy as np | |
corpus_raw = 'He is the king . The king is royal . She is the royal queen ' | |
# convert to lower case | |
corpus_raw = corpus_raw.lower() | |
words = [] | |
for word in corpus_raw.split(): |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
from __future__ import print_function | |
import json | |
import os | |
import numpy as np | |
from gensim.models import Word2Vec | |
from gensim.utils import simple_preprocess | |
from keras.engine import Input | |
from keras.layers import Embedding, merge |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
% 1. Title: Iris Plants Database | |
% | |
% 2. Sources: | |
% (a) Creator: R.A. Fisher | |
% (b) Donor: Michael Marshall (MARSHALL%PLU@io.arc.nasa.gov) | |
% (c) Date: July, 1988 | |
% | |
% 3. Past Usage: | |
% - Publications: too many to mention!!! Here are a few. | |
% 1. Fisher,R.A. "The use of multiple measurements in taxonomic problems" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
""" | |
Minimal character-level Vanilla RNN model. Written by Andrej Karpathy (@karpathy) | |
BSD License | |
""" | |
import numpy as np | |
# data I/O | |
data = open('input.txt', 'r').read() # should be simple plain text file | |
chars = list(set(data)) | |
data_size, vocab_size = len(data), len(chars) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# coding=utf8 | |
import PIL | |
from PIL import ImageFont | |
from PIL import Image | |
from PIL import ImageDraw | |
def text2png(text, fullpath, color = "#000", bgcolor = "#FFF", fontfullpath = None, fontsize = 13, leftpadding = 3, rightpadding = 3, width = 200): | |
REPLACEMENT_CHARACTER = u'\uFFFD' | |
NEWLINE_REPLACEMENT_STRING = ' ' + REPLACEMENT_CHARACTER + ' ' |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
""" | |
Use a Counter to find the most common words in "The Wonderful Wizard of Oz" by | |
L. Frank Baum. | |
Available in (mostly) plain text at: | |
https://archive.org/stream/wonderfulwizardo00baumiala/wonderfulwizardo00baumiala_djvu.txt | |
Note: This code also counts the words in the header, so it's not a *realistic* | |
applicaton, but more of a demonstration of python's Counter. |
NewerOlder