This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
def load_words(file_path: str) -> List[Word]: | |
"""Load and cleanup the data.""" | |
words = load_words_raw(file_path) | |
print(f"Loaded {len(words)} words.") | |
words = remove_stop_words(words) | |
print(f"Removed stop words, {len(words)} remain.") | |
words = remove_duplicates(words) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
def closest_analogies( | |
left2: str, left1: str, right2: str, words: List[Word] | |
) -> List[Tuple[float, Word]]: | |
word_left1 = find_word(left1, words) | |
word_left2 = find_word(left2, words) | |
word_right2 = find_word(right2, words) | |
vector = add_vectors( | |
sub_vectors(word_left1.vector, word_left2.vector), | |
word_right2.vector) | |
closest = sorted_by_similarity(words, vector)[:10] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
>>> print_related(words, 'spain') | |
britain, england, france, europe, germany, spanish, italy | |
>>> print_related(words, 'called') | |
termed, dubbed, named, referred, nicknamed, titled, described | |
>>> print_related(words, 'although') | |
though, however, but, whereas, while, since, Nevertheless | |
>>> print_related(words, 'arms') |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
def print_related(words: List[Word], text: str) -> None: | |
base_word = find_word(text, words) | |
sorted_words = [ | |
word.text for (dist, word) in | |
sorted_by_similarity(words, base_word.vector) | |
if word.text.lower() != base_word.text.lower() | |
] | |
print(', '.join(sorted_words[:7])) | |
def find_word(words: List[Word], text: str) -> Word: |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
def sorted_by_similarity(words: List[Word], base_vector: Vector) -> List[Tuple[float, Word]]: | |
"""Returns words sorted by cosine distance to a given vector, most similar first""" | |
words_with_distance = [(cosine_similarity(base_vector, w.vector), w) for w in words] | |
# We want cosine similarity to be as large as possible (close to 1) | |
return sorted(words_with_distance, key=lambda t: t[0], reverse=True) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
def vector_len(v: Vector) -> float: | |
return math.sqrt(sum([x*x for x in v])) | |
def dot_product(v1: Vector, v2: Vector) -> float: | |
assert len(v1) == len(v2) | |
return sum([x*y for (x,y) in zip(v1, v2)]) | |
def cosine_similarity(v1: Vector, v2: Vector) -> float: | |
""" | |
Returns the cosine of the angle between the two vectors. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
words = load_words('data/words.vec') |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
from typing import List | |
Vector = List[float] | |
class Word: | |
def __init__(self, text: str, vector: Vector) -> None: | |
self.text = text | |
self.vector = vector |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Thanks to 91 contributors who put [217 commits](https://github.com/facebook/react-native/compare/0.22-stable...0.23-stable) into **React Native 0.23**! | |
Special thanks to @knowbody for writing these release notes! | |
## New features | |
- Add ability to silence packager logs to stdout - https://github.com/facebook/react-native/commit/d5445d5fbc5ea6d1ec01c9d4e7692613a216967f | |
- Add the possibility to `console.error`/redbox on promise rejections - https://github.com/facebook/react-native/commit/f87b673a2959b8d05703c2c83c89d32fe38d2e75 | |
- Add more performance logs and improve Systrace support - https://github.com/facebook/react-native/commit/f6853b8eacec250df1a73f56d7ef9d32b91615e1 | |
- Initial implementation of the Navigator with NavigationExperimental. - https://github.com/facebook/react-native/commit/fa5783e3ee1c5e0b73231a2cf0703545cf9b91ac |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Thanks to 91 contributors who put [217 commits](https://github.com/facebook/react-native/compare/0.22-stable...0.23-stable) into **React Native 0.23**! | |
Special thanks to @knowbody for writing these release notes! | |
## New features | |
- Add ability to silence packager logs to stdout - https://github.com/facebook/react-native/commit/d5445d5fbc5ea6d1ec01c9d4e7692613a216967f | |
- Add the possibility to `console.error`/redbox on promise rejections - https://github.com/facebook/react-native/commit/f87b673a2959b8d05703c2c83c89d32fe38d2e75 | |
- Add more performance logs and improve Systrace support - https://github.com/facebook/react-native/commit/f6853b8eacec250df1a73f56d7ef9d32b91615e1 | |
- Initial implementation of the Navigator with NavigationExperimental. - https://github.com/facebook/react-native/commit/fa5783e3ee1c5e0b73231a2cf0703545cf9b91ac |
NewerOlder