Jiew Peng Lim jiewpeng
tokestermw
/ tf_dataset_api_text.py
Last active
December 18, 2018 06:32
Using the new Dataset API from TensorFlow 1.2.0, return padded and batched tensors from text data where each line is a sentence.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import numpy as np | |
| import tensorflow as tf | |
| _major_version, _minor_version, _ = map(int, tf.__version__.split('-')[0].split('.')) | |
| assert _major_version >= 1 and _minor_version >= 2, "requires TensorFlow 1.2.0 and above" | |
| text_data_path = "./z_sentences.txt" | |
| MAX_SEQUENCE_LENGTH = 10 |