This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
class cycle_range(Iterator): | |
"""Like `itertools.cycle` but on a range. | |
Since it works on a range it requires no extra memory contrary to | |
the standard `itertools.cycle`. | |
Examples: | |
>>> list(cycle_range(2, times=3)) | |
[0, 1, 0, 1, 0, 1] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
class shuffle_iter(Iterator): | |
"""Shuffle items of an iterator. | |
The amount of shuffling is controlled by the `capacity`. The more capacity | |
the more shuffling. | |
Args: | |
`iterable`: the source of data to shuffle | |
`capacity`: the size of the internal buffer used for shuffling | |
""" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
class batch_iter(Iterator): | |
"""Iterate by batch. | |
""" | |
def __init__(self, iterable, batch_size=1, | |
allow_smaller_final_batch=False): | |
if not isinstance(batch_size, int): | |
raise TypeError("batch_size must be int, not {}" | |
.format(type(batch_size).__name__)) | |
if batch_size <= 0: |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
class iter_shuffle_batch_range(Iterator): | |
"""Iterate over batch of (potentially shuffled) indices in range. | |
The range is seen as a list of indices (potentially shuffled) | |
that is repeated `num_cycles` times. This iterator sequentially returns | |
chunk of `batch_size` items of this sequence. | |
This object tries to follow the same API as `tf.train.shuffle_batch` | |
functions so that it can be used easily plugged with other TensorFlow's | |
routines. The `num_epochs` argument is a noticeable exception because it |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
class iter_tensors_slice(Iterator): | |
"""Yield slices of tensors. | |
Sequentially slice `tensors` along `axis` using indices provided | |
by the `slices` iterable. | |
Args: | |
`tensors`: a list of tensors or a single tensors of the same size. | |
`slices`: an iterable providing indices of each slice. | |
`axis`: slice tensors along this axis |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
class iter_shuffle_batch_tensors(Iterator): | |
"""Combine `iter_tensors_slice` and `iter_shuffle_batch_range`. | |
Args: | |
See `iter_tensors_slice` and `iter_shuffle_batch_range`. | |
Output: | |
See `iter_shuffle_batch_range`. | |
""" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
class iter_shuffle_batch_window(Iterator): | |
"""Iterate a window over an in-memory sequence of data. | |
Args: | |
`window_size`: The size of the window (must be smaller than the data) | |
`window_alignment`: How to align the window around the point: "left", | |
"right", or "center". | |
`shifts`: A list of indices to shifts the windows from. By default | |
it is `[0]` but you can set it `[0, 1]` generate batch of | |
inputs and targets windows. |