Skip to content

Instantly share code, notes, and snippets.

View nicolasdespres's full-sized avatar

Nicolas Despres nicolasdespres

View GitHub Profile
@nicolasdespres
nicolasdespres / cycle_range.py
Created March 8, 2017 08:14
Python iterator to cycle over a range
class cycle_range(Iterator):
"""Like `itertools.cycle` but on a range.
Since it works on a range it requires no extra memory contrary to
the standard `itertools.cycle`.
Examples:
>>> list(cycle_range(2, times=3))
[0, 1, 0, 1, 0, 1]
@nicolasdespres
nicolasdespres / shuffle_iter.py
Created March 8, 2017 08:14
Iterate over an iteratable whose items are shuffled
class shuffle_iter(Iterator):
"""Shuffle items of an iterator.
The amount of shuffling is controlled by the `capacity`. The more capacity
the more shuffling.
Args:
`iterable`: the source of data to shuffle
`capacity`: the size of the internal buffer used for shuffling
"""
@nicolasdespres
nicolasdespres / batch_iter.py
Created March 8, 2017 08:13
Iterate over iterable by batch.
class batch_iter(Iterator):
"""Iterate by batch.
"""
def __init__(self, iterable, batch_size=1,
allow_smaller_final_batch=False):
if not isinstance(batch_size, int):
raise TypeError("batch_size must be int, not {}"
.format(type(batch_size).__name__))
if batch_size <= 0:
@nicolasdespres
nicolasdespres / iter_shuffle_batch_range.py
Created March 8, 2017 08:13
Iterator over batch of shuffled indices from a range.
class iter_shuffle_batch_range(Iterator):
"""Iterate over batch of (potentially shuffled) indices in range.
The range is seen as a list of indices (potentially shuffled)
that is repeated `num_cycles` times. This iterator sequentially returns
chunk of `batch_size` items of this sequence.
This object tries to follow the same API as `tf.train.shuffle_batch`
functions so that it can be used easily plugged with other TensorFlow's
routines. The `num_epochs` argument is a noticeable exception because it
@nicolasdespres
nicolasdespres / iter_tensors_slice.py
Created March 8, 2017 08:12
Iterate over slice of tensors.
class iter_tensors_slice(Iterator):
"""Yield slices of tensors.
Sequentially slice `tensors` along `axis` using indices provided
by the `slices` iterable.
Args:
`tensors`: a list of tensors or a single tensors of the same size.
`slices`: an iterable providing indices of each slice.
`axis`: slice tensors along this axis
@nicolasdespres
nicolasdespres / iter_shuffle_batch_tensors.py
Created March 8, 2017 08:12
Iterate of shuffled batch of tensors slices.
class iter_shuffle_batch_tensors(Iterator):
"""Combine `iter_tensors_slice` and `iter_shuffle_batch_range`.
Args:
See `iter_tensors_slice` and `iter_shuffle_batch_range`.
Output:
See `iter_shuffle_batch_range`.
"""
@nicolasdespres
nicolasdespres / iter_shuffle_batch_window.py
Created March 8, 2017 08:11
Iterate over shuffled batch of sliding window on a sequence.
class iter_shuffle_batch_window(Iterator):
"""Iterate a window over an in-memory sequence of data.
Args:
`window_size`: The size of the window (must be smaller than the data)
`window_alignment`: How to align the window around the point: "left",
"right", or "center".
`shifts`: A list of indices to shifts the windows from. By default
it is `[0]` but you can set it `[0, 1]` generate batch of
inputs and targets windows.