Skip to content

Instantly share code, notes, and snippets.

@alamb
Created January 7, 2022 21:58
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save alamb/949009f36c9e251833b30813acf78886 to your computer and use it in GitHub Desktop.
Save alamb/949009f36c9e251833b30813acf78886 to your computer and use it in GitHub Desktop.
Test of creating arrow arrays from iterators that incorrectly report their bounds
use arrow::array::UInt64Array;
// Make an iterator that is untruthful about its actual length (say it is shorter than it really is):
struct BadIterator {
/// where the iterator currently is
cur: u64,
/// How many items will this iterator *actually* make
pub limit: u64,
/// How many items this iterator claims it will make
pub claimed: u64,
}
impl BadIterator {
/// Create a new iterator for <limit> items, but that reports to
/// produce <claimed> items
fn new(limit: u64, claimed: u64) -> Self {
Self{
cur: 0,
limit,
claimed
}
}
}
impl Iterator for BadIterator {
type Item = Option<u64>;
fn next(&mut self) -> Option<Self::Item> {
if self.cur < self.limit {
let t = self.cur;
self.cur += 1;
Some(Some(t))
} else {
None
}
}
/// report whatever the iterator says to
fn size_hint(&self) -> (usize, Option<usize>) {
(0, Some(self.claimed as usize))
}
}
fn from_vec(v: Vec<u64>) -> UInt64Array {
v.into_iter()
.map(Some)
.collect()
}
fn main() {
println!("Starting...");
// correctly report size:
let array: UInt64Array = BadIterator::new(3,3).collect();
assert_eq!(array, from_vec(vec![0,1,2]));
// incorrectly correctly report size to be 10
let array: UInt64Array = BadIterator::new(3,10).collect();
assert_eq!(array, from_vec(vec![0,1,2]));
// incorrectly correctly report size to be 1 (too small)
let array: UInt64Array = BadIterator::new(3,1).collect();
assert_eq!(array, from_vec(vec![0,1,2]));
println!("Done...");
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment