Skip to content

Instantly share code, notes, and snippets.

@lshifr
Created March 1, 2021 20:10
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save lshifr/2fa8ff7c950595d2ca32adc41b76bd7d to your computer and use it in GitHub Desktop.
Save lshifr/2fa8ff7c950595d2ca32adc41b76bd7d to your computer and use it in GitHub Desktop.
A version of LazyListSelect, which would allow per-chunk selector
ClearAll[LazyListSelect];
LazyListSelect[l:Streaming`LazyList`$PLazyList, crit_, fullSelect_:None]:=
Module[{selFun, chunkSizeF, mappedF, result, id = Streaming`Common`StreamingUUID[]},
selFun = If[fullSelect =!= None && crit === None,
fullSelect,
Function[chunkData, Select[chunkData, crit]]
];
chunkSizeF =
Function[index,
With[{chunk = Streaming`Components`RandomAccessList`ListPart[Streaming`LazyList`LazyListData[result], index]},
If[chunk === Streaming`Components`RandomAccessList`ListEndElement,
-1,
(* else *)
Length[Streaming`Components`DataChunk`ToData[chunk, Streaming`Components`Cache`$GlobalStreamingCache]]
]
]
];
mappedF = Function[{chunk},
Streaming`Components`DataChunk`DataChunkApplyDelayed[Streaming`Components`DataChunk`FunctionObject[id, selFun],chunk]
];
result =
Streaming`LazyList`LazyListCreate[
Streaming`LazyList`Chunking`ChunkedData[
Streaming`Components`RandomAccessList`ListMap[mappedF, Streaming`LazyList`LazyListData[l]],
Streaming`LazyList`Chunking`FunctionChunkSizesObject[chunkSizeF],
None,
None
],
"ParentList" -> l,
"DelayedVirtualChunkGeneratorConstruction" -> l @ OO`delayedVirtualChunksQ[],
If[Streaming`LazyList`LazyListFiniteListQ[l], "FiniteList" -> True, Sequence @@ {}]
]
];
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment