Skip to content

Instantly share code, notes, and snippets.

@zzzeek
Last active February 20, 2020 21:16
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save zzzeek/64239e16d25ff799c8ea3276219ac239 to your computer and use it in GitHub Desktop.
Save zzzeek/64239e16d25ff799c8ea3276219ac239 to your computer and use it in GitHub Desktop.
###################################################################
# OLD WAY:
q = session.query(User).options(selectinload(User.addresses))
# result_iter is an iterator
result_iter = iter(q.yield_per(5))
###################################################################
# NEW WAY:
stmt = select(User).options(selectinload(User.addresses))
# result_iter is an iterator that will buffer 5 rows at a time
result_iter = session.execution_options(stream_per=5).execute()
# result_iter is an iterator of sequences with 5 elements per item
result_iter = session.execution_options(stream_per=5).execute(stmt).chunks()
# maybe we also have execution options as kwarg to execute. params are always
# second positional argument. (hey, py3k positional only can we do that?)
result_iter = session.execute(stmt, stream_per=5).chunks()
# this raises: "chunks() requires that stream_per is used; use partitions() to partition a buffered result"
# we have to do it this way because fetching in chunks would like to use
# server side cursors if available / required by the backend for this
# to actually work so we need that intent before we actually execute
# the statement
session.execute(stmt).chunks()
# here's partition - this is iterator of sequences from a
# buffered result, unless stream_results=True
session.execute(stmt).partitions(5)
# this raises: "doesn't make sense to use all() with stream_per option"
result_iter = session.execution_options(stream_per=5).execute().all()
# without stream_per, FutureResult buffers on the client side in all cases.
# Core / ORM.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment