Skip to content

Instantly share code, notes, and snippets.

@robdmc
Last active August 29, 2015 14:17
Show Gist options
  • Save robdmc/fcb99f80ecc1532d2b2d to your computer and use it in GitHub Desktop.
Save robdmc/fcb99f80ecc1532d2b2d to your computer and use it in GitHub Desktop.
django queryset chunker
from sys import maxint
def queryset_chunker(queryset, chunksize=10000):
pk = -maxint
last_pk = queryset.order_by('-pk')[0][0]
queryset = queryset.order_by('pk')
while pk < last_pk:
for row in queryset.filter(pk__gt=pk)[:chunksize]:
pk = row[0]
yield row
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment