Skip to content

Instantly share code, notes, and snippets.

@boychaboy

boychaboy/batching

Last active August 19, 2023 07:42
Show Gist options
  • Save boychaboy/f05d4ce31c65f32329c5ce83c3f37ee8 to your computer and use it in GitHub Desktop.
Save boychaboy/f05d4ce31c65f32329c5ce83c3f37ee8 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
batching
def batch(iterable, n=1):
total = len(iterable)
for ndx in range(0, total, n):
yield iterable[ndx:min(ndx + n, total)]
for batch_response in batch(responses, args.batch_size):
prompt_hate = beep_hate([r['prompt'] for r in batch_response], args.device)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment