Skip to content

Instantly share code, notes, and snippets.

@intellectronica
Last active September 23, 2024 19:45
Show Gist options
  • Save intellectronica/ac7269805884436822b4654003605b5d to your computer and use it in GitHub Desktop.
Save intellectronica/ac7269805884436822b4654003605b5d to your computer and use it in GitHub Desktop.
Running multiple LLM calls in parallel with asyncio
import asyncio
from openai import AsyncOpenAI
import random
oai = AsyncOpenAI()
numbers = [str(random.randint(0, 123456789)) for _ in range(7)]
async def explain_number(number):
print(f'START: Explain the number {number}') # DEBUG
result = await oai.chat.completions.create(
model='gpt-4o-mini',
messages=[{'role': 'user', 'content': f'Explain the number {number}'}],
temperature=1.0,
max_tokens=23,
)
print(f'END: Explain the number {number}') # DEBUG
return result.choices[0].message.content
async def main():
tasks = [explain_number(number) for number in numbers]
explanations = await asyncio.gather(*tasks)
for explanation in explanations:
print(explanation + '\n---\n')
asyncio.run(main())
##########
# Output #
##########
# START: Explain the number 67124777
# START: Explain the number 116823779
# START: Explain the number 43996485
# START: Explain the number 2485252
# START: Explain the number 20357363
# START: Explain the number 19815246
# START: Explain the number 41277191
# END: Explain the number 67124777
# END: Explain the number 43996485
# END: Explain the number 41277191
# END: Explain the number 2485252
# END: Explain the number 20357363
# END: Explain the number 116823779
# END: Explain the number 19815246
# The number 67,124,777 can be understood in different contexts, such as mathematics, numerology, or
# ---
#
# The number 116823779 is simply a numerical value and can be examined in various mathematical contexts. Here are a
# ---
#
# The number 43996485 can be analyzed in various ways, such as its mathematical properties, significance, or context
# ---
#
# The number 2485252 can be analyzed from various perspectives:
#
# 1. **Numerical Properties**:
# -
# ---
#
# The number 20,357,363 can be analyzed in various ways depending on the context you want to explore.
# ---
#
# The number 19815246 is simply a numerical value, and without specific context, it can represent various things.
# ---
#
# The number 41277191 can be interpreted or analyzed in various ways depending on the context. Here are a few
# ---
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment