Last active
September 23, 2024 19:45
-
-
Save intellectronica/ac7269805884436822b4654003605b5d to your computer and use it in GitHub Desktop.
Running multiple LLM calls in parallel with asyncio
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import asyncio | |
from openai import AsyncOpenAI | |
import random | |
oai = AsyncOpenAI() | |
numbers = [str(random.randint(0, 123456789)) for _ in range(7)] | |
async def explain_number(number): | |
print(f'START: Explain the number {number}') # DEBUG | |
result = await oai.chat.completions.create( | |
model='gpt-4o-mini', | |
messages=[{'role': 'user', 'content': f'Explain the number {number}'}], | |
temperature=1.0, | |
max_tokens=23, | |
) | |
print(f'END: Explain the number {number}') # DEBUG | |
return result.choices[0].message.content | |
async def main(): | |
tasks = [explain_number(number) for number in numbers] | |
explanations = await asyncio.gather(*tasks) | |
for explanation in explanations: | |
print(explanation + '\n---\n') | |
asyncio.run(main()) | |
########## | |
# Output # | |
########## | |
# START: Explain the number 67124777 | |
# START: Explain the number 116823779 | |
# START: Explain the number 43996485 | |
# START: Explain the number 2485252 | |
# START: Explain the number 20357363 | |
# START: Explain the number 19815246 | |
# START: Explain the number 41277191 | |
# END: Explain the number 67124777 | |
# END: Explain the number 43996485 | |
# END: Explain the number 41277191 | |
# END: Explain the number 2485252 | |
# END: Explain the number 20357363 | |
# END: Explain the number 116823779 | |
# END: Explain the number 19815246 | |
# The number 67,124,777 can be understood in different contexts, such as mathematics, numerology, or | |
# --- | |
# | |
# The number 116823779 is simply a numerical value and can be examined in various mathematical contexts. Here are a | |
# --- | |
# | |
# The number 43996485 can be analyzed in various ways, such as its mathematical properties, significance, or context | |
# --- | |
# | |
# The number 2485252 can be analyzed from various perspectives: | |
# | |
# 1. **Numerical Properties**: | |
# - | |
# --- | |
# | |
# The number 20,357,363 can be analyzed in various ways depending on the context you want to explore. | |
# --- | |
# | |
# The number 19815246 is simply a numerical value, and without specific context, it can represent various things. | |
# --- | |
# | |
# The number 41277191 can be interpreted or analyzed in various ways depending on the context. Here are a few | |
# --- |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment