Skip to content

Instantly share code, notes, and snippets.

@canismarko
Created February 13, 2024 21:13
Show Gist options
  • Save canismarko/c02165ef10e9caa5717cddbf38c586e7 to your computer and use it in GitHub Desktop.
Save canismarko/c02165ef10e9caa5717cddbf38c586e7 to your computer and use it in GitHub Desktop.
Display the source blob
Display the rendered blob
Raw
{
"cells": [
{
"cell_type": "code",
"execution_count": 11,
"id": "4b359644-6417-4cb2-b024-9eca5998d852",
"metadata": {},
"outputs": [],
"source": [
"import asyncio\n",
"from functools import wraps, partial\n",
"import time"
]
},
{
"cell_type": "markdown",
"id": "43afda0f-27ea-4b7e-9b21-f6ee7fdc900c",
"metadata": {},
"source": [
"# Synchronous\n",
"\n",
"Each call blocks, so we can only run sequentially."
]
},
{
"cell_type": "code",
"execution_count": 27,
"id": "e034888e-0cf4-4c2f-92b8-04c114c5e93d",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Starting thing 1\n",
"Finished thing 1\n",
"Starting thing 2\n",
"Finished thing 2\n",
"Starting thing 3\n",
"Finished thing 3\n"
]
},
{
"data": {
"text/plain": [
"[1, 2, 3]"
]
},
"execution_count": 27,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"def do_slow_thing(sleep_time):\n",
" \"\"\"Some function that takes a while, maybe hits the tiled server.\"\"\"\n",
" print(f\"Starting thing {sleep_time}\")\n",
" time.sleep(sleep_time)\n",
" print(f\"Finished thing {sleep_time}\")\n",
" return sleep_time\n",
"\n",
"[\n",
" do_slow_thing(1),\n",
" do_slow_thing(2),\n",
" do_slow_thing(3),\n",
"]"
]
},
{
"cell_type": "markdown",
"id": "158c0f47-f6db-4a2b-9f34-dd7e6046bb39",
"metadata": {},
"source": [
"# With an Async Decorator\n",
"\n",
"With a decorator, we can make the existing synchronous functions run in a thread and be awaitable. Using ``asyncio.gather`` we can then run them at the same time and get the result."
]
},
{
"cell_type": "code",
"execution_count": 28,
"id": "4dea5e78-0cee-4e24-a7f8-8eb58b08420d",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Starting thing 1\n",
"Starting thing 2\n",
"Starting thing 3\n",
"Finished thing 1\n",
"Finished thing 2\n",
"Finished thing 3\n"
]
},
{
"data": {
"text/plain": [
"[1, 2, 3]"
]
},
"execution_count": 28,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"def awaitable(fn):\n",
" \"\"\"Makes the wrapped function awaitable by running it in a separate thread.\"\"\"\n",
" @wraps(fn)\n",
" async def inner(*args, **kwargs):\n",
" # Create a new partial so we can run it in a new thread\n",
" new_fn = partial(fn, *args, **kwargs)\n",
" # Execute the wrapped function in a thread\n",
" loop = asyncio.get_running_loop()\n",
" return await loop.run_in_executor(None, new_fn)\n",
" return inner\n",
"\n",
"@awaitable\n",
"def do_slow_thing(sleep_time):\n",
" \"\"\"Some function that takes a while, maybe hits the tiled server.\"\"\"\n",
" print(f\"Starting thing {sleep_time}\")\n",
" time.sleep(sleep_time)\n",
" print(f\"Finished thing {sleep_time}\")\n",
" return sleep_time\n",
"\n",
"# Now execute a few of the slow things together\n",
"await asyncio.gather(do_slow_thing(1), do_slow_thing(2), do_slow_thing(3))"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "836f7dbd-ed8c-48ac-8ced-59abf08ec74e",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.15"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment