block-coroutine.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Making an async coroutine blocking\n",
    "\n",
    "For @amueller"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import asyncio\n",
    "import threading\n",
    "from concurrent.futures import ThreadPoolExecutor"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's say I have a coroutine, but want to use it from a blocking API:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "async def some_coroutine(n=1):\n",
    "    \"\"\"A coroutine that takes some time\"\"\"\n",
    "    for i in range(n):\n",
    "        await asyncio.sleep(0.1)\n",
    "        print(i)\n",
    "    return f\"I ran in thread {threading.get_ident()}\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "IPython suports top-level await to run on the current eventloop"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0\n",
      "1\n",
      "2\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'I ran in thread 4449029632'"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "await some_coroutine(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "At the library level, you can pass coroutines to `asyncio.run` in a different thread to turn any `async def` function into a blocking one."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0\n",
      "1\n",
      "2\n",
      "3\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'I ran in thread 123145466920960'"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def block_coroutine(n):\n",
    "    with ThreadPoolExecutor(1) as pool:\n",
    "        return pool.submit(lambda: asyncio.run(some_coroutine(n))).result()\n",
    "\n",
    "block_coroutine(4)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This assumes it's self-contained, i.e. doesn't refer to other things running on the same loop,\n",
    "as the loop will be created and destroyed around just this call.\n",
    "\n",
    "A similar, but more complicated approach can hand off things to a *persistent* eventloop in another thread,\n",
    "using the threadsafe concurrent.futures to coordinate. This is how IPython Parallel's client works!."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}