Skip to content

Instantly share code, notes, and snippets.

@zonca

zonca/01_environment.ipynb

Created October 26, 2023 15:47
Show Gist options
  • Save zonca/ab3f9f3db475331f6d8d68731636a70e to your computer and use it in GitHub Desktop.
Save zonca/ab3f9f3db475331f6d8d68731636a70e to your computer and use it in GitHub Desktop.
Download ZIP
Display the source blob
Display the rendered blob
Raw
01_environment.ipynb
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Display the source blob
Display the rendered blob
Raw
02_dataframe.ipynb
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Display the source blob
Display the rendered blob
Raw
03_array.ipynb
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Display the source blob
Display the rendered blob
Raw
04_delayed.ipynb
{
"cells": [
{
"cell_type": "markdown",
"id": "ab072a3e",
"metadata": {},
"source": [
"<img src=\"https://raw.githubusercontent.com/dask/dask/main/docs/source/images/dask_horizontal.svg\"\n",
" width=\"60%\"\n",
" alt=\"Dask logo\\\" />\n",
"\n",
"# Dask Delayed\n",
"\n",
"Dask DataFrames, Dask Arrays and Dask-ML are parallel versions of PyData libraries you likely know and love. But sometimes we encounter problems that could benefit from parallel computing but that do no fit neatly into a DataFrame, Array or Machine-Learning workflow.\n",
"\n",
"Dask delayed is an interface that can be used to parallelize existing Python code and custom algorithms. \n",
"\n",
"A first step to determine if we can use `dask.delayed` is to identify if there is some level of parallelism that we haven't exploit and hopefully `dask.delayed` will take care of it. \n",
"\n",
"The following two functions will perform simple computations, where we use the `sleep` to simulate work. "
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "ddfb8053",
"metadata": {},
"outputs": [],
"source": [
"from time import sleep\n",
"\n",
"def inc(x):\n",
" \"\"\"Increments x by one\"\"\"\n",
" sleep(1)\n",
" return x + 1\n",
"\n",
"def add(x, y):\n",
" \"\"\"Adds x and y\"\"\"\n",
" sleep(1)\n",
" return x + y"
]
},
{
"cell_type": "markdown",
"id": "54956d17",
"metadata": {},
"source": [
"Let's do some operations and time these functions using the `%%time` magic at the beginning of the cell. "
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "45bb4aa1",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"CPU times: user 6.53 ms, sys: 0 ns, total: 6.53 ms\n",
"Wall time: 3 s\n"
]
}
],
"source": [
"%%time\n",
"\n",
"x = inc(1)\n",
"y = inc(2)\n",
"z = add(x, y)"
]
},
{
"cell_type": "markdown",
"id": "f7c10bb2",
"metadata": {},
"source": [
"The execution of the cell above took three seconds, this happens because we are calling each function sequentially. The computations above can be represented by the following graph:\n",
"\n",
"<img src=\"https://raw.githubusercontent.com/dask/dask/main/docs/source/images/inc-add.svg\" \n",
" width=\"55%\"\n",
" alt=\"Dask graph\\\" />\n",
"\n",
"From looking at the task graph, the opportunity for parallelization is more evident since the the two calls to the `inc` function are completely independent of one-another. Let's explore how `dask.delayed` can help us with this.\n",
"\n",
"\n",
"### `dask.delayed` \n",
"\n",
"We can use `dask.delayed` to transform the `inc` and `add` functions into \"lazy\" versions of themselves. "
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "acfae103",
"metadata": {},
"outputs": [],
"source": [
"from dask import delayed"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "99e3dbf8",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"CPU times: user 576 µs, sys: 106 µs, total: 682 µs\n",
"Wall time: 630 µs\n"
]
}
],
"source": [
"%%time\n",
"\n",
"# x = inc(1)\n",
"# y = inc(2)\n",
"# z = add(x, y)\n",
"\n",
"a = delayed(inc)(1)\n",
"b = delayed(inc)(2)\n",
"c = delayed(add)(a, b)"
]
},
{
"cell_type": "markdown",
"id": "f12638d6",
"metadata": {},
"source": [
"When we call the `delayed` version of the functions by passing the arguments, the original function is isn't actually called yet, that's why the execution finishes very quickly. When we called the `delayed` version of the functions, a `delayed` object is made, which keeps track of the functions to call and what arguments to pass to it. \n",
"\n",
"If we inspect `c`, we will notice that it instead of having the value five, we have what is called a `delayed` object."
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "d885f706",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Delayed('add-5c32e425-aacf-428c-81b7-a6a171b26835')\n"
]
}
],
"source": [
"print(c)"
]
},
{
"cell_type": "markdown",
"id": "37007eaf",
"metadata": {},
"source": [
"We can visualize this objects by doing:"
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "063f5c18",
"metadata": {},
"outputs": [
{
"data": {
"image/svg+xml": [
"<svg xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"134pt\" height=\"289pt\" viewBox=\"0.00 0.00 134.00 289.04\">\n",
"<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(4 285.0351)\">\n",
"<title>%3</title>\n",
"<polygon fill=\"#ffffff\" stroke=\"transparent\" points=\"-4,4 -4,-285.0351 130,-285.0351 130,4 -4,4\"/>\n",
"<!-- 3425153061629363430 -->\n",
"<g id=\"node1\" class=\"node\">\n",
"<title>3425153061629363430</title>\n",
"<ellipse fill=\"none\" stroke=\"#000000\" cx=\"63\" cy=\"-181.8937\" rx=\"27.2835\" ry=\"27.2835\"/>\n",
"<text text-anchor=\"middle\" x=\"63\" y=\"-177.6937\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">add</text>\n",
"</g>\n",
"<!-- &#45;7350354309216508293 -->\n",
"<g id=\"node2\" class=\"node\">\n",
"<title>-7350354309216508293</title>\n",
"<polygon fill=\"none\" stroke=\"#000000\" points=\"90,-281.0351 36,-281.0351 36,-245.0351 90,-245.0351 90,-281.0351\"/>\n",
"</g>\n",
"<!-- 3425153061629363430&#45;&gt;&#45;7350354309216508293 -->\n",
"<g id=\"edge1\" class=\"edge\">\n",
"<title>3425153061629363430-&gt;-7350354309216508293</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M63,-209.3645C63,-217.5482 63,-226.5286 63,-234.7626\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"59.5001,-235.0272 63,-245.0272 66.5001,-235.0272 59.5001,-235.0272\"/>\n",
"</g>\n",
"<!-- 2758571765155845516 -->\n",
"<g id=\"node3\" class=\"node\">\n",
"<title>2758571765155845516</title>\n",
"<polygon fill=\"none\" stroke=\"#000000\" points=\"54,-118.7523 0,-118.7523 0,-82.7523 54,-82.7523 54,-118.7523\"/>\n",
"</g>\n",
"<!-- 2758571765155845516&#45;&gt;3425153061629363430 -->\n",
"<g id=\"edge2\" class=\"edge\">\n",
"<title>2758571765155845516-&gt;3425153061629363430</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M34.9896,-118.7602C38.757,-127.2518 43.3824,-137.677 47.7808,-147.5906\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"44.638,-149.1375 51.8928,-156.8588 51.0365,-146.2986 44.638,-149.1375\"/>\n",
"</g>\n",
"<!-- &#45;7491119705248626972 -->\n",
"<g id=\"node4\" class=\"node\">\n",
"<title>-7491119705248626972</title>\n",
"<polygon fill=\"none\" stroke=\"#000000\" points=\"126,-118.7523 72,-118.7523 72,-82.7523 126,-82.7523 126,-118.7523\"/>\n",
"</g>\n",
"<!-- &#45;7491119705248626972&#45;&gt;3425153061629363430 -->\n",
"<g id=\"edge3\" class=\"edge\">\n",
"<title>-7491119705248626972-&gt;3425153061629363430</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M91.0104,-118.7602C87.243,-127.2518 82.6176,-137.677 78.2192,-147.5906\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"74.9635,-146.2986 74.1072,-156.8588 81.362,-149.1375 74.9635,-146.2986\"/>\n",
"</g>\n",
"<!-- &#45;4753774747094927609 -->\n",
"<g id=\"node5\" class=\"node\">\n",
"<title>-4753774747094927609</title>\n",
"<ellipse fill=\"none\" stroke=\"#000000\" cx=\"99\" cy=\"-23.3762\" rx=\"23.253\" ry=\"23.253\"/>\n",
"<text text-anchor=\"middle\" x=\"99\" y=\"-19.1762\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">inc</text>\n",
"</g>\n",
"<!-- &#45;4753774747094927609&#45;&gt;&#45;7491119705248626972 -->\n",
"<g id=\"edge4\" class=\"edge\">\n",
"<title>-4753774747094927609-&gt;-7491119705248626972</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M99,-47.012C99,-55.0118 99,-64.0288 99,-72.3591\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"95.5001,-72.4152 99,-82.4152 102.5001,-72.4153 95.5001,-72.4152\"/>\n",
"</g>\n",
"<!-- 6397015006435916734 -->\n",
"<g id=\"node6\" class=\"node\">\n",
"<title>6397015006435916734</title>\n",
"<ellipse fill=\"none\" stroke=\"#000000\" cx=\"27\" cy=\"-23.3762\" rx=\"23.253\" ry=\"23.253\"/>\n",
"<text text-anchor=\"middle\" x=\"27\" y=\"-19.1762\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">inc</text>\n",
"</g>\n",
"<!-- 6397015006435916734&#45;&gt;2758571765155845516 -->\n",
"<g id=\"edge5\" class=\"edge\">\n",
"<title>6397015006435916734-&gt;2758571765155845516</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M27,-47.012C27,-55.0118 27,-64.0288 27,-72.3591\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"23.5001,-72.4152 27,-82.4152 30.5001,-72.4153 23.5001,-72.4152\"/>\n",
"</g>\n",
"</g>\n",
"</svg>"
],
"text/plain": [
"<IPython.core.display.SVG object>"
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"c.visualize(format=\"svg\")"
]
},
{
"cell_type": "markdown",
"id": "4c2386a1",
"metadata": {},
"source": [
"Up to this point the object `c` holds all the information we need to compute the result. We can evaluate the result with `.compute()`."
]
},
{
"cell_type": "code",
"execution_count": 7,
"id": "41558b67",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"CPU times: user 280 ms, sys: 197 ms, total: 477 ms\n",
"Wall time: 2.51 s\n"
]
},
{
"data": {
"text/plain": [
"5"
]
},
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"%%time\n",
"\n",
"c.compute()"
]
},
{
"cell_type": "markdown",
"id": "ed58cb96",
"metadata": {},
"source": [
"Notice that now the computation took 2s instead of 3s, this is because the two `inc` computations are run in parallel. "
]
},
{
"cell_type": "markdown",
"id": "556904bb",
"metadata": {},
"source": [
"## Parallelizing a `for`-loop\n",
"\n",
"When we perform the same group of operation multiple times in the form of `for-loop`, there is a chance that we can perform this computations in parallel. For example, the following serial code, can be parallelized using `delayed`: "
]
},
{
"cell_type": "code",
"execution_count": 8,
"id": "cad984eb",
"metadata": {},
"outputs": [],
"source": [
"data = list(range(8))"
]
},
{
"cell_type": "markdown",
"id": "594b7744",
"metadata": {},
"source": [
"#### Sequential code"
]
},
{
"cell_type": "code",
"execution_count": 9,
"id": "44fa16ea",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"CPU times: user 1.45 ms, sys: 501 µs, total: 1.95 ms\n",
"Wall time: 8 s\n"
]
}
],
"source": [
"%%time\n",
"results = []\n",
"for i in data:\n",
" y = inc(i) # do something here\n",
" results.append(y)\n",
" \n",
"total = sum(results) # do something here"
]
},
{
"cell_type": "code",
"execution_count": 10,
"id": "ed5cceae",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"total = 36\n"
]
}
],
"source": [
"print(f'{total = }')"
]
},
{
"cell_type": "markdown",
"id": "8a68dff0",
"metadata": {},
"source": [
"### Exercise 1 \n",
"\n",
"Notice that both the `inc` and `sum` operations can be done in parallel, use `delayed` to parallelize the sequential code above, compute the `total` and time it using `%%time` "
]
},
{
"cell_type": "markdown",
"id": "836ccbe1",
"metadata": {},
"source": [
"Uncomment and run the cell below to see the solution."
]
},
{
"cell_type": "code",
"execution_count": 11,
"id": "47f2ee0b",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"results = []\n",
"for i in data:\n",
" y = delayed(inc)(i) \n",
" results.append(y)\n",
" \n",
"total = delayed(sum)(results)"
]
},
{
"cell_type": "markdown",
"id": "7394312f",
"metadata": {},
"source": [
"In the code above, the `sum` step is not run in parallel, but it depends on each of the `inc` steps, that's why it needs the `delayed` decorator too. The `inc`steps will be parallelized, then aggregated with the `sum` step.\n",
"\n",
"Notice that we can apply delayed to built-in functions, as we did in the case of `sum` in the code above. "
]
},
{
"cell_type": "code",
"execution_count": 12,
"id": "9ab8ec9d",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"Delayed('sum-212abc32-64fb-4d3b-87ab-cb6629ab96b5')"
]
},
"execution_count": 12,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"total"
]
},
{
"cell_type": "code",
"execution_count": 13,
"id": "85ccafbf",
"metadata": {},
"outputs": [
{
"data": {
"image/svg+xml": [
"<svg xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"566pt\" height=\"293pt\" viewBox=\"0.00 0.00 566.00 293.31\">\n",
"<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(4 289.3054)\">\n",
"<title>%3</title>\n",
"<polygon fill=\"#ffffff\" stroke=\"transparent\" points=\"-4,4 -4,-289.3054 562,-289.3054 562,4 -4,4\"/>\n",
"<!-- 5261301441360739609 -->\n",
"<g id=\"node1\" class=\"node\">\n",
"<title>5261301441360739609</title>\n",
"<ellipse fill=\"none\" stroke=\"#000000\" cx=\"279\" cy=\"-184.0288\" rx=\"29.0548\" ry=\"29.0548\"/>\n",
"<text text-anchor=\"middle\" x=\"279\" y=\"-179.8288\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">sum</text>\n",
"</g>\n",
"<!-- &#45;5816541526044055795 -->\n",
"<g id=\"node2\" class=\"node\">\n",
"<title>-5816541526044055795</title>\n",
"<polygon fill=\"none\" stroke=\"#000000\" points=\"306,-285.3054 252,-285.3054 252,-249.3054 306,-249.3054 306,-285.3054\"/>\n",
"</g>\n",
"<!-- 5261301441360739609&#45;&gt;&#45;5816541526044055795 -->\n",
"<g id=\"edge1\" class=\"edge\">\n",
"<title>5261301441360739609-&gt;-5816541526044055795</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M279,-213.6235C279,-221.815 279,-230.6869 279,-238.8142\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"275.5001,-238.9494 279,-248.9495 282.5001,-238.9495 275.5001,-238.9494\"/>\n",
"</g>\n",
"<!-- &#45;8954775949651450911 -->\n",
"<g id=\"node3\" class=\"node\">\n",
"<title>-8954775949651450911</title>\n",
"<polygon fill=\"none\" stroke=\"#000000\" points=\"54,-118.7523 0,-118.7523 0,-82.7523 54,-82.7523 54,-118.7523\"/>\n",
"</g>\n",
"<!-- &#45;8954775949651450911&#45;&gt;5261301441360739609 -->\n",
"<g id=\"edge2\" class=\"edge\">\n",
"<title>-8954775949651450911-&gt;5261301441360739609</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M54.2974,-115.002C57.2048,-116.3385 60.1431,-117.616 63,-118.7523 123.6593,-142.8794 196.8716,-163.2551 240.5917,-174.54\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"239.9611,-177.9913 250.5163,-177.0761 241.6942,-171.2092 239.9611,-177.9913\"/>\n",
"</g>\n",
"<!-- 485668467439289958 -->\n",
"<g id=\"node4\" class=\"node\">\n",
"<title>485668467439289958</title>\n",
"<polygon fill=\"none\" stroke=\"#000000\" points=\"126,-118.7523 72,-118.7523 72,-82.7523 126,-82.7523 126,-118.7523\"/>\n",
"</g>\n",
"<!-- 485668467439289958&#45;&gt;5261301441360739609 -->\n",
"<g id=\"edge3\" class=\"edge\">\n",
"<title>485668467439289958-&gt;5261301441360739609</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M126.0726,-114.4266C129.0768,-115.9029 132.0982,-117.3698 135,-118.7523 171.4003,-136.095 213.4398,-155.0276 242.6618,-168.0153\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"241.4936,-171.3259 252.0539,-172.181 244.3318,-164.9271 241.4936,-171.3259\"/>\n",
"</g>\n",
"<!-- 7048654010339225001 -->\n",
"<g id=\"node5\" class=\"node\">\n",
"<title>7048654010339225001</title>\n",
"<polygon fill=\"none\" stroke=\"#000000\" points=\"198,-118.7523 144,-118.7523 144,-82.7523 198,-82.7523 198,-118.7523\"/>\n",
"</g>\n",
"<!-- 7048654010339225001&#45;&gt;5261301441360739609 -->\n",
"<g id=\"edge4\" class=\"edge\">\n",
"<title>7048654010339225001-&gt;5261301441360739609</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M194.4344,-118.8221C210.0022,-130.8261 230.6235,-146.7268 247.6053,-159.821\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"245.5549,-162.6597 255.6113,-165.9943 249.8294,-157.1163 245.5549,-162.6597\"/>\n",
"</g>\n",
"<!-- &#45;3067789041605449749 -->\n",
"<g id=\"node6\" class=\"node\">\n",
"<title>-3067789041605449749</title>\n",
"<polygon fill=\"none\" stroke=\"#000000\" points=\"270,-118.7523 216,-118.7523 216,-82.7523 270,-82.7523 270,-118.7523\"/>\n",
"</g>\n",
"<!-- &#45;3067789041605449749&#45;&gt;5261301441360739609 -->\n",
"<g id=\"edge5\" class=\"edge\">\n",
"<title>-3067789041605449749-&gt;5261301441360739609</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M250.8115,-118.8221C254.4785,-127.3048 258.9868,-137.7335 263.3137,-147.7427\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"260.1924,-149.3431 267.3732,-157.1333 266.6178,-146.5654 260.1924,-149.3431\"/>\n",
"</g>\n",
"<!-- &#45;1868123491625956628 -->\n",
"<g id=\"node7\" class=\"node\">\n",
"<title>-1868123491625956628</title>\n",
"<polygon fill=\"none\" stroke=\"#000000\" points=\"342,-118.7523 288,-118.7523 288,-82.7523 342,-82.7523 342,-118.7523\"/>\n",
"</g>\n",
"<!-- &#45;1868123491625956628&#45;&gt;5261301441360739609 -->\n",
"<g id=\"edge6\" class=\"edge\">\n",
"<title>-1868123491625956628-&gt;5261301441360739609</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M307.1885,-118.8221C303.5215,-127.3048 299.0132,-137.7335 294.6863,-147.7427\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"291.3822,-146.5654 290.6268,-157.1333 297.8076,-149.3431 291.3822,-146.5654\"/>\n",
"</g>\n",
"<!-- 4308633083496618061 -->\n",
"<g id=\"node8\" class=\"node\">\n",
"<title>4308633083496618061</title>\n",
"<polygon fill=\"none\" stroke=\"#000000\" points=\"414,-118.7523 360,-118.7523 360,-82.7523 414,-82.7523 414,-118.7523\"/>\n",
"</g>\n",
"<!-- 4308633083496618061&#45;&gt;5261301441360739609 -->\n",
"<g id=\"edge7\" class=\"edge\">\n",
"<title>4308633083496618061-&gt;5261301441360739609</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M363.5656,-118.8221C347.9978,-130.8261 327.3765,-146.7268 310.3947,-159.821\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"308.1706,-157.1163 302.3887,-165.9943 312.4451,-162.6597 308.1706,-157.1163\"/>\n",
"</g>\n",
"<!-- 960874615402565618 -->\n",
"<g id=\"node9\" class=\"node\">\n",
"<title>960874615402565618</title>\n",
"<polygon fill=\"none\" stroke=\"#000000\" points=\"486,-118.7523 432,-118.7523 432,-82.7523 486,-82.7523 486,-118.7523\"/>\n",
"</g>\n",
"<!-- 960874615402565618&#45;&gt;5261301441360739609 -->\n",
"<g id=\"edge8\" class=\"edge\">\n",
"<title>960874615402565618-&gt;5261301441360739609</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M431.9274,-114.4266C428.9232,-115.9029 425.9018,-117.3698 423,-118.7523 386.5997,-136.095 344.5602,-155.0276 315.3382,-168.0153\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"313.6682,-164.9271 305.9461,-172.181 316.5064,-171.3259 313.6682,-164.9271\"/>\n",
"</g>\n",
"<!-- &#45;2248432844476801828 -->\n",
"<g id=\"node10\" class=\"node\">\n",
"<title>-2248432844476801828</title>\n",
"<polygon fill=\"none\" stroke=\"#000000\" points=\"558,-118.7523 504,-118.7523 504,-82.7523 558,-82.7523 558,-118.7523\"/>\n",
"</g>\n",
"<!-- &#45;2248432844476801828&#45;&gt;5261301441360739609 -->\n",
"<g id=\"edge9\" class=\"edge\">\n",
"<title>-2248432844476801828-&gt;5261301441360739609</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M503.7026,-115.002C500.7952,-116.3385 497.8569,-117.616 495,-118.7523 434.3407,-142.8794 361.1284,-163.2551 317.4083,-174.54\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"316.3058,-171.2092 307.4837,-177.0761 318.0389,-177.9913 316.3058,-171.2092\"/>\n",
"</g>\n",
"<!-- 3987629320146452180 -->\n",
"<g id=\"node11\" class=\"node\">\n",
"<title>3987629320146452180</title>\n",
"<ellipse fill=\"none\" stroke=\"#000000\" cx=\"243\" cy=\"-23.3762\" rx=\"23.253\" ry=\"23.253\"/>\n",
"<text text-anchor=\"middle\" x=\"243\" y=\"-19.1762\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">inc</text>\n",
"</g>\n",
"<!-- 3987629320146452180&#45;&gt;&#45;3067789041605449749 -->\n",
"<g id=\"edge10\" class=\"edge\">\n",
"<title>3987629320146452180-&gt;-3067789041605449749</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M243,-47.012C243,-55.0118 243,-64.0288 243,-72.3591\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"239.5001,-72.4152 243,-82.4152 246.5001,-72.4153 239.5001,-72.4152\"/>\n",
"</g>\n",
"<!-- &#45;6485520676882700270 -->\n",
"<g id=\"node12\" class=\"node\">\n",
"<title>-6485520676882700270</title>\n",
"<ellipse fill=\"none\" stroke=\"#000000\" cx=\"27\" cy=\"-23.3762\" rx=\"23.253\" ry=\"23.253\"/>\n",
"<text text-anchor=\"middle\" x=\"27\" y=\"-19.1762\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">inc</text>\n",
"</g>\n",
"<!-- &#45;6485520676882700270&#45;&gt;&#45;8954775949651450911 -->\n",
"<g id=\"edge11\" class=\"edge\">\n",
"<title>-6485520676882700270-&gt;-8954775949651450911</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M27,-47.012C27,-55.0118 27,-64.0288 27,-72.3591\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"23.5001,-72.4152 27,-82.4152 30.5001,-72.4153 23.5001,-72.4152\"/>\n",
"</g>\n",
"<!-- &#45;5765705297715155945 -->\n",
"<g id=\"node13\" class=\"node\">\n",
"<title>-5765705297715155945</title>\n",
"<ellipse fill=\"none\" stroke=\"#000000\" cx=\"99\" cy=\"-23.3762\" rx=\"23.253\" ry=\"23.253\"/>\n",
"<text text-anchor=\"middle\" x=\"99\" y=\"-19.1762\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">inc</text>\n",
"</g>\n",
"<!-- &#45;5765705297715155945&#45;&gt;485668467439289958 -->\n",
"<g id=\"edge12\" class=\"edge\">\n",
"<title>-5765705297715155945-&gt;485668467439289958</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M99,-47.012C99,-55.0118 99,-64.0288 99,-72.3591\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"95.5001,-72.4152 99,-82.4152 102.5001,-72.4153 95.5001,-72.4152\"/>\n",
"</g>\n",
"<!-- &#45;4457940680233048560 -->\n",
"<g id=\"node14\" class=\"node\">\n",
"<title>-4457940680233048560</title>\n",
"<ellipse fill=\"none\" stroke=\"#000000\" cx=\"171\" cy=\"-23.3762\" rx=\"23.253\" ry=\"23.253\"/>\n",
"<text text-anchor=\"middle\" x=\"171\" y=\"-19.1762\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">inc</text>\n",
"</g>\n",
"<!-- &#45;4457940680233048560&#45;&gt;7048654010339225001 -->\n",
"<g id=\"edge13\" class=\"edge\">\n",
"<title>-4457940680233048560-&gt;7048654010339225001</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M171,-47.012C171,-55.0118 171,-64.0288 171,-72.3591\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"167.5001,-72.4152 171,-82.4152 174.5001,-72.4153 167.5001,-72.4152\"/>\n",
"</g>\n",
"<!-- &#45;3315071298719144832 -->\n",
"<g id=\"node15\" class=\"node\">\n",
"<title>-3315071298719144832</title>\n",
"<ellipse fill=\"none\" stroke=\"#000000\" cx=\"315\" cy=\"-23.3762\" rx=\"23.253\" ry=\"23.253\"/>\n",
"<text text-anchor=\"middle\" x=\"315\" y=\"-19.1762\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">inc</text>\n",
"</g>\n",
"<!-- &#45;3315071298719144832&#45;&gt;&#45;1868123491625956628 -->\n",
"<g id=\"edge14\" class=\"edge\">\n",
"<title>-3315071298719144832-&gt;-1868123491625956628</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M315,-47.012C315,-55.0118 315,-64.0288 315,-72.3591\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"311.5001,-72.4152 315,-82.4152 318.5001,-72.4153 311.5001,-72.4152\"/>\n",
"</g>\n",
"<!-- &#45;8500677254425202961 -->\n",
"<g id=\"node16\" class=\"node\">\n",
"<title>-8500677254425202961</title>\n",
"<ellipse fill=\"none\" stroke=\"#000000\" cx=\"387\" cy=\"-23.3762\" rx=\"23.253\" ry=\"23.253\"/>\n",
"<text text-anchor=\"middle\" x=\"387\" y=\"-19.1762\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">inc</text>\n",
"</g>\n",
"<!-- &#45;8500677254425202961&#45;&gt;4308633083496618061 -->\n",
"<g id=\"edge15\" class=\"edge\">\n",
"<title>-8500677254425202961-&gt;4308633083496618061</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M387,-47.012C387,-55.0118 387,-64.0288 387,-72.3591\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"383.5001,-72.4152 387,-82.4152 390.5001,-72.4153 383.5001,-72.4152\"/>\n",
"</g>\n",
"<!-- 7838036694978058625 -->\n",
"<g id=\"node17\" class=\"node\">\n",
"<title>7838036694978058625</title>\n",
"<ellipse fill=\"none\" stroke=\"#000000\" cx=\"531\" cy=\"-23.3762\" rx=\"23.253\" ry=\"23.253\"/>\n",
"<text text-anchor=\"middle\" x=\"531\" y=\"-19.1762\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">inc</text>\n",
"</g>\n",
"<!-- 7838036694978058625&#45;&gt;&#45;2248432844476801828 -->\n",
"<g id=\"edge16\" class=\"edge\">\n",
"<title>7838036694978058625-&gt;-2248432844476801828</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M531,-47.012C531,-55.0118 531,-64.0288 531,-72.3591\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"527.5001,-72.4152 531,-82.4152 534.5001,-72.4153 527.5001,-72.4152\"/>\n",
"</g>\n",
"<!-- &#45;8376360177206427750 -->\n",
"<g id=\"node18\" class=\"node\">\n",
"<title>-8376360177206427750</title>\n",
"<ellipse fill=\"none\" stroke=\"#000000\" cx=\"459\" cy=\"-23.3762\" rx=\"23.253\" ry=\"23.253\"/>\n",
"<text text-anchor=\"middle\" x=\"459\" y=\"-19.1762\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">inc</text>\n",
"</g>\n",
"<!-- &#45;8376360177206427750&#45;&gt;960874615402565618 -->\n",
"<g id=\"edge17\" class=\"edge\">\n",
"<title>-8376360177206427750-&gt;960874615402565618</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M459,-47.012C459,-55.0118 459,-64.0288 459,-72.3591\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"455.5001,-72.4152 459,-82.4152 462.5001,-72.4153 455.5001,-72.4152\"/>\n",
"</g>\n",
"</g>\n",
"</svg>"
],
"text/plain": [
"<IPython.core.display.SVG object>"
]
},
"execution_count": 13,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"total.visualize(format=\"svg\")"
]
},
{
"cell_type": "code",
"execution_count": 14,
"id": "bd495500",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"CPU times: user 3.15 ms, sys: 1.13 ms, total: 4.28 ms\n",
"Wall time: 4 s\n"
]
},
{
"data": {
"text/plain": [
"36"
]
},
"execution_count": 14,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"%%time\n",
"total.compute()"
]
},
{
"cell_type": "markdown",
"id": "72ab18d7",
"metadata": {},
"source": [
"### The `@delayed` syntax \n",
"\n",
"The `delayed` decorator can be also used by \"decorating\" with `@delayed` the function you want to parallelize."
]
},
{
"cell_type": "code",
"execution_count": 15,
"id": "1faf2ba1",
"metadata": {},
"outputs": [],
"source": [
"@delayed \n",
"def double(x):\n",
" \"\"\"Decrease x by one\"\"\"\n",
" sleep(1)\n",
" return 2*x "
]
},
{
"cell_type": "markdown",
"id": "409f9557",
"metadata": {},
"source": [
"Then when we call this new `double` function we obtain a delayed object:"
]
},
{
"cell_type": "code",
"execution_count": 16,
"id": "4501eeda",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Delayed('double-5065cb2c-c4f9-4b01-8a5b-28a3b5e33b4a')\n"
]
}
],
"source": [
"d = double(4)\n",
"print(d)"
]
},
{
"cell_type": "code",
"execution_count": 17,
"id": "1a313693",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"CPU times: user 2.19 ms, sys: 0 ns, total: 2.19 ms\n",
"Wall time: 1.01 s\n"
]
},
{
"data": {
"text/plain": [
"8"
]
},
"execution_count": 17,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"%%time\n",
"d.compute()"
]
},
{
"cell_type": "markdown",
"id": "24a13213",
"metadata": {},
"source": [
"### Exercise 2\n",
"\n",
"Using the `delayed` decorator create the parallel versions of `inc` and `add`\n",
"\n",
"```python\n",
"def inc(x):\n",
" \"\"\"Increments x by one\"\"\"\n",
" sleep(1)\n",
" return x + 1\n",
"\n",
"def add(x, y):\n",
" \"\"\"Adds x and y\"\"\"\n",
" sleep(1)\n",
" return x + y\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": 18,
"id": "7d2847d2",
"metadata": {
"jupyter": {
"source_hidden": true
},
"tags": []
},
"outputs": [],
"source": [
"@delayed\n",
"def inc(x):\n",
" \"\"\"Increments x by one\"\"\"\n",
" sleep(1)\n",
" return x + 1\n",
"\n",
"@delayed\n",
"def add(x, y):\n",
" \"\"\"Adds x and y\"\"\"\n",
" sleep(1)\n",
" return x + y"
]
},
{
"cell_type": "markdown",
"id": "4a12256b",
"metadata": {},
"source": [
"``Delayed`` objects support several standard Python operations, each of which creates another ``Delayed`` object representing the result:\n",
"\n",
"- Arithmetic operators, e.g. `*`, `-`, `+`\n",
"- Item access and slicing, e.g. `x[0]`, `x[1:3]`\n",
"- Attribute access, e.g. `x.size`\n",
"- Method calls, e.g. `x.index(0)`\n",
"\n",
"For example you can do:"
]
},
{
"cell_type": "code",
"execution_count": 19,
"id": "e5ed731f",
"metadata": {},
"outputs": [
{
"data": {
"image/svg+xml": [
"<svg xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"278pt\" height=\"450pt\" viewBox=\"0.00 0.00 278.00 450.22\">\n",
"<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(4 446.217)\">\n",
"<title>%3</title>\n",
"<polygon fill=\"#ffffff\" stroke=\"transparent\" points=\"-4,4 -4,-446.217 274,-446.217 274,4 -4,4\"/>\n",
"<!-- &#45;5396752035593474589 -->\n",
"<g id=\"node1\" class=\"node\">\n",
"<title>-5396752035593474589</title>\n",
"<ellipse fill=\"none\" stroke=\"#000000\" cx=\"135\" cy=\"-343.0756\" rx=\"27.2835\" ry=\"27.2835\"/>\n",
"<text text-anchor=\"middle\" x=\"135\" y=\"-338.8756\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">add</text>\n",
"</g>\n",
"<!-- 3843805722671841820 -->\n",
"<g id=\"node2\" class=\"node\">\n",
"<title>3843805722671841820</title>\n",
"<polygon fill=\"none\" stroke=\"#000000\" points=\"162,-442.217 108,-442.217 108,-406.217 162,-406.217 162,-442.217\"/>\n",
"</g>\n",
"<!-- &#45;5396752035593474589&#45;&gt;3843805722671841820 -->\n",
"<g id=\"edge1\" class=\"edge\">\n",
"<title>-5396752035593474589-&gt;3843805722671841820</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M135,-370.5464C135,-378.7301 135,-387.7105 135,-395.9445\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"131.5001,-396.2091 135,-406.2091 138.5001,-396.2091 131.5001,-396.2091\"/>\n",
"</g>\n",
"<!-- &#45;682748415356192292 -->\n",
"<g id=\"node3\" class=\"node\">\n",
"<title>-682748415356192292</title>\n",
"<polygon fill=\"none\" stroke=\"#000000\" points=\"126,-279.9342 72,-279.9342 72,-243.9342 126,-243.9342 126,-279.9342\"/>\n",
"</g>\n",
"<!-- &#45;682748415356192292&#45;&gt;&#45;5396752035593474589 -->\n",
"<g id=\"edge2\" class=\"edge\">\n",
"<title>-682748415356192292-&gt;-5396752035593474589</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M106.9896,-279.9421C110.757,-288.4337 115.3824,-298.8589 119.7808,-308.7725\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"116.638,-310.3194 123.8928,-318.0407 123.0365,-307.4805 116.638,-310.3194\"/>\n",
"</g>\n",
"<!-- &#45;5269959232102492266 -->\n",
"<g id=\"node4\" class=\"node\">\n",
"<title>-5269959232102492266</title>\n",
"<polygon fill=\"none\" stroke=\"#000000\" points=\"198,-279.9342 144,-279.9342 144,-243.9342 198,-243.9342 198,-279.9342\"/>\n",
"</g>\n",
"<!-- &#45;5269959232102492266&#45;&gt;&#45;5396752035593474589 -->\n",
"<g id=\"edge3\" class=\"edge\">\n",
"<title>-5269959232102492266-&gt;-5396752035593474589</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M163.0104,-279.9421C159.243,-288.4337 154.6176,-298.8589 150.2192,-308.7725\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"146.9635,-307.4805 146.1072,-318.0407 153.362,-310.3194 146.9635,-307.4805\"/>\n",
"</g>\n",
"<!-- 1611918149223275519 -->\n",
"<g id=\"node5\" class=\"node\">\n",
"<title>1611918149223275519</title>\n",
"<ellipse fill=\"none\" stroke=\"#000000\" cx=\"171\" cy=\"-181.3433\" rx=\"26.6822\" ry=\"26.6822\"/>\n",
"<text text-anchor=\"middle\" x=\"171\" y=\"-177.1433\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">mul</text>\n",
"</g>\n",
"<!-- 1611918149223275519&#45;&gt;&#45;5269959232102492266 -->\n",
"<g id=\"edge4\" class=\"edge\">\n",
"<title>1611918149223275519-&gt;-5269959232102492266</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M171,-208.1789C171,-216.4209 171,-225.5117 171,-233.8358\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"167.5001,-233.848 171,-243.848 174.5001,-233.8481 167.5001,-233.848\"/>\n",
"</g>\n",
"<!-- 692342779377178138 -->\n",
"<g id=\"node6\" class=\"node\">\n",
"<title>692342779377178138</title>\n",
"<polygon fill=\"none\" stroke=\"#000000\" points=\"198,-118.7523 144,-118.7523 144,-82.7523 198,-82.7523 198,-118.7523\"/>\n",
"</g>\n",
"<!-- 692342779377178138&#45;&gt;1611918149223275519 -->\n",
"<g id=\"edge5\" class=\"edge\">\n",
"<title>692342779377178138-&gt;1611918149223275519</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M171,-119.0396C171,-126.5834 171,-135.6116 171,-144.423\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"167.5001,-144.5653 171,-154.5654 174.5001,-144.5654 167.5001,-144.5653\"/>\n",
"</g>\n",
"<!-- 7936578953574285430 -->\n",
"<g id=\"node7\" class=\"node\">\n",
"<title>7936578953574285430</title>\n",
"<polygon fill=\"none\" stroke=\"#000000\" points=\"270,-118.7523 216,-118.7523 216,-82.7523 270,-82.7523 270,-118.7523\"/>\n",
"</g>\n",
"<!-- 7936578953574285430&#45;&gt;1611918149223275519 -->\n",
"<g id=\"edge6\" class=\"edge\">\n",
"<title>7936578953574285430-&gt;1611918149223275519</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M226.6621,-119.0396C217.4927,-129.3031 205.8685,-142.3143 195.617,-153.789\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"192.8386,-151.6456 188.7862,-161.4348 198.0587,-156.3093 192.8386,-151.6456\"/>\n",
"</g>\n",
"<!-- &#45;5556042075465988853 -->\n",
"<g id=\"node8\" class=\"node\">\n",
"<title>-5556042075465988853</title>\n",
"<ellipse fill=\"none\" stroke=\"#000000\" cx=\"171\" cy=\"-23.3762\" rx=\"23.253\" ry=\"23.253\"/>\n",
"<text text-anchor=\"middle\" x=\"171\" y=\"-19.1762\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">inc</text>\n",
"</g>\n",
"<!-- &#45;5556042075465988853&#45;&gt;692342779377178138 -->\n",
"<g id=\"edge7\" class=\"edge\">\n",
"<title>-5556042075465988853-&gt;692342779377178138</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M171,-47.012C171,-55.0118 171,-64.0288 171,-72.3591\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"167.5001,-72.4152 171,-82.4152 174.5001,-72.4153 167.5001,-72.4152\"/>\n",
"</g>\n",
"<!-- &#45;702554000153505667 -->\n",
"<g id=\"node9\" class=\"node\">\n",
"<title>-702554000153505667</title>\n",
"<ellipse fill=\"none\" stroke=\"#000000\" cx=\"243\" cy=\"-23.3762\" rx=\"23.253\" ry=\"23.253\"/>\n",
"<text text-anchor=\"middle\" x=\"243\" y=\"-19.1762\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">inc</text>\n",
"</g>\n",
"<!-- &#45;702554000153505667&#45;&gt;7936578953574285430 -->\n",
"<g id=\"edge8\" class=\"edge\">\n",
"<title>-702554000153505667-&gt;7936578953574285430</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M243,-47.012C243,-55.0118 243,-64.0288 243,-72.3591\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"239.5001,-72.4152 243,-82.4152 246.5001,-72.4153 239.5001,-72.4152\"/>\n",
"</g>\n",
"<!-- &#45;7195393178264714921 -->\n",
"<g id=\"node10\" class=\"node\">\n",
"<title>-7195393178264714921</title>\n",
"<ellipse fill=\"none\" stroke=\"#000000\" cx=\"99\" cy=\"-181.3433\" rx=\"26.6822\" ry=\"26.6822\"/>\n",
"<text text-anchor=\"middle\" x=\"99\" y=\"-177.1433\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">mul</text>\n",
"</g>\n",
"<!-- &#45;7195393178264714921&#45;&gt;&#45;682748415356192292 -->\n",
"<g id=\"edge9\" class=\"edge\">\n",
"<title>-7195393178264714921-&gt;-682748415356192292</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M99,-208.1789C99,-216.4209 99,-225.5117 99,-233.8358\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"95.5001,-233.848 99,-243.848 102.5001,-233.8481 95.5001,-233.848\"/>\n",
"</g>\n",
"<!-- 7487076418952249282 -->\n",
"<g id=\"node11\" class=\"node\">\n",
"<title>7487076418952249282</title>\n",
"<polygon fill=\"none\" stroke=\"#000000\" points=\"54,-118.7523 0,-118.7523 0,-82.7523 54,-82.7523 54,-118.7523\"/>\n",
"</g>\n",
"<!-- 7487076418952249282&#45;&gt;&#45;7195393178264714921 -->\n",
"<g id=\"edge10\" class=\"edge\">\n",
"<title>7487076418952249282-&gt;-7195393178264714921</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M43.3379,-119.0396C52.5073,-129.3031 64.1315,-142.3143 74.383,-153.789\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"71.9413,-156.3093 81.2138,-161.4348 77.1614,-151.6456 71.9413,-156.3093\"/>\n",
"</g>\n",
"<!-- 376264532275344309 -->\n",
"<g id=\"node12\" class=\"node\">\n",
"<title>376264532275344309</title>\n",
"<polygon fill=\"none\" stroke=\"#000000\" points=\"126,-118.7523 72,-118.7523 72,-82.7523 126,-82.7523 126,-118.7523\"/>\n",
"</g>\n",
"<!-- 376264532275344309&#45;&gt;&#45;7195393178264714921 -->\n",
"<g id=\"edge11\" class=\"edge\">\n",
"<title>376264532275344309-&gt;-7195393178264714921</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M99,-119.0396C99,-126.5834 99,-135.6116 99,-144.423\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"95.5001,-144.5653 99,-154.5654 102.5001,-144.5654 95.5001,-144.5653\"/>\n",
"</g>\n",
"<!-- &#45;4564546017710553651 -->\n",
"<g id=\"node13\" class=\"node\">\n",
"<title>-4564546017710553651</title>\n",
"<ellipse fill=\"none\" stroke=\"#000000\" cx=\"99\" cy=\"-23.3762\" rx=\"23.253\" ry=\"23.253\"/>\n",
"<text text-anchor=\"middle\" x=\"99\" y=\"-19.1762\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">inc</text>\n",
"</g>\n",
"<!-- &#45;4564546017710553651&#45;&gt;376264532275344309 -->\n",
"<g id=\"edge12\" class=\"edge\">\n",
"<title>-4564546017710553651-&gt;376264532275344309</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M99,-47.012C99,-55.0118 99,-64.0288 99,-72.3591\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"95.5001,-72.4152 99,-82.4152 102.5001,-72.4153 95.5001,-72.4152\"/>\n",
"</g>\n",
"<!-- &#45;4613384065635414969 -->\n",
"<g id=\"node14\" class=\"node\">\n",
"<title>-4613384065635414969</title>\n",
"<ellipse fill=\"none\" stroke=\"#000000\" cx=\"27\" cy=\"-23.3762\" rx=\"23.253\" ry=\"23.253\"/>\n",
"<text text-anchor=\"middle\" x=\"27\" y=\"-19.1762\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">inc</text>\n",
"</g>\n",
"<!-- &#45;4613384065635414969&#45;&gt;7487076418952249282 -->\n",
"<g id=\"edge13\" class=\"edge\">\n",
"<title>-4613384065635414969-&gt;7487076418952249282</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M27,-47.012C27,-55.0118 27,-64.0288 27,-72.3591\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"23.5001,-72.4152 27,-82.4152 30.5001,-72.4153 23.5001,-72.4152\"/>\n",
"</g>\n",
"</g>\n",
"</svg>"
],
"text/plain": [
"<IPython.core.display.SVG object>"
]
},
"execution_count": 19,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"result = (inc(5) * inc(7)) + (inc(3) * inc(2))\n",
"result.visualize(format=\"svg\")"
]
},
{
"cell_type": "code",
"execution_count": 20,
"id": "cacd838b",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"CPU times: user 22.8 ms, sys: 9.3 ms, total: 32.1 ms\n",
"Wall time: 2 s\n"
]
},
{
"data": {
"text/plain": [
"60"
]
},
"execution_count": 20,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"%%time\n",
"result.compute()"
]
},
{
"cell_type": "markdown",
"id": "d1ca4b69",
"metadata": {},
"source": [
"## Another for-loop example "
]
},
{
"cell_type": "markdown",
"id": "58dcb089",
"metadata": {},
"source": [
"Let's say we want to perform some operations like `inc`, `double` and `add` on a list of data, and finally aggregate all the results. We can use our `delayed` decorated functions to perform this computations faster. \n",
"The serial version of the code below would take approximately 24 seconds, let's see how long does the parallel version takes:"
]
},
{
"cell_type": "code",
"execution_count": 21,
"id": "7b6b24b1",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"Delayed('sum-209f4ef7-25a3-4f55-a54b-e7e0cbef1675')"
]
},
"execution_count": 21,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"data = list(range(8))\n",
"\n",
"output = []\n",
"for x in data:\n",
" a = inc(x) #parallel version\n",
" b = double(x) #parallel version\n",
" c = add(a, b) #parallel version\n",
" output.append(c)\n",
"\n",
"total = delayed(sum)(output)\n",
"total"
]
},
{
"cell_type": "markdown",
"id": "3e12ae2b",
"metadata": {},
"source": [
"Noticed that `inc`, `double` and `add` in the code above are already the parallel versions, since we decorated with `@delayed`"
]
},
{
"cell_type": "code",
"execution_count": 22,
"id": "e0f2d3e5",
"metadata": {},
"outputs": [
{
"data": {
"image/svg+xml": [
"<svg xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"1287pt\" height=\"489pt\" viewBox=\"0.00 0.00 1287.40 488.88\">\n",
"<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(4 484.8776)\">\n",
"<title>%3</title>\n",
"<polygon fill=\"#ffffff\" stroke=\"transparent\" points=\"-4,4 -4,-484.8776 1283.397,-484.8776 1283.397,4 -4,4\"/>\n",
"<!-- &#45;3337329094717189066 -->\n",
"<g id=\"node1\" class=\"node\">\n",
"<title>-3337329094717189066</title>\n",
"<ellipse fill=\"none\" stroke=\"#000000\" cx=\"632.3762\" cy=\"-379.6011\" rx=\"29.0548\" ry=\"29.0548\"/>\n",
"<text text-anchor=\"middle\" x=\"632.3762\" y=\"-375.4011\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">sum</text>\n",
"</g>\n",
"<!-- 4931417350112528636 -->\n",
"<g id=\"node2\" class=\"node\">\n",
"<title>4931417350112528636</title>\n",
"<polygon fill=\"none\" stroke=\"#000000\" points=\"659.3762,-480.8776 605.3762,-480.8776 605.3762,-444.8776 659.3762,-444.8776 659.3762,-480.8776\"/>\n",
"</g>\n",
"<!-- &#45;3337329094717189066&#45;&gt;4931417350112528636 -->\n",
"<g id=\"edge1\" class=\"edge\">\n",
"<title>-3337329094717189066-&gt;4931417350112528636</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M632.3762,-409.1957C632.3762,-417.3872 632.3762,-426.2592 632.3762,-434.3864\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"628.8763,-434.5217 632.3762,-444.5217 635.8763,-434.5217 628.8763,-434.5217\"/>\n",
"</g>\n",
"<!-- &#45;1204816254996573728 -->\n",
"<g id=\"node3\" class=\"node\">\n",
"<title>-1204816254996573728</title>\n",
"<polygon fill=\"none\" stroke=\"#000000\" points=\"169.3762,-314.3245 115.3762,-314.3245 115.3762,-278.3245 169.3762,-278.3245 169.3762,-314.3245\"/>\n",
"</g>\n",
"<!-- &#45;1204816254996573728&#45;&gt;&#45;3337329094717189066 -->\n",
"<g id=\"edge2\" class=\"edge\">\n",
"<title>-1204816254996573728-&gt;-3337329094717189066</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M169.4186,-300.9204C251.3988,-314.8531 497.3915,-356.6601 593.3958,-372.9763\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"592.8325,-376.4306 603.2776,-374.6557 594.0054,-369.5296 592.8325,-376.4306\"/>\n",
"</g>\n",
"<!-- &#45;7011943705079010742 -->\n",
"<g id=\"node4\" class=\"node\">\n",
"<title>-7011943705079010742</title>\n",
"<polygon fill=\"none\" stroke=\"#000000\" points=\"376.3762,-314.3245 322.3762,-314.3245 322.3762,-278.3245 376.3762,-278.3245 376.3762,-314.3245\"/>\n",
"</g>\n",
"<!-- &#45;7011943705079010742&#45;&gt;&#45;3337329094717189066 -->\n",
"<g id=\"edge3\" class=\"edge\">\n",
"<title>-7011943705079010742-&gt;-3337329094717189066</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M376.5114,-304.3094C427.1489,-319.2102 536.15,-351.2852 594.1885,-368.3638\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"593.4865,-371.8056 604.0678,-371.2709 595.4626,-365.0903 593.4865,-371.8056\"/>\n",
"</g>\n",
"<!-- &#45;2365601346067850963 -->\n",
"<g id=\"node5\" class=\"node\">\n",
"<title>-2365601346067850963</title>\n",
"<polygon fill=\"none\" stroke=\"#000000\" points=\"515.3762,-314.3245 461.3762,-314.3245 461.3762,-278.3245 515.3762,-278.3245 515.3762,-314.3245\"/>\n",
"</g>\n",
"<!-- &#45;2365601346067850963&#45;&gt;&#45;3337329094717189066 -->\n",
"<g id=\"edge4\" class=\"edge\">\n",
"<title>-2365601346067850963-&gt;-3337329094717189066</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M515.4615,-311.9882C538.8259,-325.5001 572.6967,-345.0879 597.974,-359.706\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"596.4536,-362.8698 606.8625,-364.8463 599.958,-356.8102 596.4536,-362.8698\"/>\n",
"</g>\n",
"<!-- &#45;1179691361049595533 -->\n",
"<g id=\"node6\" class=\"node\">\n",
"<title>-1179691361049595533</title>\n",
"<polygon fill=\"none\" stroke=\"#000000\" points=\"621.3762,-314.3245 567.3762,-314.3245 567.3762,-278.3245 621.3762,-278.3245 621.3762,-314.3245\"/>\n",
"</g>\n",
"<!-- &#45;1179691361049595533&#45;&gt;&#45;3337329094717189066 -->\n",
"<g id=\"edge5\" class=\"edge\">\n",
"<title>-1179691361049595533-&gt;-3337329094717189066</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M602.6216,-314.3943C606.4924,-322.877 611.2511,-333.3057 615.8184,-343.3149\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"612.7679,-345.0609 620.1034,-352.7055 619.1362,-342.1549 612.7679,-345.0609\"/>\n",
"</g>\n",
"<!-- &#45;7972377348068524168 -->\n",
"<g id=\"node7\" class=\"node\">\n",
"<title>-7972377348068524168</title>\n",
"<polygon fill=\"none\" stroke=\"#000000\" points=\"696.3762,-314.3245 642.3762,-314.3245 642.3762,-278.3245 696.3762,-278.3245 696.3762,-314.3245\"/>\n",
"</g>\n",
"<!-- &#45;7972377348068524168&#45;&gt;&#45;3337329094717189066 -->\n",
"<g id=\"edge6\" class=\"edge\">\n",
"<title>-7972377348068524168-&gt;-3337329094717189066</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M661.3477,-314.3943C657.5788,-322.877 652.9453,-333.3057 648.4982,-343.3149\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"645.1878,-342.1458 644.3259,-352.7055 651.5848,-344.988 645.1878,-342.1458\"/>\n",
"</g>\n",
"<!-- 2589336623913312223 -->\n",
"<g id=\"node8\" class=\"node\">\n",
"<title>2589336623913312223</title>\n",
"<polygon fill=\"none\" stroke=\"#000000\" points=\"814.3762,-314.3245 760.3762,-314.3245 760.3762,-278.3245 814.3762,-278.3245 814.3762,-314.3245\"/>\n",
"</g>\n",
"<!-- 2589336623913312223&#45;&gt;&#45;3337329094717189066 -->\n",
"<g id=\"edge7\" class=\"edge\">\n",
"<title>2589336623913312223-&gt;-3337329094717189066</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M760.0221,-311.021C734.1768,-324.9068 695.2923,-345.7982 667.2347,-360.8727\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"665.3217,-357.9273 658.1691,-365.7433 668.6347,-364.0936 665.3217,-357.9273\"/>\n",
"</g>\n",
"<!-- 4141995095793171198 -->\n",
"<g id=\"node9\" class=\"node\">\n",
"<title>4141995095793171198</title>\n",
"<polygon fill=\"none\" stroke=\"#000000\" points=\"963.3762,-314.3245 909.3762,-314.3245 909.3762,-278.3245 963.3762,-278.3245 963.3762,-314.3245\"/>\n",
"</g>\n",
"<!-- 4141995095793171198&#45;&gt;&#45;3337329094717189066 -->\n",
"<g id=\"edge8\" class=\"edge\">\n",
"<title>4141995095793171198-&gt;-3337329094717189066</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M909.1306,-303.788C854.8765,-318.6502 733.2056,-351.9802 670.8383,-369.0649\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"669.7154,-365.7435 660.9955,-371.7612 671.5649,-372.4947 669.7154,-365.7435\"/>\n",
"</g>\n",
"<!-- &#45;2365057590157724193 -->\n",
"<g id=\"node10\" class=\"node\">\n",
"<title>-2365057590157724193</title>\n",
"<polygon fill=\"none\" stroke=\"#000000\" points=\"1072.3762,-314.3245 1018.3762,-314.3245 1018.3762,-278.3245 1072.3762,-278.3245 1072.3762,-314.3245\"/>\n",
"</g>\n",
"<!-- &#45;2365057590157724193&#45;&gt;&#45;3337329094717189066 -->\n",
"<g id=\"edge9\" class=\"edge\">\n",
"<title>-2365057590157724193-&gt;-3337329094717189066</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M1018.0694,-303.4155C1004.3901,-306.8764 987.5494,-310.9996 972.3762,-314.3245 863.9238,-338.0893 734.9634,-361.5444 671.5541,-372.7639\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"670.8283,-369.3378 661.5884,-374.5218 672.0444,-376.2313 670.8283,-369.3378\"/>\n",
"</g>\n",
"<!-- 3894598345026229320 -->\n",
"<g id=\"node11\" class=\"node\">\n",
"<title>3894598345026229320</title>\n",
"<ellipse fill=\"none\" stroke=\"#000000\" cx=\"590.3762\" cy=\"-215.1831\" rx=\"27.2835\" ry=\"27.2835\"/>\n",
"<text text-anchor=\"middle\" x=\"590.3762\" y=\"-210.9831\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">add</text>\n",
"</g>\n",
"<!-- 3894598345026229320&#45;&gt;&#45;1179691361049595533 -->\n",
"<g id=\"edge10\" class=\"edge\">\n",
"<title>3894598345026229320-&gt;-1179691361049595533</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M591.7304,-242.6539C592.1338,-250.8376 592.5765,-259.818 592.9824,-268.052\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"589.5002,-268.5011 593.4884,-278.3166 596.4917,-268.1564 589.5002,-268.5011\"/>\n",
"</g>\n",
"<!-- &#45;5926879604182365696 -->\n",
"<g id=\"node12\" class=\"node\">\n",
"<title>-5926879604182365696</title>\n",
"<polygon fill=\"none\" stroke=\"#000000\" points=\"540.3762,-152.0417 486.3762,-152.0417 486.3762,-116.0417 540.3762,-116.0417 540.3762,-152.0417\"/>\n",
"</g>\n",
"<!-- &#45;5926879604182365696&#45;&gt;3894598345026229320 -->\n",
"<g id=\"edge11\" class=\"edge\">\n",
"<title>-5926879604182365696-&gt;3894598345026229320</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M530.4649,-152.0496C540.4235,-162.5438 553.1844,-175.991 564.3681,-187.7762\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"562.0219,-190.3885 571.4443,-195.233 567.0996,-185.57 562.0219,-190.3885\"/>\n",
"</g>\n",
"<!-- 4542243777661968322 -->\n",
"<g id=\"node13\" class=\"node\">\n",
"<title>4542243777661968322</title>\n",
"<polygon fill=\"none\" stroke=\"#000000\" points=\"617.3762,-152.0417 563.3762,-152.0417 563.3762,-116.0417 617.3762,-116.0417 617.3762,-152.0417\"/>\n",
"</g>\n",
"<!-- 4542243777661968322&#45;&gt;3894598345026229320 -->\n",
"<g id=\"edge12\" class=\"edge\">\n",
"<title>4542243777661968322-&gt;3894598345026229320</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M590.3762,-152.0496C590.3762,-159.6337 590.3762,-168.7602 590.3762,-177.687\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"586.8762,-177.9702 590.3762,-187.9703 593.8762,-177.9703 586.8762,-177.9702\"/>\n",
"</g>\n",
"<!-- &#45;6658501972735111299 -->\n",
"<g id=\"node14\" class=\"node\">\n",
"<title>-6658501972735111299</title>\n",
"<ellipse fill=\"none\" stroke=\"#000000\" cx=\"509.3762\" cy=\"-40.0209\" rx=\"23.253\" ry=\"23.253\"/>\n",
"<text text-anchor=\"middle\" x=\"509.3762\" y=\"-35.8209\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">inc</text>\n",
"</g>\n",
"<!-- &#45;6658501972735111299&#45;&gt;&#45;5926879604182365696 -->\n",
"<g id=\"edge13\" class=\"edge\">\n",
"<title>-6658501972735111299-&gt;-5926879604182365696</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M510.3855,-63.7463C510.9297,-76.538 511.6049,-92.4069 512.1759,-105.8286\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"508.685,-106.1201 512.607,-115.9623 515.6787,-105.8225 508.685,-106.1201\"/>\n",
"</g>\n",
"<!-- &#45;5805331072419779438 -->\n",
"<g id=\"node15\" class=\"node\">\n",
"<title>-5805331072419779438</title>\n",
"<ellipse fill=\"none\" stroke=\"#000000\" cx=\"590.3762\" cy=\"-40.0209\" rx=\"40.0417\" ry=\"40.0417\"/>\n",
"<text text-anchor=\"middle\" x=\"590.3762\" y=\"-35.8209\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">double</text>\n",
"</g>\n",
"<!-- &#45;5805331072419779438&#45;&gt;4542243777661968322 -->\n",
"<g id=\"edge14\" class=\"edge\">\n",
"<title>-5805331072419779438-&gt;4542243777661968322</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M590.3762,-80.1669C590.3762,-88.6768 590.3762,-97.4933 590.3762,-105.4687\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"586.8762,-105.7148 590.3762,-115.7148 593.8762,-105.7148 586.8762,-105.7148\"/>\n",
"</g>\n",
"<!-- 1953088039107852858 -->\n",
"<g id=\"node16\" class=\"node\">\n",
"<title>1953088039107852858</title>\n",
"<ellipse fill=\"none\" stroke=\"#000000\" cx=\"1158.3762\" cy=\"-215.1831\" rx=\"27.2835\" ry=\"27.2835\"/>\n",
"<text text-anchor=\"middle\" x=\"1158.3762\" y=\"-210.9831\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">add</text>\n",
"</g>\n",
"<!-- 1953088039107852858&#45;&gt;&#45;2365057590157724193 -->\n",
"<g id=\"edge15\" class=\"edge\">\n",
"<title>1953088039107852858-&gt;-2365057590157724193</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M1136.3204,-231.0206C1119.9451,-242.7791 1097.3276,-259.02 1078.7451,-272.3635\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"1076.6093,-269.5881 1070.528,-278.2639 1080.6922,-275.2741 1076.6093,-269.5881\"/>\n",
"</g>\n",
"<!-- 2501776079920602568 -->\n",
"<g id=\"node17\" class=\"node\">\n",
"<title>2501776079920602568</title>\n",
"<polygon fill=\"none\" stroke=\"#000000\" points=\"1185.3762,-152.0417 1131.3762,-152.0417 1131.3762,-116.0417 1185.3762,-116.0417 1185.3762,-152.0417\"/>\n",
"</g>\n",
"<!-- 2501776079920602568&#45;&gt;1953088039107852858 -->\n",
"<g id=\"edge16\" class=\"edge\">\n",
"<title>2501776079920602568-&gt;1953088039107852858</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M1158.3762,-152.0496C1158.3762,-159.6337 1158.3762,-168.7602 1158.3762,-177.687\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"1154.8762,-177.9702 1158.3762,-187.9703 1161.8762,-177.9703 1154.8762,-177.9702\"/>\n",
"</g>\n",
"<!-- &#45;800983515101727410 -->\n",
"<g id=\"node18\" class=\"node\">\n",
"<title>-800983515101727410</title>\n",
"<polygon fill=\"none\" stroke=\"#000000\" points=\"1261.3762,-152.0417 1207.3762,-152.0417 1207.3762,-116.0417 1261.3762,-116.0417 1261.3762,-152.0417\"/>\n",
"</g>\n",
"<!-- &#45;800983515101727410&#45;&gt;1953088039107852858 -->\n",
"<g id=\"edge17\" class=\"edge\">\n",
"<title>-800983515101727410-&gt;1953088039107852858</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M1217.5093,-152.0496C1207.6801,-162.5438 1195.0849,-175.991 1184.0464,-187.7762\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"1181.3437,-185.5419 1177.0621,-195.233 1186.4527,-190.3271 1181.3437,-185.5419\"/>\n",
"</g>\n",
"<!-- 6070412558838979907 -->\n",
"<g id=\"node19\" class=\"node\">\n",
"<title>6070412558838979907</title>\n",
"<ellipse fill=\"none\" stroke=\"#000000\" cx=\"1158.3762\" cy=\"-40.0209\" rx=\"23.253\" ry=\"23.253\"/>\n",
"<text text-anchor=\"middle\" x=\"1158.3762\" y=\"-35.8209\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">inc</text>\n",
"</g>\n",
"<!-- 6070412558838979907&#45;&gt;2501776079920602568 -->\n",
"<g id=\"edge18\" class=\"edge\">\n",
"<title>6070412558838979907-&gt;2501776079920602568</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M1158.3762,-63.7463C1158.3762,-76.538 1158.3762,-92.4069 1158.3762,-105.8286\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"1154.8763,-105.9622 1158.3762,-115.9623 1161.8763,-105.9623 1154.8763,-105.9622\"/>\n",
"</g>\n",
"<!-- 1865208193528400739 -->\n",
"<g id=\"node20\" class=\"node\">\n",
"<title>1865208193528400739</title>\n",
"<ellipse fill=\"none\" stroke=\"#000000\" cx=\"1239.3762\" cy=\"-40.0209\" rx=\"40.0417\" ry=\"40.0417\"/>\n",
"<text text-anchor=\"middle\" x=\"1239.3762\" y=\"-35.8209\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">double</text>\n",
"</g>\n",
"<!-- 1865208193528400739&#45;&gt;&#45;800983515101727410 -->\n",
"<g id=\"edge19\" class=\"edge\">\n",
"<title>1865208193528400739-&gt;-800983515101727410</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M1237.2412,-80.1669C1236.7886,-88.6768 1236.3198,-97.4933 1235.8957,-105.4687\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"1232.3868,-105.543 1235.3508,-115.7148 1239.377,-105.9148 1232.3868,-105.543\"/>\n",
"</g>\n",
"<!-- 2826333909955140359 -->\n",
"<g id=\"node21\" class=\"node\">\n",
"<title>2826333909955140359</title>\n",
"<ellipse fill=\"none\" stroke=\"#000000\" cx=\"266.3762\" cy=\"-215.1831\" rx=\"27.2835\" ry=\"27.2835\"/>\n",
"<text text-anchor=\"middle\" x=\"266.3762\" y=\"-210.9831\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">add</text>\n",
"</g>\n",
"<!-- 2826333909955140359&#45;&gt;&#45;7011943705079010742 -->\n",
"<g id=\"edge20\" class=\"edge\">\n",
"<title>2826333909955140359-&gt;-7011943705079010742</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M286.046,-234.4125C297.241,-245.3568 311.3989,-259.1977 323.4878,-271.0158\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"321.1051,-273.5811 330.7025,-278.069 325.9985,-268.5756 321.1051,-273.5811\"/>\n",
"</g>\n",
"<!-- 352255841873122587 -->\n",
"<g id=\"node22\" class=\"node\">\n",
"<title>352255841873122587</title>\n",
"<polygon fill=\"none\" stroke=\"#000000\" points=\"216.3762,-152.0417 162.3762,-152.0417 162.3762,-116.0417 216.3762,-116.0417 216.3762,-152.0417\"/>\n",
"</g>\n",
"<!-- 352255841873122587&#45;&gt;2826333909955140359 -->\n",
"<g id=\"edge21\" class=\"edge\">\n",
"<title>352255841873122587-&gt;2826333909955140359</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M206.4649,-152.0496C216.4235,-162.5438 229.1844,-175.991 240.3681,-187.7762\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"238.0219,-190.3885 247.4443,-195.233 243.0996,-185.57 238.0219,-190.3885\"/>\n",
"</g>\n",
"<!-- 4991529381192323350 -->\n",
"<g id=\"node23\" class=\"node\">\n",
"<title>4991529381192323350</title>\n",
"<polygon fill=\"none\" stroke=\"#000000\" points=\"293.3762,-152.0417 239.3762,-152.0417 239.3762,-116.0417 293.3762,-116.0417 293.3762,-152.0417\"/>\n",
"</g>\n",
"<!-- 4991529381192323350&#45;&gt;2826333909955140359 -->\n",
"<g id=\"edge22\" class=\"edge\">\n",
"<title>4991529381192323350-&gt;2826333909955140359</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M266.3762,-152.0496C266.3762,-159.6337 266.3762,-168.7602 266.3762,-177.687\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"262.8762,-177.9702 266.3762,-187.9703 269.8762,-177.9703 262.8762,-177.9702\"/>\n",
"</g>\n",
"<!-- 2173199344526537320 -->\n",
"<g id=\"node24\" class=\"node\">\n",
"<title>2173199344526537320</title>\n",
"<ellipse fill=\"none\" stroke=\"#000000\" cx=\"185.3762\" cy=\"-40.0209\" rx=\"23.253\" ry=\"23.253\"/>\n",
"<text text-anchor=\"middle\" x=\"185.3762\" y=\"-35.8209\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">inc</text>\n",
"</g>\n",
"<!-- 2173199344526537320&#45;&gt;352255841873122587 -->\n",
"<g id=\"edge23\" class=\"edge\">\n",
"<title>2173199344526537320-&gt;352255841873122587</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M186.3855,-63.7463C186.9297,-76.538 187.6049,-92.4069 188.1759,-105.8286\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"184.685,-106.1201 188.607,-115.9623 191.6787,-105.8225 184.685,-106.1201\"/>\n",
"</g>\n",
"<!-- &#45;8232929685752262434 -->\n",
"<g id=\"node25\" class=\"node\">\n",
"<title>-8232929685752262434</title>\n",
"<ellipse fill=\"none\" stroke=\"#000000\" cx=\"266.3762\" cy=\"-40.0209\" rx=\"40.0417\" ry=\"40.0417\"/>\n",
"<text text-anchor=\"middle\" x=\"266.3762\" y=\"-35.8209\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">double</text>\n",
"</g>\n",
"<!-- &#45;8232929685752262434&#45;&gt;4991529381192323350 -->\n",
"<g id=\"edge24\" class=\"edge\">\n",
"<title>-8232929685752262434-&gt;4991529381192323350</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M266.3762,-80.1669C266.3762,-88.6768 266.3762,-97.4933 266.3762,-105.4687\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"262.8762,-105.7148 266.3762,-115.7148 269.8762,-105.7148 262.8762,-105.7148\"/>\n",
"</g>\n",
"<!-- 876146845608260714 -->\n",
"<g id=\"node26\" class=\"node\">\n",
"<title>876146845608260714</title>\n",
"<ellipse fill=\"none\" stroke=\"#000000\" cx=\"1012.3762\" cy=\"-215.1831\" rx=\"27.2835\" ry=\"27.2835\"/>\n",
"<text text-anchor=\"middle\" x=\"1012.3762\" y=\"-210.9831\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">add</text>\n",
"</g>\n",
"<!-- 876146845608260714&#45;&gt;4141995095793171198 -->\n",
"<g id=\"edge25\" class=\"edge\">\n",
"<title>876146845608260714-&gt;4141995095793171198</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M993.5896,-235.2405C983.5895,-245.9172 971.1803,-259.1659 960.4773,-270.593\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"957.6722,-268.4678 953.3906,-278.159 962.7812,-273.2531 957.6722,-268.4678\"/>\n",
"</g>\n",
"<!-- &#45;6236950212767681374 -->\n",
"<g id=\"node27\" class=\"node\">\n",
"<title>-6236950212767681374</title>\n",
"<polygon fill=\"none\" stroke=\"#000000\" points=\"1039.3762,-152.0417 985.3762,-152.0417 985.3762,-116.0417 1039.3762,-116.0417 1039.3762,-152.0417\"/>\n",
"</g>\n",
"<!-- &#45;6236950212767681374&#45;&gt;876146845608260714 -->\n",
"<g id=\"edge26\" class=\"edge\">\n",
"<title>-6236950212767681374-&gt;876146845608260714</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M1012.3762,-152.0496C1012.3762,-159.6337 1012.3762,-168.7602 1012.3762,-177.687\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"1008.8762,-177.9702 1012.3762,-187.9703 1015.8762,-177.9703 1008.8762,-177.9702\"/>\n",
"</g>\n",
"<!-- &#45;5509088482219438291 -->\n",
"<g id=\"node28\" class=\"node\">\n",
"<title>-5509088482219438291</title>\n",
"<polygon fill=\"none\" stroke=\"#000000\" points=\"1112.3762,-152.0417 1058.3762,-152.0417 1058.3762,-116.0417 1112.3762,-116.0417 1112.3762,-152.0417\"/>\n",
"</g>\n",
"<!-- &#45;5509088482219438291&#45;&gt;876146845608260714 -->\n",
"<g id=\"edge27\" class=\"edge\">\n",
"<title>-5509088482219438291-&gt;876146845608260714</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M1069.1751,-152.0496C1059.8921,-162.368 1048.0405,-175.5413 1037.5673,-187.1825\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"1034.6717,-185.168 1030.5853,-194.9431 1039.8756,-189.8498 1034.6717,-185.168\"/>\n",
"</g>\n",
"<!-- &#45;1602778503065735963 -->\n",
"<g id=\"node29\" class=\"node\">\n",
"<title>-1602778503065735963</title>\n",
"<ellipse fill=\"none\" stroke=\"#000000\" cx=\"1093.3762\" cy=\"-40.0209\" rx=\"23.253\" ry=\"23.253\"/>\n",
"<text text-anchor=\"middle\" x=\"1093.3762\" y=\"-35.8209\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">inc</text>\n",
"</g>\n",
"<!-- &#45;1602778503065735963&#45;&gt;&#45;5509088482219438291 -->\n",
"<g id=\"edge28\" class=\"edge\">\n",
"<title>-1602778503065735963-&gt;-5509088482219438291</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M1091.3574,-63.7463C1090.269,-76.538 1088.9188,-92.4069 1087.7767,-105.8286\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"1084.275,-105.7015 1086.9145,-115.9623 1091.2498,-106.295 1084.275,-105.7015\"/>\n",
"</g>\n",
"<!-- &#45;1065207319948964455 -->\n",
"<g id=\"node30\" class=\"node\">\n",
"<title>-1065207319948964455</title>\n",
"<ellipse fill=\"none\" stroke=\"#000000\" cx=\"1012.3762\" cy=\"-40.0209\" rx=\"40.0417\" ry=\"40.0417\"/>\n",
"<text text-anchor=\"middle\" x=\"1012.3762\" y=\"-35.8209\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">double</text>\n",
"</g>\n",
"<!-- &#45;1065207319948964455&#45;&gt;&#45;6236950212767681374 -->\n",
"<g id=\"edge29\" class=\"edge\">\n",
"<title>-1065207319948964455-&gt;-6236950212767681374</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M1012.3762,-80.1669C1012.3762,-88.6768 1012.3762,-97.4933 1012.3762,-105.4687\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"1008.8762,-105.7148 1012.3762,-115.7148 1015.8762,-105.7148 1008.8762,-105.7148\"/>\n",
"</g>\n",
"<!-- 2378268501127185724 -->\n",
"<g id=\"node31\" class=\"node\">\n",
"<title>2378268501127185724</title>\n",
"<ellipse fill=\"none\" stroke=\"#000000\" cx=\"104.3762\" cy=\"-215.1831\" rx=\"27.2835\" ry=\"27.2835\"/>\n",
"<text text-anchor=\"middle\" x=\"104.3762\" y=\"-210.9831\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">add</text>\n",
"</g>\n",
"<!-- 2378268501127185724&#45;&gt;&#45;1204816254996573728 -->\n",
"<g id=\"edge30\" class=\"edge\">\n",
"<title>2378268501127185724-&gt;-1204816254996573728</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M115.9839,-239.9691C120.2605,-249.1009 125.1291,-259.4968 129.5102,-268.8519\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"126.4433,-270.5557 133.8541,-278.1274 132.7826,-267.5869 126.4433,-270.5557\"/>\n",
"</g>\n",
"<!-- 4451604137708666884 -->\n",
"<g id=\"node32\" class=\"node\">\n",
"<title>4451604137708666884</title>\n",
"<polygon fill=\"none\" stroke=\"#000000\" points=\"55.3762,-152.0417 1.3762,-152.0417 1.3762,-116.0417 55.3762,-116.0417 55.3762,-152.0417\"/>\n",
"</g>\n",
"<!-- 4451604137708666884&#45;&gt;2378268501127185724 -->\n",
"<g id=\"edge31\" class=\"edge\">\n",
"<title>4451604137708666884-&gt;2378268501127185724</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M45.243,-152.0496C55.0722,-162.5438 67.6674,-175.991 78.7059,-187.7762\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"76.2996,-190.3271 85.6902,-195.233 81.4086,-185.5419 76.2996,-190.3271\"/>\n",
"</g>\n",
"<!-- 1654974966482612270 -->\n",
"<g id=\"node33\" class=\"node\">\n",
"<title>1654974966482612270</title>\n",
"<polygon fill=\"none\" stroke=\"#000000\" points=\"131.3762,-152.0417 77.3762,-152.0417 77.3762,-116.0417 131.3762,-116.0417 131.3762,-152.0417\"/>\n",
"</g>\n",
"<!-- 1654974966482612270&#45;&gt;2378268501127185724 -->\n",
"<g id=\"edge32\" class=\"edge\">\n",
"<title>1654974966482612270-&gt;2378268501127185724</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M104.3762,-152.0496C104.3762,-159.6337 104.3762,-168.7602 104.3762,-177.687\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"100.8762,-177.9702 104.3762,-187.9703 107.8762,-177.9703 100.8762,-177.9702\"/>\n",
"</g>\n",
"<!-- &#45;4598232661024917459 -->\n",
"<g id=\"node34\" class=\"node\">\n",
"<title>-4598232661024917459</title>\n",
"<ellipse fill=\"none\" stroke=\"#000000\" cx=\"23.3762\" cy=\"-40.0209\" rx=\"23.253\" ry=\"23.253\"/>\n",
"<text text-anchor=\"middle\" x=\"23.3762\" y=\"-35.8209\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">inc</text>\n",
"</g>\n",
"<!-- &#45;4598232661024917459&#45;&gt;4451604137708666884 -->\n",
"<g id=\"edge33\" class=\"edge\">\n",
"<title>-4598232661024917459-&gt;4451604137708666884</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M24.6379,-63.7463C25.3181,-76.538 26.162,-92.4069 26.8758,-105.8286\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"23.3885,-106.1623 27.4147,-115.9623 30.3786,-105.7905 23.3885,-106.1623\"/>\n",
"</g>\n",
"<!-- &#45;2316752338848272319 -->\n",
"<g id=\"node35\" class=\"node\">\n",
"<title>-2316752338848272319</title>\n",
"<ellipse fill=\"none\" stroke=\"#000000\" cx=\"104.3762\" cy=\"-40.0209\" rx=\"40.0417\" ry=\"40.0417\"/>\n",
"<text text-anchor=\"middle\" x=\"104.3762\" y=\"-35.8209\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">double</text>\n",
"</g>\n",
"<!-- &#45;2316752338848272319&#45;&gt;1654974966482612270 -->\n",
"<g id=\"edge34\" class=\"edge\">\n",
"<title>-2316752338848272319-&gt;1654974966482612270</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M104.3762,-80.1669C104.3762,-88.6768 104.3762,-97.4933 104.3762,-105.4687\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"100.8762,-105.7148 104.3762,-115.7148 107.8762,-105.7148 100.8762,-105.7148\"/>\n",
"</g>\n",
"<!-- &#45;6991554486989835896 -->\n",
"<g id=\"node36\" class=\"node\">\n",
"<title>-6991554486989835896</title>\n",
"<ellipse fill=\"none\" stroke=\"#000000\" cx=\"671.3762\" cy=\"-215.1831\" rx=\"27.2835\" ry=\"27.2835\"/>\n",
"<text text-anchor=\"middle\" x=\"671.3762\" y=\"-210.9831\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">add</text>\n",
"</g>\n",
"<!-- &#45;6991554486989835896&#45;&gt;&#45;7972377348068524168 -->\n",
"<g id=\"edge35\" class=\"edge\">\n",
"<title>-6991554486989835896-&gt;-7972377348068524168</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M670.699,-242.6539C670.4973,-250.8376 670.276,-259.818 670.073,-268.052\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"666.5676,-268.2334 669.82,-278.3166 673.5655,-268.4059 666.5676,-268.2334\"/>\n",
"</g>\n",
"<!-- 475514313699713208 -->\n",
"<g id=\"node37\" class=\"node\">\n",
"<title>475514313699713208</title>\n",
"<polygon fill=\"none\" stroke=\"#000000\" points=\"698.3762,-152.0417 644.3762,-152.0417 644.3762,-116.0417 698.3762,-116.0417 698.3762,-152.0417\"/>\n",
"</g>\n",
"<!-- 475514313699713208&#45;&gt;&#45;6991554486989835896 -->\n",
"<g id=\"edge36\" class=\"edge\">\n",
"<title>475514313699713208-&gt;-6991554486989835896</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M671.3762,-152.0496C671.3762,-159.6337 671.3762,-168.7602 671.3762,-177.687\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"667.8762,-177.9702 671.3762,-187.9703 674.8762,-177.9703 667.8762,-177.9702\"/>\n",
"</g>\n",
"<!-- 1022643533905243033 -->\n",
"<g id=\"node38\" class=\"node\">\n",
"<title>1022643533905243033</title>\n",
"<polygon fill=\"none\" stroke=\"#000000\" points=\"775.3762,-152.0417 721.3762,-152.0417 721.3762,-116.0417 775.3762,-116.0417 775.3762,-152.0417\"/>\n",
"</g>\n",
"<!-- 1022643533905243033&#45;&gt;&#45;6991554486989835896 -->\n",
"<g id=\"edge37\" class=\"edge\">\n",
"<title>1022643533905243033-&gt;-6991554486989835896</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M731.2874,-152.0496C721.3288,-162.5438 708.5679,-175.991 697.3842,-187.7762\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"694.6527,-185.57 690.308,-195.233 699.7304,-190.3885 694.6527,-185.57\"/>\n",
"</g>\n",
"<!-- 1881197833997340504 -->\n",
"<g id=\"node39\" class=\"node\">\n",
"<title>1881197833997340504</title>\n",
"<ellipse fill=\"none\" stroke=\"#000000\" cx=\"671.3762\" cy=\"-40.0209\" rx=\"23.253\" ry=\"23.253\"/>\n",
"<text text-anchor=\"middle\" x=\"671.3762\" y=\"-35.8209\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">inc</text>\n",
"</g>\n",
"<!-- 1881197833997340504&#45;&gt;475514313699713208 -->\n",
"<g id=\"edge38\" class=\"edge\">\n",
"<title>1881197833997340504-&gt;475514313699713208</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M671.3762,-63.7463C671.3762,-76.538 671.3762,-92.4069 671.3762,-105.8286\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"667.8763,-105.9622 671.3762,-115.9623 674.8763,-105.9623 667.8763,-105.9622\"/>\n",
"</g>\n",
"<!-- 7877877054425398760 -->\n",
"<g id=\"node40\" class=\"node\">\n",
"<title>7877877054425398760</title>\n",
"<ellipse fill=\"none\" stroke=\"#000000\" cx=\"752.3762\" cy=\"-40.0209\" rx=\"40.0417\" ry=\"40.0417\"/>\n",
"<text text-anchor=\"middle\" x=\"752.3762\" y=\"-35.8209\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">double</text>\n",
"</g>\n",
"<!-- 7877877054425398760&#45;&gt;1022643533905243033 -->\n",
"<g id=\"edge39\" class=\"edge\">\n",
"<title>7877877054425398760-&gt;1022643533905243033</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M750.6682,-80.1669C750.3061,-88.6768 749.9311,-97.4933 749.5918,-105.4687\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"746.0842,-105.5751 749.1558,-115.7148 753.0778,-105.8727 746.0842,-105.5751\"/>\n",
"</g>\n",
"<!-- 5184250992179802999 -->\n",
"<g id=\"node41\" class=\"node\">\n",
"<title>5184250992179802999</title>\n",
"<ellipse fill=\"none\" stroke=\"#000000\" cx=\"833.3762\" cy=\"-215.1831\" rx=\"27.2835\" ry=\"27.2835\"/>\n",
"<text text-anchor=\"middle\" x=\"833.3762\" y=\"-210.9831\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">add</text>\n",
"</g>\n",
"<!-- 5184250992179802999&#45;&gt;2589336623913312223 -->\n",
"<g id=\"edge40\" class=\"edge\">\n",
"<title>5184250992179802999-&gt;2589336623913312223</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M819.8234,-239.0893C814.4411,-248.5835 808.2233,-259.5512 802.6798,-269.3297\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"799.5078,-267.8282 797.6208,-278.2536 805.5973,-271.2804 799.5078,-267.8282\"/>\n",
"</g>\n",
"<!-- 6007852228271299656 -->\n",
"<g id=\"node42\" class=\"node\">\n",
"<title>6007852228271299656</title>\n",
"<polygon fill=\"none\" stroke=\"#000000\" points=\"860.3762,-152.0417 806.3762,-152.0417 806.3762,-116.0417 860.3762,-116.0417 860.3762,-152.0417\"/>\n",
"</g>\n",
"<!-- 6007852228271299656&#45;&gt;5184250992179802999 -->\n",
"<g id=\"edge41\" class=\"edge\">\n",
"<title>6007852228271299656-&gt;5184250992179802999</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M833.3762,-152.0496C833.3762,-159.6337 833.3762,-168.7602 833.3762,-177.687\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"829.8762,-177.9702 833.3762,-187.9703 836.8762,-177.9703 829.8762,-177.9702\"/>\n",
"</g>\n",
"<!-- &#45;966884809769286588 -->\n",
"<g id=\"node43\" class=\"node\">\n",
"<title>-966884809769286588</title>\n",
"<polygon fill=\"none\" stroke=\"#000000\" points=\"937.3762,-152.0417 883.3762,-152.0417 883.3762,-116.0417 937.3762,-116.0417 937.3762,-152.0417\"/>\n",
"</g>\n",
"<!-- &#45;966884809769286588&#45;&gt;5184250992179802999 -->\n",
"<g id=\"edge42\" class=\"edge\">\n",
"<title>-966884809769286588-&gt;5184250992179802999</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M893.2874,-152.0496C883.3288,-162.5438 870.5679,-175.991 859.3842,-187.7762\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"856.6527,-185.57 852.308,-195.233 861.7304,-190.3885 856.6527,-185.57\"/>\n",
"</g>\n",
"<!-- &#45;4386444781234331556 -->\n",
"<g id=\"node44\" class=\"node\">\n",
"<title>-4386444781234331556</title>\n",
"<ellipse fill=\"none\" stroke=\"#000000\" cx=\"833.3762\" cy=\"-40.0209\" rx=\"23.253\" ry=\"23.253\"/>\n",
"<text text-anchor=\"middle\" x=\"833.3762\" y=\"-35.8209\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">inc</text>\n",
"</g>\n",
"<!-- &#45;4386444781234331556&#45;&gt;6007852228271299656 -->\n",
"<g id=\"edge43\" class=\"edge\">\n",
"<title>-4386444781234331556-&gt;6007852228271299656</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M833.3762,-63.7463C833.3762,-76.538 833.3762,-92.4069 833.3762,-105.8286\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"829.8763,-105.9622 833.3762,-115.9623 836.8763,-105.9623 829.8763,-105.9622\"/>\n",
"</g>\n",
"<!-- 7412290374691392822 -->\n",
"<g id=\"node45\" class=\"node\">\n",
"<title>7412290374691392822</title>\n",
"<ellipse fill=\"none\" stroke=\"#000000\" cx=\"914.3762\" cy=\"-40.0209\" rx=\"40.0417\" ry=\"40.0417\"/>\n",
"<text text-anchor=\"middle\" x=\"914.3762\" y=\"-35.8209\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">double</text>\n",
"</g>\n",
"<!-- 7412290374691392822&#45;&gt;&#45;966884809769286588 -->\n",
"<g id=\"edge44\" class=\"edge\">\n",
"<title>7412290374691392822-&gt;-966884809769286588</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M912.6682,-80.1669C912.3061,-88.6768 911.9311,-97.4933 911.5918,-105.4687\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"908.0842,-105.5751 911.1558,-115.7148 915.0778,-105.8727 908.0842,-105.5751\"/>\n",
"</g>\n",
"<!-- &#45;5509448600172329461 -->\n",
"<g id=\"node46\" class=\"node\">\n",
"<title>-5509448600172329461</title>\n",
"<ellipse fill=\"none\" stroke=\"#000000\" cx=\"428.3762\" cy=\"-215.1831\" rx=\"27.2835\" ry=\"27.2835\"/>\n",
"<text text-anchor=\"middle\" x=\"428.3762\" y=\"-210.9831\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">add</text>\n",
"</g>\n",
"<!-- &#45;5509448600172329461&#45;&gt;&#45;2365601346067850963 -->\n",
"<g id=\"edge45\" class=\"edge\">\n",
"<title>-5509448600172329461-&gt;-2365601346067850963</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M444.7714,-237.3554C452.273,-247.5002 461.2145,-259.5923 469.0422,-270.1781\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"466.2725,-272.3192 475.0323,-278.2788 471.9008,-268.1573 466.2725,-272.3192\"/>\n",
"</g>\n",
"<!-- 8785975155597233764 -->\n",
"<g id=\"node47\" class=\"node\">\n",
"<title>8785975155597233764</title>\n",
"<polygon fill=\"none\" stroke=\"#000000\" points=\"378.3762,-152.0417 324.3762,-152.0417 324.3762,-116.0417 378.3762,-116.0417 378.3762,-152.0417\"/>\n",
"</g>\n",
"<!-- 8785975155597233764&#45;&gt;&#45;5509448600172329461 -->\n",
"<g id=\"edge46\" class=\"edge\">\n",
"<title>8785975155597233764-&gt;-5509448600172329461</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M368.4649,-152.0496C378.4235,-162.5438 391.1844,-175.991 402.3681,-187.7762\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"400.0219,-190.3885 409.4443,-195.233 405.0996,-185.57 400.0219,-190.3885\"/>\n",
"</g>\n",
"<!-- &#45;8881968756213575315 -->\n",
"<g id=\"node48\" class=\"node\">\n",
"<title>-8881968756213575315</title>\n",
"<polygon fill=\"none\" stroke=\"#000000\" points=\"455.3762,-152.0417 401.3762,-152.0417 401.3762,-116.0417 455.3762,-116.0417 455.3762,-152.0417\"/>\n",
"</g>\n",
"<!-- &#45;8881968756213575315&#45;&gt;&#45;5509448600172329461 -->\n",
"<g id=\"edge47\" class=\"edge\">\n",
"<title>-8881968756213575315-&gt;-5509448600172329461</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M428.3762,-152.0496C428.3762,-159.6337 428.3762,-168.7602 428.3762,-177.687\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"424.8762,-177.9702 428.3762,-187.9703 431.8762,-177.9703 424.8762,-177.9702\"/>\n",
"</g>\n",
"<!-- &#45;4898384029214312305 -->\n",
"<g id=\"node49\" class=\"node\">\n",
"<title>-4898384029214312305</title>\n",
"<ellipse fill=\"none\" stroke=\"#000000\" cx=\"347.3762\" cy=\"-40.0209\" rx=\"23.253\" ry=\"23.253\"/>\n",
"<text text-anchor=\"middle\" x=\"347.3762\" y=\"-35.8209\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">inc</text>\n",
"</g>\n",
"<!-- &#45;4898384029214312305&#45;&gt;8785975155597233764 -->\n",
"<g id=\"edge48\" class=\"edge\">\n",
"<title>-4898384029214312305-&gt;8785975155597233764</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M348.3855,-63.7463C348.9297,-76.538 349.6049,-92.4069 350.1759,-105.8286\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"346.685,-106.1201 350.607,-115.9623 353.6787,-105.8225 346.685,-106.1201\"/>\n",
"</g>\n",
"<!-- 4139284163288868517 -->\n",
"<g id=\"node50\" class=\"node\">\n",
"<title>4139284163288868517</title>\n",
"<ellipse fill=\"none\" stroke=\"#000000\" cx=\"428.3762\" cy=\"-40.0209\" rx=\"40.0417\" ry=\"40.0417\"/>\n",
"<text text-anchor=\"middle\" x=\"428.3762\" y=\"-35.8209\" font-family=\"Helvetica,sans-Serif\" font-size=\"14.00\" fill=\"#000000\">double</text>\n",
"</g>\n",
"<!-- 4139284163288868517&#45;&gt;&#45;8881968756213575315 -->\n",
"<g id=\"edge49\" class=\"edge\">\n",
"<title>4139284163288868517-&gt;-8881968756213575315</title>\n",
"<path fill=\"none\" stroke=\"#000000\" d=\"M428.3762,-80.1669C428.3762,-88.6768 428.3762,-97.4933 428.3762,-105.4687\"/>\n",
"<polygon fill=\"#000000\" stroke=\"#000000\" points=\"424.8762,-105.7148 428.3762,-115.7148 431.8762,-105.7148 424.8762,-105.7148\"/>\n",
"</g>\n",
"</g>\n",
"</svg>"
],
"text/plain": [
"<IPython.core.display.SVG object>"
]
},
"execution_count": 22,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"total.visualize(format=\"svg\")"
]
},
{
"cell_type": "markdown",
"id": "57f38fc4",
"metadata": {},
"source": [
"### Exercise: How long will this task graph take to compute on a machine with 2 cores?"
]
},
{
"cell_type": "code",
"execution_count": 23,
"id": "f80ffe62",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"CPU times: user 19.9 ms, sys: 12.6 ms, total: 32.6 ms\n",
"Wall time: 13 s\n"
]
},
{
"data": {
"text/plain": [
"92"
]
},
"execution_count": 23,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"%%time\n",
"total.compute()"
]
},
{
"cell_type": "markdown",
"id": "f2ead99e",
"metadata": {},
"source": [
"## Extra resources\n",
"\n",
"For more examples on `dask.delayed` check:\n",
"- Main Dask tutorial: [Delayed lesson](https://github.com/dask/dask-tutorial/blob/main/01_dask.delayed.ipynb)\n",
"- More examples on Delayed: [PyData global - Dask tutorial - Delayed](https://github.com/coiled/pydata-global-dask/blob/master/1-delayed.ipynb)\n",
"- Short screencast on Dask delayed: [How to parallelize Python code with Dask Delayed (3min)](https://www.youtube.com/watch?v=-EUlNJI2QYs)\n",
"- [Dask Delayed documentation](https://docs.dask.org/en/latest/delayed.html)\n",
"- [Delayed Best Practices](https://docs.dask.org/en/latest/delayed-best-practices.html)\n"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.5"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
Display the source blob
Display the rendered blob
Raw
05_dask_gateway.ipynb
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Display the source blob
Display the rendered blob
Raw
06_array_distributed.ipynb
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Display the source blob
Display the rendered blob
Raw
07_zarr_object_store.ipynb
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment