Skip to content

Instantly share code, notes, and snippets.

@mrocklin

mrocklin/cluster-logs.ipynb

Created July 26, 2019 18:42
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save mrocklin/b350ed9462963f0df784544fd2e07fd9 to your computer and use it in GitHub Desktop.
Save mrocklin/b350ed9462963f0df784544fd2e07fd9 to your computer and use it in GitHub Desktop.
Download ZIP
Display the source blob
Display the rendered blob
Raw
cluster-logs.ipynb
{
"cells": [
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<details><summary>Scheduler</summary><pre><code>distributed.scheduler - INFO - Clear task state\n",
"distributed.scheduler - INFO - Scheduler at: tcp://127.0.0.1:63657\n",
"distributed.scheduler - INFO - dashboard at: 127.0.0.1:8787\n",
"distributed.scheduler - INFO - Register tcp://127.0.0.1:63662\n",
"distributed.scheduler - INFO - Starting worker compute stream, tcp://127.0.0.1:63662\n",
"distributed.scheduler - INFO - Register tcp://127.0.0.1:63664\n",
"distributed.scheduler - INFO - Register tcp://127.0.0.1:63665\n",
"distributed.scheduler - INFO - Register tcp://127.0.0.1:63663\n",
"distributed.scheduler - INFO - Starting worker compute stream, tcp://127.0.0.1:63664\n",
"distributed.scheduler - INFO - Starting worker compute stream, tcp://127.0.0.1:63665\n",
"distributed.scheduler - INFO - Starting worker compute stream, tcp://127.0.0.1:63663\n",
"distributed.scheduler - INFO - Receive client connection: Client-fb1aa426-afd4-11e9-afab-acde48001122</code></pre></details>\n",
"<details><summary>tcp://127.0.0.1:63662</summary><pre><code>distributed.worker - INFO - Start worker at: tcp://127.0.0.1:63662\n",
"distributed.worker - INFO - Listening to: tcp://127.0.0.1:63662\n",
"distributed.worker - INFO - Waiting to connect to: tcp://127.0.0.1:63657\n",
"distributed.worker - INFO - -------------------------------------------------\n",
"distributed.worker - INFO - Threads: 3\n",
"distributed.worker - INFO - Memory: 4.29 GB\n",
"distributed.worker - INFO - Local Directory: /Users/mrocklin/dask-worker-space/worker-jgj4clli\n",
"distributed.worker - INFO - -------------------------------------------------\n",
"distributed.worker - INFO - Registered to: tcp://127.0.0.1:63657\n",
"distributed.worker - INFO - -------------------------------------------------</code></pre></details>\n",
"<details><summary>tcp://127.0.0.1:63663</summary><pre><code>distributed.worker - INFO - Start worker at: tcp://127.0.0.1:63663\n",
"distributed.worker - INFO - Listening to: tcp://127.0.0.1:63663\n",
"distributed.worker - INFO - Waiting to connect to: tcp://127.0.0.1:63657\n",
"distributed.worker - INFO - -------------------------------------------------\n",
"distributed.worker - INFO - Threads: 3\n",
"distributed.worker - INFO - Memory: 4.29 GB\n",
"distributed.worker - INFO - Local Directory: /Users/mrocklin/dask-worker-space/worker-mdrpl_55\n",
"distributed.worker - INFO - -------------------------------------------------\n",
"distributed.worker - INFO - Registered to: tcp://127.0.0.1:63657\n",
"distributed.worker - INFO - -------------------------------------------------</code></pre></details>\n",
"<details><summary>tcp://127.0.0.1:63664</summary><pre><code>distributed.worker - INFO - Start worker at: tcp://127.0.0.1:63664\n",
"distributed.worker - INFO - Listening to: tcp://127.0.0.1:63664\n",
"distributed.worker - INFO - Waiting to connect to: tcp://127.0.0.1:63657\n",
"distributed.worker - INFO - -------------------------------------------------\n",
"distributed.worker - INFO - Threads: 3\n",
"distributed.worker - INFO - Memory: 4.29 GB\n",
"distributed.worker - INFO - Local Directory: /Users/mrocklin/dask-worker-space/worker-f1vw7tet\n",
"distributed.worker - INFO - -------------------------------------------------\n",
"distributed.worker - INFO - Registered to: tcp://127.0.0.1:63657\n",
"distributed.worker - INFO - -------------------------------------------------</code></pre></details>\n",
"<details><summary>tcp://127.0.0.1:63665</summary><pre><code>distributed.worker - INFO - Start worker at: tcp://127.0.0.1:63665\n",
"distributed.worker - INFO - Listening to: tcp://127.0.0.1:63665\n",
"distributed.worker - INFO - Waiting to connect to: tcp://127.0.0.1:63657\n",
"distributed.worker - INFO - -------------------------------------------------\n",
"distributed.worker - INFO - Threads: 3\n",
"distributed.worker - INFO - Memory: 4.29 GB\n",
"distributed.worker - INFO - Local Directory: /Users/mrocklin/dask-worker-space/worker-1ax0pfpp\n",
"distributed.worker - INFO - -------------------------------------------------\n",
"distributed.worker - INFO - Registered to: tcp://127.0.0.1:63657\n",
"distributed.worker - INFO - -------------------------------------------------</code></pre></details>"
],
"text/plain": [
"{'Scheduler': 'distributed.scheduler - INFO - Clear task state\\ndistributed.scheduler - INFO - Scheduler at: tcp://127.0.0.1:63657\\ndistributed.scheduler - INFO - dashboard at: 127.0.0.1:8787\\ndistributed.scheduler - INFO - Register tcp://127.0.0.1:63662\\ndistributed.scheduler - INFO - Starting worker compute stream, tcp://127.0.0.1:63662\\ndistributed.scheduler - INFO - Register tcp://127.0.0.1:63664\\ndistributed.scheduler - INFO - Register tcp://127.0.0.1:63665\\ndistributed.scheduler - INFO - Register tcp://127.0.0.1:63663\\ndistributed.scheduler - INFO - Starting worker compute stream, tcp://127.0.0.1:63664\\ndistributed.scheduler - INFO - Starting worker compute stream, tcp://127.0.0.1:63665\\ndistributed.scheduler - INFO - Starting worker compute stream, tcp://127.0.0.1:63663\\ndistributed.scheduler - INFO - Receive client connection: Client-fb1aa426-afd4-11e9-afab-acde48001122',\n",
" 'tcp://127.0.0.1:63662': 'distributed.worker - INFO - Start worker at: tcp://127.0.0.1:63662\\ndistributed.worker - INFO - Listening to: tcp://127.0.0.1:63662\\ndistributed.worker - INFO - Waiting to connect to: tcp://127.0.0.1:63657\\ndistributed.worker - INFO - -------------------------------------------------\\ndistributed.worker - INFO - Threads: 3\\ndistributed.worker - INFO - Memory: 4.29 GB\\ndistributed.worker - INFO - Local Directory: /Users/mrocklin/dask-worker-space/worker-jgj4clli\\ndistributed.worker - INFO - -------------------------------------------------\\ndistributed.worker - INFO - Registered to: tcp://127.0.0.1:63657\\ndistributed.worker - INFO - -------------------------------------------------',\n",
" 'tcp://127.0.0.1:63663': 'distributed.worker - INFO - Start worker at: tcp://127.0.0.1:63663\\ndistributed.worker - INFO - Listening to: tcp://127.0.0.1:63663\\ndistributed.worker - INFO - Waiting to connect to: tcp://127.0.0.1:63657\\ndistributed.worker - INFO - -------------------------------------------------\\ndistributed.worker - INFO - Threads: 3\\ndistributed.worker - INFO - Memory: 4.29 GB\\ndistributed.worker - INFO - Local Directory: /Users/mrocklin/dask-worker-space/worker-mdrpl_55\\ndistributed.worker - INFO - -------------------------------------------------\\ndistributed.worker - INFO - Registered to: tcp://127.0.0.1:63657\\ndistributed.worker - INFO - -------------------------------------------------',\n",
" 'tcp://127.0.0.1:63664': 'distributed.worker - INFO - Start worker at: tcp://127.0.0.1:63664\\ndistributed.worker - INFO - Listening to: tcp://127.0.0.1:63664\\ndistributed.worker - INFO - Waiting to connect to: tcp://127.0.0.1:63657\\ndistributed.worker - INFO - -------------------------------------------------\\ndistributed.worker - INFO - Threads: 3\\ndistributed.worker - INFO - Memory: 4.29 GB\\ndistributed.worker - INFO - Local Directory: /Users/mrocklin/dask-worker-space/worker-f1vw7tet\\ndistributed.worker - INFO - -------------------------------------------------\\ndistributed.worker - INFO - Registered to: tcp://127.0.0.1:63657\\ndistributed.worker - INFO - -------------------------------------------------',\n",
" 'tcp://127.0.0.1:63665': 'distributed.worker - INFO - Start worker at: tcp://127.0.0.1:63665\\ndistributed.worker - INFO - Listening to: tcp://127.0.0.1:63665\\ndistributed.worker - INFO - Waiting to connect to: tcp://127.0.0.1:63657\\ndistributed.worker - INFO - -------------------------------------------------\\ndistributed.worker - INFO - Threads: 3\\ndistributed.worker - INFO - Memory: 4.29 GB\\ndistributed.worker - INFO - Local Directory: /Users/mrocklin/dask-worker-space/worker-1ax0pfpp\\ndistributed.worker - INFO - -------------------------------------------------\\ndistributed.worker - INFO - Registered to: tcp://127.0.0.1:63657\\ndistributed.worker - INFO - -------------------------------------------------'}"
]
},
"execution_count": 1,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from dask.distributed import Client\n",
"client = Client()\n",
"client.cluster.logs()"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.1"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment