Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
benchmark.ipynb
{
"nbformat": 4,
"nbformat_minor": 0,
"metadata": {
"colab": {
"name": "benchmark.ipynb",
"provenance": [],
"collapsed_sections": [],
"include_colab_link": true
},
"kernelspec": {
"name": "python3",
"display_name": "Python 3"
},
"accelerator": "GPU"
},
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "view-in-github",
"colab_type": "text"
},
"source": [
"<a href=\"https://colab.research.google.com/gist/nd-02110114/dd39f2d8bb5eaa1e919b1d38683934a4/benchmark.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "AzFXv5-BpZoe",
"colab_type": "text"
},
"source": [
"# Benchmark with Tox21"
]
},
{
"cell_type": "code",
"metadata": {
"id": "ELfpEnU-G-vp",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 357
},
"outputId": "adf9aa4a-f91a-4c70-fdab-d5a50689621a"
},
"source": [
"!nvidia-smi"
],
"execution_count": 1,
"outputs": [
{
"output_type": "stream",
"text": [
"Sat Jul 4 13:31:54 2020 \n",
"+-----------------------------------------------------------------------------+\n",
"| NVIDIA-SMI 450.36.06 Driver Version: 418.67 CUDA Version: 10.1 |\n",
"|-------------------------------+----------------------+----------------------+\n",
"| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |\n",
"| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |\n",
"| | | MIG M. |\n",
"|===============================+======================+======================|\n",
"| 0 Tesla P100-PCIE... Off | 00000000:00:04.0 Off | 0 |\n",
"| N/A 34C P0 26W / 250W | 0MiB / 16280MiB | 0% Default |\n",
"| | | ERR! |\n",
"+-------------------------------+----------------------+----------------------+\n",
" \n",
"+-----------------------------------------------------------------------------+\n",
"| Processes: |\n",
"| GPU GI CI PID Type Process name GPU Memory |\n",
"| ID ID Usage |\n",
"|=============================================================================|\n",
"| No running processes found |\n",
"+-----------------------------------------------------------------------------+\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "n4h2AF-gHBGD",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 85
},
"outputId": "7ad2f611-f021-44af-c7bd-6dee64f4775a"
},
"source": [
"!nvcc --version"
],
"execution_count": 2,
"outputs": [
{
"output_type": "stream",
"text": [
"nvcc: NVIDIA (R) Cuda compiler driver\n",
"Copyright (c) 2005-2019 NVIDIA Corporation\n",
"Built on Sun_Jul_28_19:07:16_PDT_2019\n",
"Cuda compilation tools, release 10.1, V10.1.243\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "ymv78zqiHI18",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 68
},
"outputId": "774a1c21-ba66-4f00-dfa1-30992a36f561"
},
"source": [
"!ls -l /usr/lib/x86_64-linux-gnu/libcudnn.so*"
],
"execution_count": 3,
"outputs": [
{
"output_type": "stream",
"text": [
"lrwxrwxrwx 1 root root 29 Dec 5 2019 /usr/lib/x86_64-linux-gnu/libcudnn.so -> /etc/alternatives/libcudnn_so\n",
"lrwxrwxrwx 1 root root 17 Oct 27 2019 /usr/lib/x86_64-linux-gnu/libcudnn.so.7 -> libcudnn.so.7.6.5\n",
"-rw-r--r-- 1 root root 428711256 Oct 27 2019 /usr/lib/x86_64-linux-gnu/libcudnn.so.7.6.5\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "TmtjyHIFrkKi",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 153
},
"outputId": "5ae9c69b-ebbc-4b42-bb7c-7cd0c77580ae"
},
"source": [
"%tensorflow_version 1.x\n",
"!curl -Lo deepchem_installer.py https://raw.githubusercontent.com/deepchem/deepchem/master/scripts/colab_install.py\n",
"import deepchem_installer\n",
"%time deepchem_installer.install(version='2.3.0')"
],
"execution_count": 4,
"outputs": [
{
"output_type": "stream",
"text": [
"TensorFlow 1.x selected.\n",
" % Total % Received % Xferd Average Speed Time Time Time Current\n",
" Dload Upload Total Spent Left Speed\n",
"100 3477 100 3477 0 0 79022 0 --:--:-- --:--:-- --:--:-- 79022\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"add /root/miniconda/lib/python3.6/site-packages to PYTHONPATH\n",
"deepchem is already installed\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"CPU times: user 2.33 ms, sys: 80 µs, total: 2.41 ms\n",
"Wall time: 3.63 ms\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "askrOx-kbXGk",
"colab_type": "text"
},
"source": [
"**Caution** : After running the following commands, you need to restart the session. If you don't restart the session, maybe you will face an error."
]
},
{
"cell_type": "code",
"metadata": {
"id": "Pcur1OJXIAJz",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 34
},
"outputId": "688e5e5b-d882-4233-c374-ce932159040c"
},
"source": [
"!pip install -q dm-haiku==0.0.1 typing-extensions==3.7.4.2 git+https://github.com/deepchem/jaxchem@baab33996dc45622b5b7cc557800838266585eaf"
],
"execution_count": 5,
"outputs": [
{
"output_type": "stream",
"text": [
" Building wheel for jaxchem (setup.py) ... \u001b[?25l\u001b[?25hdone\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "AclWlcku4SVC",
"colab_type": "code",
"colab": {}
},
"source": [
"!pip install -q dgl-cu101 dgllife"
],
"execution_count": 6,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "CJd7o5qNcps5",
"colab_type": "text"
},
"source": [
"## JAXChem perfoemance\n",
"\n",
"If we face the error `ImportError: cannot import name 'Literal'\n",
"`, we should restart the session of this notebook."
]
},
{
"cell_type": "code",
"metadata": {
"id": "2XBbmpnacoYa",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 683
},
"outputId": "202062a1-6e35-463f-81c4-41c4910508af"
},
"source": [
"import os\n",
"import time\n",
"import random\n",
"import pickle\n",
"import argparse\n",
"from typing import Any, Tuple, List\n",
"\n",
"import jax\n",
"import numpy as np\n",
"import haiku as hk\n",
"import jax.numpy as jnp\n",
"from jax.experimental import optix\n",
"\n",
"\n",
"from deepchem.molnet import load_tox21\n",
"from jaxchem.models import PadGCNPredicator as GCNPredicator\n",
"from jaxchem.loss import binary_cross_entropy_with_logits as bce_with_logits\n",
"\n",
"\n",
"# type definition\n",
"Batch = Tuple[Tuple[np.ndarray, np.ndarray], np.ndarray]\n",
"State, OptState = Any, Any\n",
"\n",
"\n",
"# load tox21 dataset\n",
"tox21_tasks, tox21_datasets, _ = load_tox21(featurizer='AdjacencyConv', reload=True)\n",
"train_dataset, valid_dataset, test_dataset = tox21_datasets\n",
"\n",
"# define utilities\n",
"def seed_everything(seed: int = 42):\n",
" random.seed(seed)\n",
" os.environ['PYTHONHASHSEED'] = str(seed)\n",
" np.random.seed(seed)\n",
"\n",
"\n",
"def collate_fn(original_batch: Any) -> Batch:\n",
" \"\"\"Make batch data as PadGCN model inputs.\"\"\"\n",
" inputs, targets, _, _ = original_batch\n",
" node_feats = np.array([inputs[i][1] for i in range(len(inputs))])\n",
" adj = np.array([inputs[i][0] for i in range(len(inputs))])\n",
" return (node_feats, adj), np.array(targets)\n",
"\n",
"\n",
"def multi_task_roc_auc_score(y_true: np.ndarray, y_score: np.ndarray) -> Tuple[float, List[float]]:\n",
" \"\"\"Calculate the roc_auc_score of all tasks for Tox21.\"\"\"\n",
" num_tasks = y_true.shape[1]\n",
" scores = []\n",
" for i in range(num_tasks):\n",
" scores.append(roc_auc_score(y_true[:, i], y_score[:, i]))\n",
" return np.mean(scores), scores\n",
"\n",
"# setup the model and optimizer\n",
"rng_seq = hk.PRNGSequence(1234)\n",
"\n",
"# model params\n",
"in_feats = train_dataset.X[0][1].shape[1] # 150\n",
"hidden_feats = [64, 64, 32]\n",
"activation, batch_norm, dropout = None, None, None # use default\n",
"predicator_hidden_feats = 32\n",
"pooling_method = 'mean'\n",
"predicator_dropout = 0.2\n",
"n_out = len(tox21_tasks)\n",
"\n",
"def forward(node_feats: jnp.ndarray, adj: jnp.ndarray, is_training: bool) -> jnp.ndarray:\n",
" \"\"\"Forward application of the GCN.\"\"\"\n",
" model = GCNPredicator(in_feats=in_feats, hidden_feats=hidden_feats, activation=activation,\n",
" batch_norm=batch_norm, dropout=dropout, pooling_method=pooling_method,\n",
" predicator_hidden_feats=predicator_hidden_feats,\n",
" predicator_dropout=predicator_dropout, n_out=n_out)\n",
" preds = model(node_feats, adj, is_training)\n",
" return preds\n",
"\n",
"# we use haiku\n",
"model = hk.transform_with_state(forward)\n",
"\n",
"lr = 0.001\n",
"optimizer = optix.adam(learning_rate=lr)\n",
"\n",
"\n",
"# define training loss\n",
"def train_loss(params: hk.Params, state: State, batch: Batch) -> Tuple[jnp.ndarray, State]:\n",
" \"\"\"Compute the loss.\"\"\"\n",
" inputs, targets = batch\n",
" preds, new_state = model.apply(params, state, next(rng_seq), *inputs, True)\n",
" loss = bce_with_logits(preds, targets)\n",
" return loss, new_state\n",
"\n",
"# define training update\n",
"@jax.jit\n",
"def update(params: hk.Params, state: State, opt_state: OptState,\n",
" batch: Batch) -> Tuple[hk.Params, State, OptState]:\n",
" \"\"\"Update the params.\"\"\"\n",
" (_, new_state), grads = jax.value_and_grad(train_loss, has_aux=True)(params, state, batch)\n",
" updates, new_opt_state = optimizer.update(grads, opt_state)\n",
" new_params = optix.apply_updates(params, updates)\n",
" return new_params, new_state, new_opt_state\n",
"\n",
"# define evaluate metrics\n",
"@jax.jit\n",
"def evaluate(params: hk.Params, state: State, batch: Batch) -> jnp.ndarray:\n",
" \"\"\"Compute evaluate metrics.\"\"\"\n",
" inputs, targets = batch\n",
" preds, _ = model.apply(params, state, next(rng_seq), *inputs, False)\n",
" loss = bce_with_logits(preds, targets)\n",
" return preds, loss, targets\n",
"\n",
"\n",
"# training params\n",
"seed = 42\n",
"batch_size = 128\n",
"num_epochs = 50\n",
"\n",
"# fix seed\n",
"seed_everything(seed)\n",
"\n",
"# initialize some values \n",
"batch_init_data = (\n",
" jnp.zeros((batch_size, *train_dataset.X[0][1].shape)),\n",
" jnp.zeros((batch_size, *train_dataset.X[0][0].shape)),\n",
" True\n",
")\n",
"params, state = model.init(next(rng_seq), *batch_init_data)\n",
"opt_state = optimizer.init(params)\n",
"\n",
"\n",
"# save performance metrics\n",
"jax_times = []\n",
"print(\"Starting training...\")\n",
"for epoch in range(num_epochs):\n",
" # train\n",
" start_time = time.time()\n",
" for original_batch in train_dataset.iterbatches(batch_size=batch_size):\n",
" batch = collate_fn(original_batch)\n",
" params, state, opt_state = update(params, state, opt_state, batch)\n",
" epoch_time = time.time() - start_time\n",
" jax_times.append(epoch_time)"
],
"execution_count": 7,
"outputs": [
{
"output_type": "stream",
"text": [
"/usr/local/lib/python3.6/dist-packages/sklearn/externals/joblib/__init__.py:15: FutureWarning: sklearn.externals.joblib is deprecated in 0.21 and will be removed in 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib. If this warning is raised when loading pickled models, you may need to re-serialize those models with scikit-learn 0.21+.\n",
" warnings.warn(msg, category=FutureWarning)\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"WARNING:tensorflow:\n",
"The TensorFlow contrib module will not be included in TensorFlow 2.0.\n",
"For more information, please see:\n",
" * https://github.com/tensorflow/community/blob/master/rfcs/20180907-contrib-sunset.md\n",
" * https://github.com/tensorflow/addons\n",
" * https://github.com/tensorflow/io (for I/O related ops)\n",
"If you depend on functionality not listed there, please file an issue.\n",
"\n",
"Loading raw samples now.\n",
"shard_size: 8192\n",
"About to start loading CSV from /tmp/tox21.csv.gz\n",
"Loading shard 1 of size 8192.\n",
"Featurizing sample 0\n",
"Featurizing sample 1000\n",
"Featurizing sample 2000\n",
"Featurizing sample 3000\n",
"Featurizing sample 4000\n",
"Featurizing sample 5000\n",
"Featurizing sample 6000\n",
"Featurizing sample 7000\n",
"TIMING: featurizing shard 0 took 9.926 s\n",
"TIMING: dataset construction took 11.882 s\n",
"Loading dataset from disk.\n",
"TIMING: dataset construction took 1.851 s\n",
"Loading dataset from disk.\n",
"TIMING: dataset construction took 0.902 s\n",
"Loading dataset from disk.\n",
"TIMING: dataset construction took 0.899 s\n",
"Loading dataset from disk.\n",
"TIMING: dataset construction took 1.688 s\n",
"Loading dataset from disk.\n",
"TIMING: dataset construction took 0.209 s\n",
"Loading dataset from disk.\n",
"TIMING: dataset construction took 0.216 s\n",
"Loading dataset from disk.\n",
"Starting training...\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "VEhz4QWKddkI",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 34
},
"outputId": "5bb46161-faff-4ac8-ddc6-16de0082d696"
},
"source": [
"print(\"Training time/epoch : mean {:.4f} std {:.4f}\".format(np.mean(jax_times), np.std(jax_times)))"
],
"execution_count": 8,
"outputs": [
{
"output_type": "stream",
"text": [
"Training time/epoch : mean 1.4271 std 0.4858\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "FiutLV8U65K2",
"colab_type": "text"
},
"source": [
"## DGL performance\n",
"\n",
"The following code is from here : https://github.com/awslabs/dgl-lifesci/blob/master/examples/property_prediction/moleculenet/classification.py"
]
},
{
"cell_type": "code",
"metadata": {
"id": "xOv1FZhXCrei",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 204
},
"outputId": "3dfe4fc3-8750-42e4-8233-654b191fda7c"
},
"source": [
"!wget https://raw.githubusercontent.com/awslabs/dgl-lifesci/master/examples/property_prediction/moleculenet/utils.py"
],
"execution_count": 9,
"outputs": [
{
"output_type": "stream",
"text": [
"--2020-07-04 13:33:52-- https://raw.githubusercontent.com/awslabs/dgl-lifesci/master/examples/property_prediction/moleculenet/utils.py\n",
"Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.0.133, 151.101.64.133, 151.101.128.133, ...\n",
"Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.0.133|:443... connected.\n",
"HTTP request sent, awaiting response... 200 OK\n",
"Length: 4337 (4.2K) [text/plain]\n",
"Saving to: ‘utils.py’\n",
"\n",
"\rutils.py 0%[ ] 0 --.-KB/s \rutils.py 100%[===================>] 4.24K --.-KB/s in 0s \n",
"\n",
"2020-07-04 13:33:52 (62.4 MB/s) - ‘utils.py’ saved [4337/4337]\n",
"\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "tVrugFkk792T",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 258
},
"outputId": "da825f92-8386-4381-e4ab-18d7d2025489"
},
"source": [
"import dgl\n",
"import random\n",
"import numpy as np\n",
"import torch\n",
"\n",
"from dgllife.utils import RandomSplitter, CanonicalAtomFeaturizer, smiles_to_bigraph\n",
"from dgllife.model import GCNPredictor\n",
"from torch.nn import BCEWithLogitsLoss\n",
"from torch.optim import Adam\n",
"from torch.utils.data import DataLoader\n",
"\n",
"from utils import set_random_seed, load_dataset_for_classification, collate_molgraphs\n",
"\n",
"\n",
"def predict(args, model, bg):\n",
" node_feats = bg.ndata.pop(args['node_data_field']).to(args['device'])\n",
" if args.get('edge_featurizer', None) is not None:\n",
" edge_feats = bg.edata.pop(args['edge_data_field']).to(args['device'])\n",
" return model(bg, node_feats, edge_feats)\n",
" else:\n",
" return model(bg, node_feats)\n",
"\n",
"\n",
"def run_a_train_epoch(args, epoch, model, data_loader, loss_criterion, optimizer):\n",
" model.train()\n",
" for batch_id, batch_data in enumerate(data_loader):\n",
" smiles, bg, labels, masks = batch_data\n",
" labels, masks = labels.to(args['device']), masks.to(args['device'])\n",
" logits = predict(args, model, bg)\n",
" # Mask non-existing labels\n",
" loss = (loss_criterion(logits, labels) * (masks != 0).float()).mean()\n",
" optimizer.zero_grad()\n",
" loss.backward()\n",
" optimizer.step()\n",
"\n",
"# same setting\n",
"args = {\n",
" 'random_seed': 42,\n",
" 'batch_size': 128,\n",
" 'device': torch.device(\"cuda\") if torch.cuda.is_available() else torch.device(\"cpu\"),\n",
" 'num_epochs' : 50,\n",
" 'in_feats': 75,\n",
" 'gcn_hidden_feats': [64, 64, 32],\n",
" 'classifier_hidden_feats': 32,\n",
" 'classifier_dropout': 0.2,\n",
" 'lr': 0.001,\n",
" 'dataset': 'Tox21',\n",
" 'exp': '_'.join(['GCN', 'Tox21']),\n",
" 'node_data_field': 'h',\n",
" 'frac_train': 0.8,\n",
" 'frac_val': 0.1,\n",
" 'frac_test': 0.1,\n",
" 'node_featurizer': CanonicalAtomFeaturizer(),\n",
" 'smiles_to_graph': smiles_to_bigraph,\n",
"}\n",
"\n",
"# fix seed\n",
"set_random_seed(args['random_seed'])\n",
"\n",
"# Interchangeable with other datasets\n",
"dataset, train_set, val_set, test_set = load_dataset_for_classification(args)\n",
"train_loader = DataLoader(train_set, batch_size=args['batch_size'], collate_fn=collate_molgraphs, shuffle=True)\n",
"val_loader = DataLoader(val_set, batch_size=args['batch_size'], collate_fn=collate_molgraphs)\n",
"test_loader = DataLoader(test_set, batch_size=args['batch_size'], collate_fn=collate_molgraphs)\n",
"\n",
"# multi task classification\n",
"args['n_tasks'] = dataset.n_tasks\n",
"model = GCNPredictor(in_feats=args['node_featurizer'].feat_size(),\n",
" hidden_feats=args['gcn_hidden_feats'],\n",
" classifier_hidden_feats=args['classifier_hidden_feats'],\n",
" classifier_dropout=args['classifier_dropout'],\n",
" n_tasks=args['n_tasks'])\n",
"loss_criterion = BCEWithLogitsLoss(pos_weight=dataset.task_pos_weights.to(args['device']), reduction='mean')\n",
"optimizer = Adam(model.parameters(), lr=args['lr'])\n",
"model.to(args['device'])\n",
"\n",
"dgl_times = []\n",
"print(\"Starting training...\")\n",
"for epoch in range(args['num_epochs']):\n",
" start_time = time.time()\n",
" run_a_train_epoch(args, epoch, model, train_loader, loss_criterion, optimizer)\n",
" epoch_time = time.time() - start_time\n",
" dgl_times.append(epoch_time)"
],
"execution_count": 10,
"outputs": [
{
"output_type": "stream",
"text": [
"DGL backend not selected or invalid. Assuming PyTorch for now.\n",
"Using backend: pytorch\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"Setting the default backend to \"pytorch\". You can change it in the ~/.dgl/config.json file or export the DGLBACKEND environment variable. Valid options are: pytorch, mxnet, tensorflow (all lowercase)\n",
"Downloading /root/.dgl/tox21.csv.gz from https://data.dgl.ai/dataset/tox21.csv.gz...\n",
"Processing dgl graphs from scratch...\n",
"Processing molecule 1000/7831\n",
"Processing molecule 2000/7831\n",
"Processing molecule 3000/7831\n",
"Processing molecule 4000/7831\n",
"Processing molecule 5000/7831\n",
"Processing molecule 6000/7831\n",
"Processing molecule 7000/7831\n",
"Starting training...\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "1MgW25vACWZX",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 34
},
"outputId": "1499bf8d-3bd9-4a60-8300-ef062dcd166c"
},
"source": [
"print(\"Training time/epoch : mean {:.4f} std {:.4f}\".format(np.mean(dgl_times), np.std(dgl_times)))"
],
"execution_count": 11,
"outputs": [
{
"output_type": "stream",
"text": [
"Training time/epoch : mean 0.9153 std 0.2563\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "KMNFIHZVDXtc",
"colab_type": "text"
},
"source": [
"## DeepChem performance\n"
]
},
{
"cell_type": "code",
"metadata": {
"id": "RT13bAbxDbAM",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 493
},
"outputId": "a75d6b5d-854b-4fc1-8d27-396098d25f61"
},
"source": [
"import warnings\n",
"warnings.simplefilter('ignore')\n",
"\n",
"import logging\n",
"import tensorflow as tf\n",
"tf.get_logger().setLevel('INFO')\n",
"tf.autograph.set_verbosity(0)\n",
"tf.get_logger().setLevel(logging.ERROR)\n",
"\n",
"from deepchem.models.graph_models import GraphConvModel\n",
"\n",
"tox21_tasks, tox21_datasets, transformers = load_tox21(featurizer='GraphConv', reload=False)\n",
"train_dataset, valid_dataset, test_dataset = tox21_datasets\n",
"\n",
"n_tasks = len(tox21_tasks)\n",
"model = GraphConvModel(n_tasks, graph_conv_layers=[64, 64, 32], batch_size=128, mode='classification', \n",
" dense_layer_size=32, dropout=[0, 0, 0, 0.2])\n",
"\n",
"num_epochs = 50\n",
"deepchem_times = []\n",
"print(\"Starting training...\")\n",
"for i in range(num_epochs):\n",
" start_time = time.time()\n",
" _ = model.fit(train_dataset, nb_epoch=1)\n",
" epoch_time = time.time() - start_time\n",
" deepchem_times.append(epoch_time)"
],
"execution_count": 12,
"outputs": [
{
"output_type": "stream",
"text": [
"Loading raw samples now.\n",
"shard_size: 8192\n",
"About to start loading CSV from /tmp/tox21.csv.gz\n",
"Loading shard 1 of size 8192.\n",
"Featurizing sample 0\n",
"Featurizing sample 1000\n",
"Featurizing sample 2000\n",
"Featurizing sample 3000\n",
"Featurizing sample 4000\n",
"Featurizing sample 5000\n",
"Featurizing sample 6000\n",
"Featurizing sample 7000\n",
"TIMING: featurizing shard 0 took 26.745 s\n",
"TIMING: dataset construction took 29.300 s\n",
"Loading dataset from disk.\n",
"TIMING: dataset construction took 2.853 s\n",
"Loading dataset from disk.\n",
"TIMING: dataset construction took 1.295 s\n",
"Loading dataset from disk.\n",
"TIMING: dataset construction took 1.300 s\n",
"Loading dataset from disk.\n",
"TIMING: dataset construction took 2.639 s\n",
"Loading dataset from disk.\n",
"TIMING: dataset construction took 0.493 s\n",
"Loading dataset from disk.\n",
"TIMING: dataset construction took 0.304 s\n",
"Loading dataset from disk.\n",
"Starting training...\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "p8BrRuljGMdb",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 34
},
"outputId": "a7acd804-ca45-4df4-c583-9bd3f0990f3a"
},
"source": [
"print(\"Training time/epoch : mean {:.4f} std {:.4f}\".format(np.mean(deepchem_times), np.std(deepchem_times)))"
],
"execution_count": 13,
"outputs": [
{
"output_type": "stream",
"text": [
"Training time/epoch : mean 3.9689 std 2.2216\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "L-5iT9JU1gU4",
"colab_type": "text"
},
"source": [
"## Result"
]
},
{
"cell_type": "code",
"metadata": {
"id": "dqRVTRXP1f7d",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 517
},
"outputId": "03f320d3-c5ca-4a22-92cc-5ebd1b0cd4c3"
},
"source": [
"import plotly.graph_objects as go\n",
"\n",
"fig = go.Figure()\n",
"fig.add_trace(go.Bar(\n",
" name='Result',\n",
" x=['JAXChem', 'DGL', 'DeepChem'], y=[np.mean(jax_times), np.mean(dgl_times), np.mean(deepchem_times)],\n",
" error_y=dict(type='data', array=[np.std(jax_times), np.std(dgl_times), np.std(deepchem_times)])\n",
"))\n",
"fig.update_layout(barmode='group', width=500, height=500)\n",
"fig.show()"
],
"execution_count": 14,
"outputs": [
{
"output_type": "display_data",
"data": {
"text/html": [
"<html>\n",
"<head><meta charset=\"utf-8\" /></head>\n",
"<body>\n",
" <div>\n",
" <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax) {MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>\n",
" <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
" <script src=\"https://cdn.plot.ly/plotly-latest.min.js\"></script> \n",
" <div id=\"63e2c829-b673-4ec1-9103-213301bbcd2a\" class=\"plotly-graph-div\" style=\"height:500px; width:500px;\"></div>\n",
" <script type=\"text/javascript\">\n",
" \n",
" window.PLOTLYENV=window.PLOTLYENV || {};\n",
" \n",
" if (document.getElementById(\"63e2c829-b673-4ec1-9103-213301bbcd2a\")) {\n",
" Plotly.newPlot(\n",
" '63e2c829-b673-4ec1-9103-213301bbcd2a',\n",
" [{\"error_y\": {\"array\": [0.4857512980775718, 0.256291220680815, 2.2215592586918826], \"type\": \"data\"}, \"name\": \"Result\", \"type\": \"bar\", \"x\": [\"JAXChem\", \"DGL\", \"DeepChem\"], \"y\": [1.4271090888977052, 0.9152973747253418, 3.9688595819473265]}],\n",
" {\"barmode\": \"group\", \"height\": 500, \"template\": {\"data\": {\"bar\": [{\"error_x\": {\"color\": \"#2a3f5f\"}, \"error_y\": {\"color\": \"#2a3f5f\"}, \"marker\": {\"line\": {\"color\": \"#E5ECF6\", \"width\": 0.5}}, \"type\": \"bar\"}], \"barpolar\": [{\"marker\": {\"line\": {\"color\": \"#E5ECF6\", \"width\": 0.5}}, \"type\": \"barpolar\"}], \"carpet\": [{\"aaxis\": {\"endlinecolor\": \"#2a3f5f\", \"gridcolor\": \"white\", \"linecolor\": \"white\", \"minorgridcolor\": \"white\", \"startlinecolor\": \"#2a3f5f\"}, \"baxis\": {\"endlinecolor\": \"#2a3f5f\", \"gridcolor\": \"white\", \"linecolor\": \"white\", \"minorgridcolor\": \"white\", \"startlinecolor\": \"#2a3f5f\"}, \"type\": \"carpet\"}], \"choropleth\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"choropleth\"}], \"contour\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"contour\"}], \"contourcarpet\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"contourcarpet\"}], \"heatmap\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"heatmap\"}], \"heatmapgl\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"heatmapgl\"}], \"histogram\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"histogram\"}], \"histogram2d\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"histogram2d\"}], \"histogram2dcontour\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"histogram2dcontour\"}], \"mesh3d\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"mesh3d\"}], \"parcoords\": [{\"line\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"parcoords\"}], \"pie\": [{\"automargin\": true, \"type\": \"pie\"}], \"scatter\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatter\"}], \"scatter3d\": [{\"line\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatter3d\"}], \"scattercarpet\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattercarpet\"}], \"scattergeo\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattergeo\"}], \"scattergl\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattergl\"}], \"scattermapbox\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattermapbox\"}], \"scatterpolar\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterpolar\"}], \"scatterpolargl\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterpolargl\"}], \"scatterternary\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterternary\"}], \"surface\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"surface\"}], \"table\": [{\"cells\": {\"fill\": {\"color\": \"#EBF0F8\"}, \"line\": {\"color\": \"white\"}}, \"header\": {\"fill\": {\"color\": \"#C8D4E3\"}, \"line\": {\"color\": \"white\"}}, \"type\": \"table\"}]}, \"layout\": {\"annotationdefaults\": {\"arrowcolor\": \"#2a3f5f\", \"arrowhead\": 0, \"arrowwidth\": 1}, \"coloraxis\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"colorscale\": {\"diverging\": [[0, \"#8e0152\"], [0.1, \"#c51b7d\"], [0.2, \"#de77ae\"], [0.3, \"#f1b6da\"], [0.4, \"#fde0ef\"], [0.5, \"#f7f7f7\"], [0.6, \"#e6f5d0\"], [0.7, \"#b8e186\"], [0.8, \"#7fbc41\"], [0.9, \"#4d9221\"], [1, \"#276419\"]], \"sequential\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"sequentialminus\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]]}, \"colorway\": [\"#636efa\", \"#EF553B\", \"#00cc96\", \"#ab63fa\", \"#FFA15A\", \"#19d3f3\", \"#FF6692\", \"#B6E880\", \"#FF97FF\", \"#FECB52\"], \"font\": {\"color\": \"#2a3f5f\"}, \"geo\": {\"bgcolor\": \"white\", \"lakecolor\": \"white\", \"landcolor\": \"#E5ECF6\", \"showlakes\": true, \"showland\": true, \"subunitcolor\": \"white\"}, \"hoverlabel\": {\"align\": \"left\"}, \"hovermode\": \"closest\", \"mapbox\": {\"style\": \"light\"}, \"paper_bgcolor\": \"white\", \"plot_bgcolor\": \"#E5ECF6\", \"polar\": {\"angularaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"bgcolor\": \"#E5ECF6\", \"radialaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}}, \"scene\": {\"xaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}, \"yaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}, \"zaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}}, \"shapedefaults\": {\"line\": {\"color\": \"#2a3f5f\"}}, \"ternary\": {\"aaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"baxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"bgcolor\": \"#E5ECF6\", \"caxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}}, \"title\": {\"x\": 0.05}, \"xaxis\": {\"automargin\": true, \"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\", \"title\": {\"standoff\": 15}, \"zerolinecolor\": \"white\", \"zerolinewidth\": 2}, \"yaxis\": {\"automargin\": true, \"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\", \"title\": {\"standoff\": 15}, \"zerolinecolor\": \"white\", \"zerolinewidth\": 2}}}, \"width\": 500},\n",
" {\"responsive\": true}\n",
" ).then(function(){\n",
" \n",
"var gd = document.getElementById('63e2c829-b673-4ec1-9103-213301bbcd2a');\n",
"var x = new MutationObserver(function (mutations, observer) {{\n",
" var display = window.getComputedStyle(gd).display;\n",
" if (!display || display === 'none') {{\n",
" console.log([gd, 'removed!']);\n",
" Plotly.purge(gd);\n",
" observer.disconnect();\n",
" }}\n",
"}});\n",
"\n",
"// Listen for the removal of the full notebook cells\n",
"var notebookContainer = gd.closest('#notebook-container');\n",
"if (notebookContainer) {{\n",
" x.observe(notebookContainer, {childList: true});\n",
"}}\n",
"\n",
"// Listen for the clearing of the current output cell\n",
"var outputEl = gd.closest('.output');\n",
"if (outputEl) {{\n",
" x.observe(outputEl, {childList: true});\n",
"}}\n",
"\n",
" })\n",
" };\n",
" \n",
" </script>\n",
" </div>\n",
"</body>\n",
"</html>"
]
},
"metadata": {
"tags": []
}
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "ybmFEjVF6QXh",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 817
},
"outputId": "aec16f86-4782-408e-896a-5f36f6b78e28"
},
"source": [
"import pandas as pd\n",
"import plotly.express as px\n",
"\n",
"\n",
"df = pd.DataFrame()\n",
"df['times'] = jax_times + dgl_times + deepchem_times\n",
"df['model'] = ['JAXChem' for _ in range(50)] + ['DGL' for _ in range(50)] + ['DeepChem' for _ in range(50)]\n",
"\n",
"fig = go.Figure()\n",
"for model in ['JAXChem', 'DGL', 'DeepChem']:\n",
" fig.add_trace(go.Box(x=df['times'][df['model'] == model], name=model))\n",
"\n",
"fig.update_layout(barmode='group', width=800, height=800)\n",
"fig.show()"
],
"execution_count": 15,
"outputs": [
{
"output_type": "display_data",
"data": {
"text/html": [
"<html>\n",
"<head><meta charset=\"utf-8\" /></head>\n",
"<body>\n",
" <div>\n",
" <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax) {MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>\n",
" <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
" <script src=\"https://cdn.plot.ly/plotly-latest.min.js\"></script> \n",
" <div id=\"a148e8b0-7fe2-4ac0-8de0-697ee0659cfe\" class=\"plotly-graph-div\" style=\"height:800px; width:800px;\"></div>\n",
" <script type=\"text/javascript\">\n",
" \n",
" window.PLOTLYENV=window.PLOTLYENV || {};\n",
" \n",
" if (document.getElementById(\"a148e8b0-7fe2-4ac0-8de0-697ee0659cfe\")) {\n",
" Plotly.newPlot(\n",
" 'a148e8b0-7fe2-4ac0-8de0-697ee0659cfe',\n",
" [{\"name\": \"JAXChem\", \"type\": \"box\", \"x\": [4.821335315704346, 1.4849739074707031, 1.3485093116760254, 1.3490703105926514, 1.3454358577728271, 1.3627369403839111, 1.3389172554016113, 1.3667304515838623, 1.3456051349639893, 1.3460328578948975, 1.3521146774291992, 1.3486034870147705, 1.357898235321045, 1.3432512283325195, 1.3612205982208252, 1.3732829093933105, 1.3644380569458008, 1.3726117610931396, 1.3789541721343994, 1.3555400371551514, 1.3857059478759766, 1.3786122798919678, 1.3565232753753662, 1.3565130233764648, 1.3493015766143799, 1.3472208976745605, 1.343790054321289, 1.3576323986053467, 1.3526289463043213, 1.3432939052581787, 1.3467130661010742, 1.4927117824554443, 1.342667818069458, 1.3478057384490967, 1.3539040088653564, 1.3376154899597168, 1.3459291458129883, 1.3406429290771484, 1.339482069015503, 1.3501205444335938, 1.3329660892486572, 1.3448030948638916, 1.3499095439910889, 1.3467018604278564, 1.340406894683838, 1.3525242805480957, 1.3508336544036865, 1.3499982357025146, 1.355203628540039, 1.3460297584533691]}, {\"name\": \"DGL\", \"type\": \"box\", \"x\": [2.6988327503204346, 0.8953413963317871, 0.8743867874145508, 0.9060776233673096, 0.8746867179870605, 0.8966937065124512, 0.8902359008789062, 0.8753786087036133, 0.8658778667449951, 0.8745756149291992, 0.8792989253997803, 0.8755795955657959, 0.8743031024932861, 0.8648576736450195, 0.8635189533233643, 0.8728554248809814, 0.8641180992126465, 0.8685383796691895, 0.8673648834228516, 0.8720741271972656, 0.8740830421447754, 0.8662161827087402, 0.8692030906677246, 1.0609076023101807, 0.8706912994384766, 0.8710954189300537, 0.8825931549072266, 0.8731322288513184, 0.8840985298156738, 0.8758857250213623, 0.8715972900390625, 0.870629072189331, 0.865602970123291, 0.86283278465271, 0.8896894454956055, 0.881455659866333, 0.8751201629638672, 0.8661856651306152, 0.8721773624420166, 0.8697452545166016, 0.8704137802124023, 0.8810818195343018, 0.8682501316070557, 0.8625953197479248, 0.869586706161499, 0.8712072372436523, 0.8980793952941895, 0.8763039112091064, 0.874321699142456, 0.8854906558990479]}, {\"name\": \"DeepChem\", \"type\": \"box\", \"x\": [19.491947650909424, 3.7802820205688477, 3.5393545627593994, 3.869814157485962, 3.611436128616333, 3.9288177490234375, 3.534818649291992, 3.5068368911743164, 3.768502950668335, 3.5111284255981445, 3.7992236614227295, 3.53842830657959, 3.5217232704162598, 3.760028123855591, 3.523338794708252, 3.796879768371582, 3.5586438179016113, 3.801560878753662, 3.5330498218536377, 3.8054819107055664, 3.567582607269287, 3.7731189727783203, 3.5751078128814697, 3.8008460998535156, 3.540738821029663, 3.4896585941314697, 3.7981250286102295, 3.512373447418213, 3.795011520385742, 3.520874261856079, 3.76708984375, 3.55607271194458, 3.825932264328003, 3.524959087371826, 3.5379624366760254, 3.812232494354248, 3.553412675857544, 3.554962635040283, 3.8437180519104004, 3.5684328079223633, 3.5755255222320557, 3.781198024749756, 3.5274529457092285, 3.5142691135406494, 3.759883403778076, 3.522552728652954, 3.7831218242645264, 3.5358848571777344, 3.804720640182495, 3.538860321044922]}],\n",
" {\"barmode\": \"group\", \"height\": 800, \"template\": {\"data\": {\"bar\": [{\"error_x\": {\"color\": \"#2a3f5f\"}, \"error_y\": {\"color\": \"#2a3f5f\"}, \"marker\": {\"line\": {\"color\": \"#E5ECF6\", \"width\": 0.5}}, \"type\": \"bar\"}], \"barpolar\": [{\"marker\": {\"line\": {\"color\": \"#E5ECF6\", \"width\": 0.5}}, \"type\": \"barpolar\"}], \"carpet\": [{\"aaxis\": {\"endlinecolor\": \"#2a3f5f\", \"gridcolor\": \"white\", \"linecolor\": \"white\", \"minorgridcolor\": \"white\", \"startlinecolor\": \"#2a3f5f\"}, \"baxis\": {\"endlinecolor\": \"#2a3f5f\", \"gridcolor\": \"white\", \"linecolor\": \"white\", \"minorgridcolor\": \"white\", \"startlinecolor\": \"#2a3f5f\"}, \"type\": \"carpet\"}], \"choropleth\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"choropleth\"}], \"contour\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"contour\"}], \"contourcarpet\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"contourcarpet\"}], \"heatmap\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"heatmap\"}], \"heatmapgl\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"heatmapgl\"}], \"histogram\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"histogram\"}], \"histogram2d\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"histogram2d\"}], \"histogram2dcontour\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"histogram2dcontour\"}], \"mesh3d\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"mesh3d\"}], \"parcoords\": [{\"line\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"parcoords\"}], \"pie\": [{\"automargin\": true, \"type\": \"pie\"}], \"scatter\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatter\"}], \"scatter3d\": [{\"line\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatter3d\"}], \"scattercarpet\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattercarpet\"}], \"scattergeo\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattergeo\"}], \"scattergl\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattergl\"}], \"scattermapbox\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattermapbox\"}], \"scatterpolar\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterpolar\"}], \"scatterpolargl\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterpolargl\"}], \"scatterternary\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterternary\"}], \"surface\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"surface\"}], \"table\": [{\"cells\": {\"fill\": {\"color\": \"#EBF0F8\"}, \"line\": {\"color\": \"white\"}}, \"header\": {\"fill\": {\"color\": \"#C8D4E3\"}, \"line\": {\"color\": \"white\"}}, \"type\": \"table\"}]}, \"layout\": {\"annotationdefaults\": {\"arrowcolor\": \"#2a3f5f\", \"arrowhead\": 0, \"arrowwidth\": 1}, \"coloraxis\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"colorscale\": {\"diverging\": [[0, \"#8e0152\"], [0.1, \"#c51b7d\"], [0.2, \"#de77ae\"], [0.3, \"#f1b6da\"], [0.4, \"#fde0ef\"], [0.5, \"#f7f7f7\"], [0.6, \"#e6f5d0\"], [0.7, \"#b8e186\"], [0.8, \"#7fbc41\"], [0.9, \"#4d9221\"], [1, \"#276419\"]], \"sequential\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"sequentialminus\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]]}, \"colorway\": [\"#636efa\", \"#EF553B\", \"#00cc96\", \"#ab63fa\", \"#FFA15A\", \"#19d3f3\", \"#FF6692\", \"#B6E880\", \"#FF97FF\", \"#FECB52\"], \"font\": {\"color\": \"#2a3f5f\"}, \"geo\": {\"bgcolor\": \"white\", \"lakecolor\": \"white\", \"landcolor\": \"#E5ECF6\", \"showlakes\": true, \"showland\": true, \"subunitcolor\": \"white\"}, \"hoverlabel\": {\"align\": \"left\"}, \"hovermode\": \"closest\", \"mapbox\": {\"style\": \"light\"}, \"paper_bgcolor\": \"white\", \"plot_bgcolor\": \"#E5ECF6\", \"polar\": {\"angularaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"bgcolor\": \"#E5ECF6\", \"radialaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}}, \"scene\": {\"xaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}, \"yaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}, \"zaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}}, \"shapedefaults\": {\"line\": {\"color\": \"#2a3f5f\"}}, \"ternary\": {\"aaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"baxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"bgcolor\": \"#E5ECF6\", \"caxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}}, \"title\": {\"x\": 0.05}, \"xaxis\": {\"automargin\": true, \"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\", \"title\": {\"standoff\": 15}, \"zerolinecolor\": \"white\", \"zerolinewidth\": 2}, \"yaxis\": {\"automargin\": true, \"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\", \"title\": {\"standoff\": 15}, \"zerolinecolor\": \"white\", \"zerolinewidth\": 2}}}, \"width\": 800},\n",
" {\"responsive\": true}\n",
" ).then(function(){\n",
" \n",
"var gd = document.getElementById('a148e8b0-7fe2-4ac0-8de0-697ee0659cfe');\n",
"var x = new MutationObserver(function (mutations, observer) {{\n",
" var display = window.getComputedStyle(gd).display;\n",
" if (!display || display === 'none') {{\n",
" console.log([gd, 'removed!']);\n",
" Plotly.purge(gd);\n",
" observer.disconnect();\n",
" }}\n",
"}});\n",
"\n",
"// Listen for the removal of the full notebook cells\n",
"var notebookContainer = gd.closest('#notebook-container');\n",
"if (notebookContainer) {{\n",
" x.observe(notebookContainer, {childList: true});\n",
"}}\n",
"\n",
"// Listen for the clearing of the current output cell\n",
"var outputEl = gd.closest('.output');\n",
"if (outputEl) {{\n",
" x.observe(outputEl, {childList: true});\n",
"}}\n",
"\n",
" })\n",
" };\n",
" \n",
" </script>\n",
" </div>\n",
"</body>\n",
"</html>"
]
},
"metadata": {
"tags": []
}
}
]
}
]
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.