Navigation Menu

Skip to content

Instantly share code, notes, and snippets.

@kiwamizamurai

kiwamizamurai/optuna-tutorial.ipynb

Created November 20, 2019 09:33
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save kiwamizamurai/6a757e9f91f0c881a75a0475b6f1a22a to your computer and use it in GitHub Desktop.
Save kiwamizamurai/6a757e9f91f0c881a75a0475b6f1a22a to your computer and use it in GitHub Desktop.
Download ZIP
optuna-tutorial.ipynb
Display the source blob
Display the rendered blob
Raw
optuna-tutorial.ipynb
{
"nbformat": 4,
"nbformat_minor": 0,
"metadata": {
"colab": {
"name": "optuna-tutorial.ipynb",
"provenance": [],
"include_colab_link": true
},
"kernelspec": {
"name": "python3",
"display_name": "Python 3"
}
},
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "view-in-github",
"colab_type": "text"
},
"source": [
"<a href=\"https://colab.research.google.com/gist/kiwamizamurai/6a757e9f91f0c881a75a0475b6f1a22a/optuna-tutorial.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
]
},
{
"cell_type": "code",
"metadata": {
"id": "nP1yY6Ark6JK",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 911
},
"outputId": "06b582e8-1b98-4177-8716-2d65e694e1a0"
},
"source": [
"pip install optuna"
],
"execution_count": 2,
"outputs": [
{
"output_type": "stream",
"text": [
"Collecting optuna\n",
"\u001b[?25l Downloading https://files.pythonhosted.org/packages/d4/6a/4d80b3014797cf318a5252afb27031e9e7502854fb7930f27db0ee10bb75/optuna-0.19.0.tar.gz (126kB)\n",
"\u001b[K |████████████████████████████████| 133kB 2.8MB/s \n",
"\u001b[?25hCollecting alembic\n",
"\u001b[?25l Downloading https://files.pythonhosted.org/packages/84/64/493c45119dce700a4b9eeecc436ef9e8835ab67bae6414f040cdc7b58f4b/alembic-1.3.1.tar.gz (1.1MB)\n",
"\u001b[K |████████████████████████████████| 1.1MB 40.3MB/s \n",
"\u001b[?25hCollecting cliff\n",
"\u001b[?25l Downloading https://files.pythonhosted.org/packages/f6/a9/e976ba91e57043c4b6add2c394e6d1ffc26712c694379c3fe72f942d2440/cliff-2.16.0-py2.py3-none-any.whl (78kB)\n",
"\u001b[K |████████████████████████████████| 81kB 7.3MB/s \n",
"\u001b[?25hCollecting colorlog\n",
" Downloading https://files.pythonhosted.org/packages/68/4d/892728b0c14547224f0ac40884e722a3d00cb54e7a146aea0b3186806c9e/colorlog-4.0.2-py2.py3-none-any.whl\n",
"Requirement already satisfied: numpy in /usr/local/lib/python3.6/dist-packages (from optuna) (1.17.4)\n",
"Requirement already satisfied: scipy in /usr/local/lib/python3.6/dist-packages (from optuna) (1.3.2)\n",
"Requirement already satisfied: six in /usr/local/lib/python3.6/dist-packages (from optuna) (1.12.0)\n",
"Requirement already satisfied: sqlalchemy>=1.1.0 in /usr/local/lib/python3.6/dist-packages (from optuna) (1.3.11)\n",
"Requirement already satisfied: tqdm in /usr/local/lib/python3.6/dist-packages (from optuna) (4.28.1)\n",
"Requirement already satisfied: typing in /usr/local/lib/python3.6/dist-packages (from optuna) (3.6.6)\n",
"Collecting Mako\n",
"\u001b[?25l Downloading https://files.pythonhosted.org/packages/b0/3c/8dcd6883d009f7cae0f3157fb53e9afb05a0d3d33b3db1268ec2e6f4a56b/Mako-1.1.0.tar.gz (463kB)\n",
"\u001b[K |████████████████████████████████| 471kB 46.9MB/s \n",
"\u001b[?25hCollecting python-editor>=0.3\n",
" Downloading https://files.pythonhosted.org/packages/c6/d3/201fc3abe391bbae6606e6f1d598c15d367033332bd54352b12f35513717/python_editor-1.0.4-py3-none-any.whl\n",
"Requirement already satisfied: python-dateutil in /usr/local/lib/python3.6/dist-packages (from alembic->optuna) (2.6.1)\n",
"Collecting pbr!=2.1.0,>=2.0.0\n",
"\u001b[?25l Downloading https://files.pythonhosted.org/packages/46/a4/d5c83831a3452713e4b4f126149bc4fbda170f7cb16a86a00ce57ce0e9ad/pbr-5.4.3-py2.py3-none-any.whl (110kB)\n",
"\u001b[K |████████████████████████████████| 112kB 42.6MB/s \n",
"\u001b[?25hRequirement already satisfied: PyYAML>=3.12 in /usr/local/lib/python3.6/dist-packages (from cliff->optuna) (3.13)\n",
"Collecting stevedore>=1.20.0\n",
"\u001b[?25l Downloading https://files.pythonhosted.org/packages/b1/e1/f5ddbd83f60b03f522f173c03e406c1bff8343f0232a292ac96aa633b47a/stevedore-1.31.0-py2.py3-none-any.whl (43kB)\n",
"\u001b[K |████████████████████████████████| 51kB 6.5MB/s \n",
"\u001b[?25hRequirement already satisfied: PrettyTable<0.8,>=0.7.2 in /usr/local/lib/python3.6/dist-packages (from cliff->optuna) (0.7.2)\n",
"Requirement already satisfied: pyparsing>=2.1.0 in /usr/local/lib/python3.6/dist-packages (from cliff->optuna) (2.4.5)\n",
"Collecting cmd2!=0.8.3,<0.9.0,>=0.8.0\n",
"\u001b[?25l Downloading https://files.pythonhosted.org/packages/e9/40/a71caa2aaff10c73612a7106e2d35f693e85b8cf6e37ab0774274bca3cf9/cmd2-0.8.9-py2.py3-none-any.whl (53kB)\n",
"\u001b[K |████████████████████████████████| 61kB 7.7MB/s \n",
"\u001b[?25hRequirement already satisfied: MarkupSafe>=0.9.2 in /usr/local/lib/python3.6/dist-packages (from Mako->alembic->optuna) (1.1.1)\n",
"Collecting pyperclip\n",
" Downloading https://files.pythonhosted.org/packages/2d/0f/4eda562dffd085945d57c2d9a5da745cfb5228c02bc90f2c74bbac746243/pyperclip-1.7.0.tar.gz\n",
"Requirement already satisfied: wcwidth; sys_platform != \"win32\" in /usr/local/lib/python3.6/dist-packages (from cmd2!=0.8.3,<0.9.0,>=0.8.0->cliff->optuna) (0.1.7)\n",
"Building wheels for collected packages: optuna, alembic, Mako, pyperclip\n",
" Building wheel for optuna (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
" Created wheel for optuna: filename=optuna-0.19.0-cp36-none-any.whl size=170198 sha256=8ed4c03cc7da50906e1cc18e1cc9ef2584715125a70158e7f3ccd747d7f6a76a\n",
" Stored in directory: /root/.cache/pip/wheels/49/bf/47/090a43457caeff74397397da1c98a8aaed685257c16a5ba1f0\n",
" Building wheel for alembic (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
" Created wheel for alembic: filename=alembic-1.3.1-py2.py3-none-any.whl size=144523 sha256=21b943f063fad6d4b18c9693a1de9124f6c6f5e041fc25c7f2f1d8c5f8aecc31\n",
" Stored in directory: /root/.cache/pip/wheels/b2/d4/19/5ab879d30af7cbc79e6dcc1d421795b1aa9d78f455b0412ef7\n",
" Building wheel for Mako (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
" Created wheel for Mako: filename=Mako-1.1.0-cp36-none-any.whl size=75363 sha256=3f74c06c12c51f0f47591b21a9ca182ec500a6f3ab8c3dad954d3db498ed0194\n",
" Stored in directory: /root/.cache/pip/wheels/98/32/7b/a291926643fc1d1e02593e0d9e247c5a866a366b8343b7aa27\n",
" Building wheel for pyperclip (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
" Created wheel for pyperclip: filename=pyperclip-1.7.0-cp36-none-any.whl size=8359 sha256=91c816e04cbe30e3b00e6ce9269ff1b67cfb117ba53b58c4005bc91cb98741c1\n",
" Stored in directory: /root/.cache/pip/wheels/92/f0/ac/2ba2972034e98971c3654ece337ac61e546bdeb34ca960dc8c\n",
"Successfully built optuna alembic Mako pyperclip\n",
"Installing collected packages: Mako, python-editor, alembic, pbr, stevedore, pyperclip, cmd2, cliff, colorlog, optuna\n",
"Successfully installed Mako-1.1.0 alembic-1.3.1 cliff-2.16.0 cmd2-0.8.9 colorlog-4.0.2 optuna-0.19.0 pbr-5.4.3 pyperclip-1.7.0 python-editor-1.0.4 stevedore-1.31.0\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "otwP3vVAlFe2",
"colab_type": "code",
"colab": {}
},
"source": [
"import optuna"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "XmxLUImelMXK",
"colab_type": "code",
"colab": {}
},
"source": [
"def objective(trial):\n",
" x = trial.suggest_uniform('x', -10, 10)\n",
" return (x - 2) ** 2"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "07z0tcAclM6L",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 33
},
"outputId": "3a1fc99a-f043-41af-ccca-21038f9dacea"
},
"source": [
"study = optuna.create_study()\n",
"print(study)"
],
"execution_count": 25,
"outputs": [
{
"output_type": "stream",
"text": [
"<optuna.study.Study object at 0x7fca59f4eda0>\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "84kkyM_zlM8n",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 1000
},
"outputId": "12f27ac7-f0c3-4d12-b67e-c390d47452c5"
},
"source": [
"study.optimize(objective, n_trials=100)"
],
"execution_count": 26,
"outputs": [
{
"output_type": "stream",
"text": [
"\u001b[32m[I 2019-11-20 08:25:19,670]\u001b[0m Finished trial#0 resulted in value: 74.63568703296866. Current best value is 74.63568703296866 with parameters: {'x': -6.639194813926159}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:19,798]\u001b[0m Finished trial#1 resulted in value: 15.681661511136161. Current best value is 15.681661511136161 with parameters: {'x': 5.96000776654998}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:19,923]\u001b[0m Finished trial#2 resulted in value: 35.21874044795888. Current best value is 15.681661511136161 with parameters: {'x': 5.96000776654998}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:20,046]\u001b[0m Finished trial#3 resulted in value: 40.80644917799291. Current best value is 15.681661511136161 with parameters: {'x': 5.96000776654998}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:20,174]\u001b[0m Finished trial#4 resulted in value: 1.2212875319844767. Current best value is 1.2212875319844767 with parameters: {'x': 3.1051187863684504}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:20,300]\u001b[0m Finished trial#5 resulted in value: 0.14680026135230687. Current best value is 0.14680026135230687 with parameters: {'x': 2.3831452222751928}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:20,424]\u001b[0m Finished trial#6 resulted in value: 38.45009380192188. Current best value is 0.14680026135230687 with parameters: {'x': 2.3831452222751928}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:20,553]\u001b[0m Finished trial#7 resulted in value: 18.367776090877136. Current best value is 0.14680026135230687 with parameters: {'x': 2.3831452222751928}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:20,680]\u001b[0m Finished trial#8 resulted in value: 0.22413025949470658. Current best value is 0.14680026135230687 with parameters: {'x': 2.3831452222751928}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:20,809]\u001b[0m Finished trial#9 resulted in value: 11.211500494474965. Current best value is 0.14680026135230687 with parameters: {'x': 2.3831452222751928}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:20,964]\u001b[0m Finished trial#10 resulted in value: 7.594026511633403. Current best value is 0.14680026135230687 with parameters: {'x': 2.3831452222751928}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:21,091]\u001b[0m Finished trial#11 resulted in value: 0.8532783217448091. Current best value is 0.14680026135230687 with parameters: {'x': 2.3831452222751928}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:21,222]\u001b[0m Finished trial#12 resulted in value: 3.7413028343127097. Current best value is 0.14680026135230687 with parameters: {'x': 2.3831452222751928}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:21,350]\u001b[0m Finished trial#13 resulted in value: 0.7482129244912662. Current best value is 0.14680026135230687 with parameters: {'x': 2.3831452222751928}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:21,477]\u001b[0m Finished trial#14 resulted in value: 60.22366933936629. Current best value is 0.14680026135230687 with parameters: {'x': 2.3831452222751928}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:21,601]\u001b[0m Finished trial#15 resulted in value: 12.314725194926563. Current best value is 0.14680026135230687 with parameters: {'x': 2.3831452222751928}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:21,731]\u001b[0m Finished trial#16 resulted in value: 1.018223944123022. Current best value is 0.14680026135230687 with parameters: {'x': 2.3831452222751928}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:21,858]\u001b[0m Finished trial#17 resulted in value: 22.342348897163838. Current best value is 0.14680026135230687 with parameters: {'x': 2.3831452222751928}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:21,982]\u001b[0m Finished trial#18 resulted in value: 0.34239323314179865. Current best value is 0.14680026135230687 with parameters: {'x': 2.3831452222751928}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:22,106]\u001b[0m Finished trial#19 resulted in value: 134.49597666010115. Current best value is 0.14680026135230687 with parameters: {'x': 2.3831452222751928}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:22,237]\u001b[0m Finished trial#20 resulted in value: 5.828729041741306. Current best value is 0.14680026135230687 with parameters: {'x': 2.3831452222751928}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:22,378]\u001b[0m Finished trial#21 resulted in value: 1.2170748396699367. Current best value is 0.14680026135230687 with parameters: {'x': 2.3831452222751928}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:22,507]\u001b[0m Finished trial#22 resulted in value: 0.028330901609411676. Current best value is 0.028330901609411676 with parameters: {'x': 1.8316821411453565}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:22,631]\u001b[0m Finished trial#23 resulted in value: 24.99657789886226. Current best value is 0.028330901609411676 with parameters: {'x': 1.8316821411453565}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:22,755]\u001b[0m Finished trial#24 resulted in value: 5.689787583119265. Current best value is 0.028330901609411676 with parameters: {'x': 1.8316821411453565}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:22,879]\u001b[0m Finished trial#25 resulted in value: 0.12056826311217518. Current best value is 0.028330901609411676 with parameters: {'x': 1.8316821411453565}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:23,004]\u001b[0m Finished trial#26 resulted in value: 13.823020601473281. Current best value is 0.028330901609411676 with parameters: {'x': 1.8316821411453565}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:23,125]\u001b[0m Finished trial#27 resulted in value: 5.807173520270313. Current best value is 0.028330901609411676 with parameters: {'x': 1.8316821411453565}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:23,255]\u001b[0m Finished trial#28 resulted in value: 33.0798739425265. Current best value is 0.028330901609411676 with parameters: {'x': 1.8316821411453565}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:23,380]\u001b[0m Finished trial#29 resulted in value: 78.21500910384827. Current best value is 0.028330901609411676 with parameters: {'x': 1.8316821411453565}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:23,505]\u001b[0m Finished trial#30 resulted in value: 0.259312297720867. Current best value is 0.028330901609411676 with parameters: {'x': 1.8316821411453565}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:23,636]\u001b[0m Finished trial#31 resulted in value: 0.12200126346631915. Current best value is 0.028330901609411676 with parameters: {'x': 1.8316821411453565}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:23,767]\u001b[0m Finished trial#32 resulted in value: 5.200258477183383. Current best value is 0.028330901609411676 with parameters: {'x': 1.8316821411453565}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:23,894]\u001b[0m Finished trial#33 resulted in value: 0.013703612725368826. Current best value is 0.013703612725368826 with parameters: {'x': 2.1170624308878336}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:24,022]\u001b[0m Finished trial#34 resulted in value: 4.887384243129526. Current best value is 0.013703612725368826 with parameters: {'x': 2.1170624308878336}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:24,152]\u001b[0m Finished trial#35 resulted in value: 3.5389872697319937. Current best value is 0.013703612725368826 with parameters: {'x': 2.1170624308878336}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:24,279]\u001b[0m Finished trial#36 resulted in value: 12.27366888693429. Current best value is 0.013703612725368826 with parameters: {'x': 2.1170624308878336}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:24,409]\u001b[0m Finished trial#37 resulted in value: 0.0036732116343290485. Current best value is 0.0036732116343290485 with parameters: {'x': 1.9393929737214484}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:24,538]\u001b[0m Finished trial#38 resulted in value: 27.915939632915578. Current best value is 0.0036732116343290485 with parameters: {'x': 1.9393929737214484}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:24,669]\u001b[0m Finished trial#39 resulted in value: 2.3415683229678104. Current best value is 0.0036732116343290485 with parameters: {'x': 1.9393929737214484}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:24,795]\u001b[0m Finished trial#40 resulted in value: 52.03685409253796. Current best value is 0.0036732116343290485 with parameters: {'x': 1.9393929737214484}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:24,921]\u001b[0m Finished trial#41 resulted in value: 0.005122558245931236. Current best value is 0.0036732116343290485 with parameters: {'x': 1.9393929737214484}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:25,046]\u001b[0m Finished trial#42 resulted in value: 0.0015175934436004658. Current best value is 0.0015175934436004658 with parameters: {'x': 1.9610436982812733}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:25,173]\u001b[0m Finished trial#43 resulted in value: 0.02308619170169228. Current best value is 0.0015175934436004658 with parameters: {'x': 1.9610436982812733}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:25,299]\u001b[0m Finished trial#44 resulted in value: 2.071392063389823. Current best value is 0.0015175934436004658 with parameters: {'x': 1.9610436982812733}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:25,428]\u001b[0m Finished trial#45 resulted in value: 9.80195215569949. Current best value is 0.0015175934436004658 with parameters: {'x': 1.9610436982812733}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:25,556]\u001b[0m Finished trial#46 resulted in value: 3.5803904571960037. Current best value is 0.0015175934436004658 with parameters: {'x': 1.9610436982812733}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:25,681]\u001b[0m Finished trial#47 resulted in value: 9.857502895307222. Current best value is 0.0015175934436004658 with parameters: {'x': 1.9610436982812733}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:25,812]\u001b[0m Finished trial#48 resulted in value: 0.0030220228615943467. Current best value is 0.0015175934436004658 with parameters: {'x': 1.9610436982812733}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:25,939]\u001b[0m Finished trial#49 resulted in value: 17.87374821427753. Current best value is 0.0015175934436004658 with parameters: {'x': 1.9610436982812733}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:26,069]\u001b[0m Finished trial#50 resulted in value: 7.20536591581677. Current best value is 0.0015175934436004658 with parameters: {'x': 1.9610436982812733}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:26,198]\u001b[0m Finished trial#51 resulted in value: 0.04888118667602236. Current best value is 0.0015175934436004658 with parameters: {'x': 1.9610436982812733}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:26,325]\u001b[0m Finished trial#52 resulted in value: 1.6398329120457928. Current best value is 0.0015175934436004658 with parameters: {'x': 1.9610436982812733}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:26,453]\u001b[0m Finished trial#53 resulted in value: 1.8739831645513858. Current best value is 0.0015175934436004658 with parameters: {'x': 1.9610436982812733}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:26,581]\u001b[0m Finished trial#54 resulted in value: 0.5837015340809495. Current best value is 0.0015175934436004658 with parameters: {'x': 1.9610436982812733}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:26,707]\u001b[0m Finished trial#55 resulted in value: 0.002740901659309055. Current best value is 0.0015175934436004658 with parameters: {'x': 1.9610436982812733}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:26,831]\u001b[0m Finished trial#56 resulted in value: 1.3038362801828842. Current best value is 0.0015175934436004658 with parameters: {'x': 1.9610436982812733}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:26,959]\u001b[0m Finished trial#57 resulted in value: 1.7068323055099774. Current best value is 0.0015175934436004658 with parameters: {'x': 1.9610436982812733}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:27,086]\u001b[0m Finished trial#58 resulted in value: 4.502892563805704. Current best value is 0.0015175934436004658 with parameters: {'x': 1.9610436982812733}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:27,220]\u001b[0m Finished trial#59 resulted in value: 0.018832109155707495. Current best value is 0.0015175934436004658 with parameters: {'x': 1.9610436982812733}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:27,350]\u001b[0m Finished trial#60 resulted in value: 0.6736067095875267. Current best value is 0.0015175934436004658 with parameters: {'x': 1.9610436982812733}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:27,480]\u001b[0m Finished trial#61 resulted in value: 0.02156872818392232. Current best value is 0.0015175934436004658 with parameters: {'x': 1.9610436982812733}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:27,609]\u001b[0m Finished trial#62 resulted in value: 0.5372027310193397. Current best value is 0.0015175934436004658 with parameters: {'x': 1.9610436982812733}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:27,736]\u001b[0m Finished trial#63 resulted in value: 9.54043835351742. Current best value is 0.0015175934436004658 with parameters: {'x': 1.9610436982812733}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:27,867]\u001b[0m Finished trial#64 resulted in value: 18.82948057673564. Current best value is 0.0015175934436004658 with parameters: {'x': 1.9610436982812733}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:27,994]\u001b[0m Finished trial#65 resulted in value: 2.800061910268745. Current best value is 0.0015175934436004658 with parameters: {'x': 1.9610436982812733}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:28,122]\u001b[0m Finished trial#66 resulted in value: 0.6122505880055049. Current best value is 0.0015175934436004658 with parameters: {'x': 1.9610436982812733}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:28,251]\u001b[0m Finished trial#67 resulted in value: 3.1017288130935317. Current best value is 0.0015175934436004658 with parameters: {'x': 1.9610436982812733}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:28,380]\u001b[0m Finished trial#68 resulted in value: 0.1348552768751693. Current best value is 0.0015175934436004658 with parameters: {'x': 1.9610436982812733}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:28,510]\u001b[0m Finished trial#69 resulted in value: 0.1749702679547182. Current best value is 0.0015175934436004658 with parameters: {'x': 1.9610436982812733}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:28,643]\u001b[0m Finished trial#70 resulted in value: 0.8985187195128104. Current best value is 0.0015175934436004658 with parameters: {'x': 1.9610436982812733}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:28,769]\u001b[0m Finished trial#71 resulted in value: 0.07148757033510479. Current best value is 0.0015175934436004658 with parameters: {'x': 1.9610436982812733}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:28,896]\u001b[0m Finished trial#72 resulted in value: 0.00044389015974030644. Current best value is 0.00044389015974030644 with parameters: {'x': 2.0210687009504693}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:29,022]\u001b[0m Finished trial#73 resulted in value: 7.369472360192736. Current best value is 0.00044389015974030644 with parameters: {'x': 2.0210687009504693}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:29,146]\u001b[0m Finished trial#74 resulted in value: 7.375726767614576. Current best value is 0.00044389015974030644 with parameters: {'x': 2.0210687009504693}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:29,282]\u001b[0m Finished trial#75 resulted in value: 0.8727071514675215. Current best value is 0.00044389015974030644 with parameters: {'x': 2.0210687009504693}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:29,414]\u001b[0m Finished trial#76 resulted in value: 0.010382044004186347. Current best value is 0.00044389015974030644 with parameters: {'x': 2.0210687009504693}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:29,543]\u001b[0m Finished trial#77 resulted in value: 2.635184403387668. Current best value is 0.00044389015974030644 with parameters: {'x': 2.0210687009504693}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:29,672]\u001b[0m Finished trial#78 resulted in value: 2.9782091678318325. Current best value is 0.00044389015974030644 with parameters: {'x': 2.0210687009504693}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:29,800]\u001b[0m Finished trial#79 resulted in value: 4.729113750534535. Current best value is 0.00044389015974030644 with parameters: {'x': 2.0210687009504693}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:29,931]\u001b[0m Finished trial#80 resulted in value: 0.24403482002036858. Current best value is 0.00044389015974030644 with parameters: {'x': 2.0210687009504693}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:30,063]\u001b[0m Finished trial#81 resulted in value: 0.5486926135394343. Current best value is 0.00044389015974030644 with parameters: {'x': 2.0210687009504693}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:30,194]\u001b[0m Finished trial#82 resulted in value: 0.00038306803768869984. Current best value is 0.00038306803768869984 with parameters: {'x': 2.0195721239953333}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:30,327]\u001b[0m Finished trial#83 resulted in value: 0.0027532224013859462. Current best value is 0.00038306803768869984 with parameters: {'x': 2.0195721239953333}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:30,459]\u001b[0m Finished trial#84 resulted in value: 1.1783329603299877. Current best value is 0.00038306803768869984 with parameters: {'x': 2.0195721239953333}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:30,590]\u001b[0m Finished trial#85 resulted in value: 0.11919566653881611. Current best value is 0.00038306803768869984 with parameters: {'x': 2.0195721239953333}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:30,721]\u001b[0m Finished trial#86 resulted in value: 1.6160801124778839. Current best value is 0.00038306803768869984 with parameters: {'x': 2.0195721239953333}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:30,849]\u001b[0m Finished trial#87 resulted in value: 4.379138035153702. Current best value is 0.00038306803768869984 with parameters: {'x': 2.0195721239953333}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:30,980]\u001b[0m Finished trial#88 resulted in value: 0.3957440711432687. Current best value is 0.00038306803768869984 with parameters: {'x': 2.0195721239953333}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:31,118]\u001b[0m Finished trial#89 resulted in value: 10.860223820880375. Current best value is 0.00038306803768869984 with parameters: {'x': 2.0195721239953333}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:31,254]\u001b[0m Finished trial#90 resulted in value: 1.8220964007163718. Current best value is 0.00038306803768869984 with parameters: {'x': 2.0195721239953333}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:31,391]\u001b[0m Finished trial#91 resulted in value: 0.004661277092706195. Current best value is 0.00038306803768869984 with parameters: {'x': 2.0195721239953333}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:31,518]\u001b[0m Finished trial#92 resulted in value: 0.013725422022546807. Current best value is 0.00038306803768869984 with parameters: {'x': 2.0195721239953333}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:31,648]\u001b[0m Finished trial#93 resulted in value: 1.366403346908549. Current best value is 0.00038306803768869984 with parameters: {'x': 2.0195721239953333}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:31,777]\u001b[0m Finished trial#94 resulted in value: 0.03291340176895431. Current best value is 0.00038306803768869984 with parameters: {'x': 2.0195721239953333}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:31,905]\u001b[0m Finished trial#95 resulted in value: 0.6909278677197279. Current best value is 0.00038306803768869984 with parameters: {'x': 2.0195721239953333}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:32,035]\u001b[0m Finished trial#96 resulted in value: 5.633426645824235. Current best value is 0.00038306803768869984 with parameters: {'x': 2.0195721239953333}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:32,169]\u001b[0m Finished trial#97 resulted in value: 2.8777104748700775. Current best value is 0.00038306803768869984 with parameters: {'x': 2.0195721239953333}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:32,301]\u001b[0m Finished trial#98 resulted in value: 0.2727919794867617. Current best value is 0.00038306803768869984 with parameters: {'x': 2.0195721239953333}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:25:32,434]\u001b[0m Finished trial#99 resulted in value: 2.566803190789853. Current best value is 0.00038306803768869984 with parameters: {'x': 2.0195721239953333}.\u001b[0m\n"
],
"name": "stderr"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "CVzh7xF56jjT",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 542
},
"outputId": "923152f2-13c6-44f1-9929-9c1085e8eb1f"
},
"source": [
"from optuna.visualization import plot_optimization_history\n",
"plot_optimization_history(study)"
],
"execution_count": 28,
"outputs": [
{
"output_type": "display_data",
"data": {
"text/html": [
"<html>\n",
"<head><meta charset=\"utf-8\" /></head>\n",
"<body>\n",
" <div>\n",
" <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax) {MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>\n",
" <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
" <script src=\"https://cdn.plot.ly/plotly-latest.min.js\"></script> \n",
" <div id=\"6df5ac49-875a-4bda-af44-aceabfd648c1\" class=\"plotly-graph-div\" style=\"height:525px; width:100%;\"></div>\n",
" <script type=\"text/javascript\">\n",
" \n",
" window.PLOTLYENV=window.PLOTLYENV || {};\n",
" \n",
" if (document.getElementById(\"6df5ac49-875a-4bda-af44-aceabfd648c1\")) {\n",
" Plotly.newPlot(\n",
" '6df5ac49-875a-4bda-af44-aceabfd648c1',\n",
" [{\"mode\": \"markers\", \"name\": \"Objective Value\", \"type\": \"scatter\", \"x\": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99], \"y\": [74.63568703296866, 15.681661511136161, 35.21874044795888, 40.80644917799291, 1.2212875319844767, 0.14680026135230687, 38.45009380192188, 18.367776090877136, 0.22413025949470658, 11.211500494474965, 7.594026511633403, 0.8532783217448091, 3.7413028343127097, 0.7482129244912662, 60.22366933936629, 12.314725194926563, 1.018223944123022, 22.342348897163838, 0.34239323314179865, 134.49597666010115, 5.828729041741306, 1.2170748396699367, 0.028330901609411676, 24.99657789886226, 5.689787583119265, 0.12056826311217518, 13.823020601473281, 5.807173520270313, 33.0798739425265, 78.21500910384827, 0.259312297720867, 0.12200126346631915, 5.200258477183383, 0.013703612725368826, 4.887384243129526, 3.5389872697319937, 12.27366888693429, 0.0036732116343290485, 27.915939632915578, 2.3415683229678104, 52.03685409253796, 0.005122558245931236, 0.0015175934436004658, 0.02308619170169228, 2.071392063389823, 9.80195215569949, 3.5803904571960037, 9.857502895307222, 0.0030220228615943467, 17.87374821427753, 7.20536591581677, 0.04888118667602236, 1.6398329120457928, 1.8739831645513858, 0.5837015340809495, 0.002740901659309055, 1.3038362801828842, 1.7068323055099774, 4.502892563805704, 0.018832109155707495, 0.6736067095875267, 0.02156872818392232, 0.5372027310193397, 9.54043835351742, 18.82948057673564, 2.800061910268745, 0.6122505880055049, 3.1017288130935317, 0.1348552768751693, 0.1749702679547182, 0.8985187195128104, 0.07148757033510479, 0.00044389015974030644, 7.369472360192736, 7.375726767614576, 0.8727071514675215, 0.010382044004186347, 2.635184403387668, 2.9782091678318325, 4.729113750534535, 0.24403482002036858, 0.5486926135394343, 0.00038306803768869984, 0.0027532224013859462, 1.1783329603299877, 0.11919566653881611, 1.6160801124778839, 4.379138035153702, 0.3957440711432687, 10.860223820880375, 1.8220964007163718, 0.004661277092706195, 0.013725422022546807, 1.366403346908549, 0.03291340176895431, 0.6909278677197279, 5.633426645824235, 2.8777104748700775, 0.2727919794867617, 2.566803190789853]}, {\"name\": \"Best Value\", \"type\": \"scatter\", \"x\": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99], \"y\": [74.63568703296866, 15.681661511136161, 15.681661511136161, 15.681661511136161, 1.2212875319844767, 0.14680026135230687, 0.14680026135230687, 0.14680026135230687, 0.14680026135230687, 0.14680026135230687, 0.14680026135230687, 0.14680026135230687, 0.14680026135230687, 0.14680026135230687, 0.14680026135230687, 0.14680026135230687, 0.14680026135230687, 0.14680026135230687, 0.14680026135230687, 0.14680026135230687, 0.14680026135230687, 0.14680026135230687, 0.028330901609411676, 0.028330901609411676, 0.028330901609411676, 0.028330901609411676, 0.028330901609411676, 0.028330901609411676, 0.028330901609411676, 0.028330901609411676, 0.028330901609411676, 0.028330901609411676, 0.028330901609411676, 0.013703612725368826, 0.013703612725368826, 0.013703612725368826, 0.013703612725368826, 0.0036732116343290485, 0.0036732116343290485, 0.0036732116343290485, 0.0036732116343290485, 0.0036732116343290485, 0.0015175934436004658, 0.0015175934436004658, 0.0015175934436004658, 0.0015175934436004658, 0.0015175934436004658, 0.0015175934436004658, 0.0015175934436004658, 0.0015175934436004658, 0.0015175934436004658, 0.0015175934436004658, 0.0015175934436004658, 0.0015175934436004658, 0.0015175934436004658, 0.0015175934436004658, 0.0015175934436004658, 0.0015175934436004658, 0.0015175934436004658, 0.0015175934436004658, 0.0015175934436004658, 0.0015175934436004658, 0.0015175934436004658, 0.0015175934436004658, 0.0015175934436004658, 0.0015175934436004658, 0.0015175934436004658, 0.0015175934436004658, 0.0015175934436004658, 0.0015175934436004658, 0.0015175934436004658, 0.0015175934436004658, 0.00044389015974030644, 0.00044389015974030644, 0.00044389015974030644, 0.00044389015974030644, 0.00044389015974030644, 0.00044389015974030644, 0.00044389015974030644, 0.00044389015974030644, 0.00044389015974030644, 0.00044389015974030644, 0.00038306803768869984, 0.00038306803768869984, 0.00038306803768869984, 0.00038306803768869984, 0.00038306803768869984, 0.00038306803768869984, 0.00038306803768869984, 0.00038306803768869984, 0.00038306803768869984, 0.00038306803768869984, 0.00038306803768869984, 0.00038306803768869984, 0.00038306803768869984, 0.00038306803768869984, 0.00038306803768869984, 0.00038306803768869984, 0.00038306803768869984, 0.00038306803768869984]}],\n",
" {\"template\": {\"data\": {\"bar\": [{\"error_x\": {\"color\": \"#2a3f5f\"}, \"error_y\": {\"color\": \"#2a3f5f\"}, \"marker\": {\"line\": {\"color\": \"#E5ECF6\", \"width\": 0.5}}, \"type\": \"bar\"}], \"barpolar\": [{\"marker\": {\"line\": {\"color\": \"#E5ECF6\", \"width\": 0.5}}, \"type\": \"barpolar\"}], \"carpet\": [{\"aaxis\": {\"endlinecolor\": \"#2a3f5f\", \"gridcolor\": \"white\", \"linecolor\": \"white\", \"minorgridcolor\": \"white\", \"startlinecolor\": \"#2a3f5f\"}, \"baxis\": {\"endlinecolor\": \"#2a3f5f\", \"gridcolor\": \"white\", \"linecolor\": \"white\", \"minorgridcolor\": \"white\", \"startlinecolor\": \"#2a3f5f\"}, \"type\": \"carpet\"}], \"choropleth\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"choropleth\"}], \"contour\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"contour\"}], \"contourcarpet\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"contourcarpet\"}], \"heatmap\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"heatmap\"}], \"heatmapgl\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"heatmapgl\"}], \"histogram\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"histogram\"}], \"histogram2d\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"histogram2d\"}], \"histogram2dcontour\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"histogram2dcontour\"}], \"mesh3d\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"mesh3d\"}], \"parcoords\": [{\"line\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"parcoords\"}], \"scatter\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatter\"}], \"scatter3d\": [{\"line\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatter3d\"}], \"scattercarpet\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattercarpet\"}], \"scattergeo\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattergeo\"}], \"scattergl\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattergl\"}], \"scattermapbox\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattermapbox\"}], \"scatterpolar\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterpolar\"}], \"scatterpolargl\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterpolargl\"}], \"scatterternary\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterternary\"}], \"surface\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"surface\"}], \"table\": [{\"cells\": {\"fill\": {\"color\": \"#EBF0F8\"}, \"line\": {\"color\": \"white\"}}, \"header\": {\"fill\": {\"color\": \"#C8D4E3\"}, \"line\": {\"color\": \"white\"}}, \"type\": \"table\"}]}, \"layout\": {\"annotationdefaults\": {\"arrowcolor\": \"#2a3f5f\", \"arrowhead\": 0, \"arrowwidth\": 1}, \"colorscale\": {\"diverging\": [[0, \"#8e0152\"], [0.1, \"#c51b7d\"], [0.2, \"#de77ae\"], [0.3, \"#f1b6da\"], [0.4, \"#fde0ef\"], [0.5, \"#f7f7f7\"], [0.6, \"#e6f5d0\"], [0.7, \"#b8e186\"], [0.8, \"#7fbc41\"], [0.9, \"#4d9221\"], [1, \"#276419\"]], \"sequential\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"sequentialminus\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]]}, \"colorway\": [\"#636efa\", \"#EF553B\", \"#00cc96\", \"#ab63fa\", \"#FFA15A\", \"#19d3f3\", \"#FF6692\", \"#B6E880\", \"#FF97FF\", \"#FECB52\"], \"font\": {\"color\": \"#2a3f5f\"}, \"geo\": {\"bgcolor\": \"white\", \"lakecolor\": \"white\", \"landcolor\": \"#E5ECF6\", \"showlakes\": true, \"showland\": true, \"subunitcolor\": \"white\"}, \"hoverlabel\": {\"align\": \"left\"}, \"hovermode\": \"closest\", \"mapbox\": {\"style\": \"light\"}, \"paper_bgcolor\": \"white\", \"plot_bgcolor\": \"#E5ECF6\", \"polar\": {\"angularaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"bgcolor\": \"#E5ECF6\", \"radialaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}}, \"scene\": {\"xaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}, \"yaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}, \"zaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}}, \"shapedefaults\": {\"line\": {\"color\": \"#2a3f5f\"}}, \"ternary\": {\"aaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"baxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"bgcolor\": \"#E5ECF6\", \"caxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}}, \"title\": {\"x\": 0.05}, \"xaxis\": {\"automargin\": true, \"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\", \"zerolinecolor\": \"white\", \"zerolinewidth\": 2}, \"yaxis\": {\"automargin\": true, \"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\", \"zerolinecolor\": \"white\", \"zerolinewidth\": 2}}}, \"title\": {\"text\": \"Optimization History Plot\"}, \"xaxis\": {\"title\": {\"text\": \"#Trials\"}}, \"yaxis\": {\"title\": {\"text\": \"Objective Value\"}}},\n",
" {\"responsive\": true}\n",
" ).then(function(){\n",
" \n",
"var gd = document.getElementById('6df5ac49-875a-4bda-af44-aceabfd648c1');\n",
"var x = new MutationObserver(function (mutations, observer) {{\n",
" var display = window.getComputedStyle(gd).display;\n",
" if (!display || display === 'none') {{\n",
" console.log([gd, 'removed!']);\n",
" Plotly.purge(gd);\n",
" observer.disconnect();\n",
" }}\n",
"}});\n",
"\n",
"// Listen for the removal of the full notebook cells\n",
"var notebookContainer = gd.closest('#notebook-container');\n",
"if (notebookContainer) {{\n",
" x.observe(notebookContainer, {childList: true});\n",
"}}\n",
"\n",
"// Listen for the clearing of the current output cell\n",
"var outputEl = gd.closest('.output');\n",
"if (outputEl) {{\n",
" x.observe(outputEl, {childList: true});\n",
"}}\n",
"\n",
" })\n",
" };\n",
" \n",
" </script>\n",
" </div>\n",
"</body>\n",
"</html>"
]
},
"metadata": {
"tags": []
}
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "2aARicx96juz",
"colab_type": "code",
"colab": {}
},
"source": [
"from optuna.visualization import plot_intermediate_values"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "IlSiSZJslM_O",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 33
},
"outputId": "65dd66cb-2db0-4e4a-9901-7b3f81f957c6"
},
"source": [
"study.best_params"
],
"execution_count": 31,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"{'x': 2.0195721239953333}"
]
},
"metadata": {
"tags": []
},
"execution_count": 31
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "8j35NhxAlNBq",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 33
},
"outputId": "7da505b0-9379-4812-d715-f7425dea9ce7"
},
"source": [
"study.best_value"
],
"execution_count": 32,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"0.00038306803768869984"
]
},
"metadata": {
"tags": []
},
"execution_count": 32
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "OphziSLhl3Zu",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 53
},
"outputId": "3e6e73f2-17d9-4163-d26e-8f1be87d9020"
},
"source": [
"study.best_trial"
],
"execution_count": 33,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"FrozenTrial(number=82, state=TrialState.COMPLETE, value=0.00038306803768869984, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 30, 64038), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 30, 194203), params={'x': 2.0195721239953333}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 82}, intermediate_values={}, trial_id=82)"
]
},
"metadata": {
"tags": []
},
"execution_count": 33
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "-66Txf7TlxmD",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 1000
},
"outputId": "0c54de36-126e-4f20-8e06-f1c1c63c68e9"
},
"source": [
"study.trials"
],
"execution_count": 34,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"[FrozenTrial(number=0, state=TrialState.COMPLETE, value=74.63568703296866, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 19, 513711), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 19, 670522), params={'x': -6.639194813926159}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 0}, intermediate_values={}, trial_id=0),\n",
" FrozenTrial(number=1, state=TrialState.COMPLETE, value=15.681661511136161, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 19, 672205), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 19, 797744), params={'x': 5.96000776654998}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 1}, intermediate_values={}, trial_id=1),\n",
" FrozenTrial(number=2, state=TrialState.COMPLETE, value=35.21874044795888, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 19, 799281), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 19, 923551), params={'x': -3.9345379304507677}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 2}, intermediate_values={}, trial_id=2),\n",
" FrozenTrial(number=3, state=TrialState.COMPLETE, value=40.80644917799291, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 19, 924865), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 20, 45960), params={'x': 8.38799257811035}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 3}, intermediate_values={}, trial_id=3),\n",
" FrozenTrial(number=4, state=TrialState.COMPLETE, value=1.2212875319844767, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 20, 47188), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 20, 174085), params={'x': 3.1051187863684504}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 4}, intermediate_values={}, trial_id=4),\n",
" FrozenTrial(number=5, state=TrialState.COMPLETE, value=0.14680026135230687, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 20, 175457), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 20, 300427), params={'x': 2.3831452222751928}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 5}, intermediate_values={}, trial_id=5),\n",
" FrozenTrial(number=6, state=TrialState.COMPLETE, value=38.45009380192188, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 20, 301767), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 20, 424575), params={'x': -4.200813962853738}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 6}, intermediate_values={}, trial_id=6),\n",
" FrozenTrial(number=7, state=TrialState.COMPLETE, value=18.367776090877136, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 20, 426832), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 20, 553349), params={'x': 6.285764353167021}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 7}, intermediate_values={}, trial_id=7),\n",
" FrozenTrial(number=8, state=TrialState.COMPLETE, value=0.22413025949470658, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 20, 554718), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 20, 679749), params={'x': 2.473423974355658}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 8}, intermediate_values={}, trial_id=8),\n",
" FrozenTrial(number=9, state=TrialState.COMPLETE, value=11.211500494474965, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 20, 682090), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 20, 809346), params={'x': 5.34835788028624}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 9}, intermediate_values={}, trial_id=9),\n",
" FrozenTrial(number=10, state=TrialState.COMPLETE, value=7.594026511633403, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 20, 810856), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 20, 964572), params={'x': -0.7557261314639745}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 10}, intermediate_values={}, trial_id=10),\n",
" FrozenTrial(number=11, state=TrialState.COMPLETE, value=0.8532783217448091, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 20, 966082), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 21, 91492), params={'x': 1.0762693456722305}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 11}, intermediate_values={}, trial_id=11),\n",
" FrozenTrial(number=12, state=TrialState.COMPLETE, value=3.7413028343127097, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 21, 92897), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 21, 221820), params={'x': 0.06575522895557251}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 12}, intermediate_values={}, trial_id=12),\n",
" FrozenTrial(number=13, state=TrialState.COMPLETE, value=0.7482129244912662, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 21, 223539), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 21, 350497), params={'x': 2.8649930199089853}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 13}, intermediate_values={}, trial_id=13),\n",
" FrozenTrial(number=14, state=TrialState.COMPLETE, value=60.22366933936629, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 21, 351875), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 21, 477748), params={'x': 9.760391055827425}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 14}, intermediate_values={}, trial_id=14),\n",
" FrozenTrial(number=15, state=TrialState.COMPLETE, value=12.314725194926563, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 21, 478961), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 21, 601087), params={'x': -1.5092342747281156}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 15}, intermediate_values={}, trial_id=15),\n",
" FrozenTrial(number=16, state=TrialState.COMPLETE, value=1.018223944123022, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 21, 602702), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 21, 731145), params={'x': 3.0090708320643413}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 16}, intermediate_values={}, trial_id=16),\n",
" FrozenTrial(number=17, state=TrialState.COMPLETE, value=22.342348897163838, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 21, 732767), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 21, 858400), params={'x': -2.7267693932710357}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 17}, intermediate_values={}, trial_id=17),\n",
" FrozenTrial(number=18, state=TrialState.COMPLETE, value=0.34239323314179865, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 21, 859583), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 21, 981769), params={'x': 1.4148562286567525}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 18}, intermediate_values={}, trial_id=18),\n",
" FrozenTrial(number=19, state=TrialState.COMPLETE, value=134.49597666010115, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 21, 983103), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 22, 106015), params={'x': -9.597240044946089}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 19}, intermediate_values={}, trial_id=19),\n",
" FrozenTrial(number=20, state=TrialState.COMPLETE, value=5.828729041741306, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 22, 107268), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 22, 236878), params={'x': 4.414276090620397}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 20}, intermediate_values={}, trial_id=20),\n",
" FrozenTrial(number=21, state=TrialState.COMPLETE, value=1.2170748396699367, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 22, 238035), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 22, 378195), params={'x': 0.8967888508223203}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 21}, intermediate_values={}, trial_id=21),\n",
" FrozenTrial(number=22, state=TrialState.COMPLETE, value=0.028330901609411676, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 22, 379434), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 22, 507609), params={'x': 1.8316821411453565}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 22}, intermediate_values={}, trial_id=22),\n",
" FrozenTrial(number=23, state=TrialState.COMPLETE, value=24.99657789886226, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 22, 508793), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 22, 631065), params={'x': 6.999657778174648}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 23}, intermediate_values={}, trial_id=23),\n",
" FrozenTrial(number=24, state=TrialState.COMPLETE, value=5.689787583119265, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 22, 632413), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 22, 755128), params={'x': 4.3853275630653465}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 24}, intermediate_values={}, trial_id=24),\n",
" FrozenTrial(number=25, state=TrialState.COMPLETE, value=0.12056826311217518, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 22, 756567), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 22, 878815), params={'x': 2.3472294099182487}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 25}, intermediate_values={}, trial_id=25),\n",
" FrozenTrial(number=26, state=TrialState.COMPLETE, value=13.823020601473281, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 22, 880134), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 23, 4077), params={'x': -1.7179323018948693}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 26}, intermediate_values={}, trial_id=26),\n",
" FrozenTrial(number=27, state=TrialState.COMPLETE, value=5.807173520270313, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 23, 5423), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 23, 125095), params={'x': 4.4098077766225074}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 27}, intermediate_values={}, trial_id=27),\n",
" FrozenTrial(number=28, state=TrialState.COMPLETE, value=33.0798739425265, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 23, 126411), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 23, 254896), params={'x': 7.751510579189305}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 28}, intermediate_values={}, trial_id=28),\n",
" FrozenTrial(number=29, state=TrialState.COMPLETE, value=78.21500910384827, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 23, 256092), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 23, 380383), params={'x': -6.843924982938755}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 29}, intermediate_values={}, trial_id=29),\n",
" FrozenTrial(number=30, state=TrialState.COMPLETE, value=0.259312297720867, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 23, 381838), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 23, 505185), params={'x': 1.4907728427107731}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 30}, intermediate_values={}, trial_id=30),\n",
" FrozenTrial(number=31, state=TrialState.COMPLETE, value=0.12200126346631915, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 23, 506554), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 23, 636475), params={'x': 2.3492867925735514}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 31}, intermediate_values={}, trial_id=31),\n",
" FrozenTrial(number=32, state=TrialState.COMPLETE, value=5.200258477183383, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 23, 638095), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 23, 767180), params={'x': -0.2804075243656303}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 32}, intermediate_values={}, trial_id=32),\n",
" FrozenTrial(number=33, state=TrialState.COMPLETE, value=0.013703612725368826, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 23, 768618), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 23, 894300), params={'x': 2.1170624308878336}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 33}, intermediate_values={}, trial_id=33),\n",
" FrozenTrial(number=34, state=TrialState.COMPLETE, value=4.887384243129526, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 23, 896869), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 24, 22341), params={'x': 4.210742916562106}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 34}, intermediate_values={}, trial_id=34),\n",
" FrozenTrial(number=35, state=TrialState.COMPLETE, value=3.5389872697319937, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 24, 23559), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 24, 152143), params={'x': 0.11878037706067035}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 35}, intermediate_values={}, trial_id=35),\n",
" FrozenTrial(number=36, state=TrialState.COMPLETE, value=12.27366888693429, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 24, 153565), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 24, 278755), params={'x': 5.503379637854609}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 36}, intermediate_values={}, trial_id=36),\n",
" FrozenTrial(number=37, state=TrialState.COMPLETE, value=0.0036732116343290485, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 24, 280364), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 24, 409580), params={'x': 1.9393929737214484}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 37}, intermediate_values={}, trial_id=37),\n",
" FrozenTrial(number=38, state=TrialState.COMPLETE, value=27.915939632915578, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 24, 411023), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 24, 538009), params={'x': -3.283553693577418}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 38}, intermediate_values={}, trial_id=38),\n",
" FrozenTrial(number=39, state=TrialState.COMPLETE, value=2.3415683229678104, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 24, 539273), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 24, 669659), params={'x': 3.530218390612206}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 39}, intermediate_values={}, trial_id=39),\n",
" FrozenTrial(number=40, state=TrialState.COMPLETE, value=52.03685409253796, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 24, 673766), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 24, 795412), params={'x': -5.21365746986492}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 40}, intermediate_values={}, trial_id=40),\n",
" FrozenTrial(number=41, state=TrialState.COMPLETE, value=0.005122558245931236, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 24, 796872), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 24, 921631), params={'x': 1.928427950665562}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 41}, intermediate_values={}, trial_id=41),\n",
" FrozenTrial(number=42, state=TrialState.COMPLETE, value=0.0015175934436004658, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 24, 923007), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 25, 45746), params={'x': 1.9610436982812733}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 42}, intermediate_values={}, trial_id=42),\n",
" FrozenTrial(number=43, state=TrialState.COMPLETE, value=0.02308619170169228, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 25, 47070), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 25, 172965), params={'x': 1.8480585912211807}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 43}, intermediate_values={}, trial_id=43),\n",
" FrozenTrial(number=44, state=TrialState.COMPLETE, value=2.071392063389823, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 25, 174040), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 25, 298910), params={'x': 0.560766848842821}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 44}, intermediate_values={}, trial_id=44),\n",
" FrozenTrial(number=45, state=TrialState.COMPLETE, value=9.80195215569949, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 25, 300076), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 25, 427919), params={'x': -1.1308069496057227}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 45}, intermediate_values={}, trial_id=45),\n",
" FrozenTrial(number=46, state=TrialState.COMPLETE, value=3.5803904571960037, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 25, 430365), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 25, 556536), params={'x': 3.8921919715493996}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 46}, intermediate_values={}, trial_id=46),\n",
" FrozenTrial(number=47, state=TrialState.COMPLETE, value=9.857502895307222, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 25, 557703), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 25, 681613), params={'x': 5.139666048373174}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 47}, intermediate_values={}, trial_id=47),\n",
" FrozenTrial(number=48, state=TrialState.COMPLETE, value=0.0030220228615943467, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 25, 682763), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 25, 812360), params={'x': 1.9450270715570441}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 48}, intermediate_values={}, trial_id=48),\n",
" FrozenTrial(number=49, state=TrialState.COMPLETE, value=17.87374821427753, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 25, 813534), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 25, 938788), params={'x': -2.2277355894470894}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 49}, intermediate_values={}, trial_id=49),\n",
" FrozenTrial(number=50, state=TrialState.COMPLETE, value=7.20536591581677, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 25, 939915), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 26, 68985), params={'x': -0.6842812661524071}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 50}, intermediate_values={}, trial_id=50),\n",
" FrozenTrial(number=51, state=TrialState.COMPLETE, value=0.04888118667602236, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 26, 70169), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 26, 198325), params={'x': 1.7789090986132121}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 51}, intermediate_values={}, trial_id=51),\n",
" FrozenTrial(number=52, state=TrialState.COMPLETE, value=1.6398329120457928, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 26, 199501), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 26, 325647), params={'x': 3.2805596089389173}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 52}, intermediate_values={}, trial_id=52),\n",
" FrozenTrial(number=53, state=TrialState.COMPLETE, value=1.8739831645513858, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 26, 326815), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 26, 453598), params={'x': 0.6310649523986225}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 53}, intermediate_values={}, trial_id=53),\n",
" FrozenTrial(number=54, state=TrialState.COMPLETE, value=0.5837015340809495, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 26, 454748), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 26, 581394), params={'x': 2.7640036217721415}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 54}, intermediate_values={}, trial_id=54),\n",
" FrozenTrial(number=55, state=TrialState.COMPLETE, value=0.002740901659309055, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 26, 582576), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 26, 707267), params={'x': 1.9476463787373877}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 55}, intermediate_values={}, trial_id=55),\n",
" FrozenTrial(number=56, state=TrialState.COMPLETE, value=1.3038362801828842, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 26, 711283), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 26, 830808), params={'x': 0.858143494048887}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 56}, intermediate_values={}, trial_id=56),\n",
" FrozenTrial(number=57, state=TrialState.COMPLETE, value=1.7068323055099774, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 26, 834458), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 26, 958836), params={'x': 3.306457923359944}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 57}, intermediate_values={}, trial_id=57),\n",
" FrozenTrial(number=58, state=TrialState.COMPLETE, value=4.502892563805704, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 26, 962468), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 27, 86568), params={'x': -0.12200201786089337}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 58}, intermediate_values={}, trial_id=58),\n",
" FrozenTrial(number=59, state=TrialState.COMPLETE, value=0.018832109155707495, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 27, 87879), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 27, 219977), params={'x': 2.137230132098266}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 59}, intermediate_values={}, trial_id=59),\n",
" FrozenTrial(number=60, state=TrialState.COMPLETE, value=0.6736067095875267, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 27, 221356), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 27, 350609), params={'x': 2.8207354687032398}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 60}, intermediate_values={}, trial_id=60),\n",
" FrozenTrial(number=61, state=TrialState.COMPLETE, value=0.02156872818392232, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 27, 351903), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 27, 480132), params={'x': 2.1468629571536755}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 61}, intermediate_values={}, trial_id=61),\n",
" FrozenTrial(number=62, state=TrialState.COMPLETE, value=0.5372027310193397, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 27, 481443), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 27, 609439), params={'x': 1.2670588488702932}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 62}, intermediate_values={}, trial_id=62),\n",
" FrozenTrial(number=63, state=TrialState.COMPLETE, value=9.54043835351742, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 27, 610649), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 27, 735823), params={'x': 5.088760002576668}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 63}, intermediate_values={}, trial_id=63),\n",
" FrozenTrial(number=64, state=TrialState.COMPLETE, value=18.82948057673564, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 27, 739469), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 27, 867334), params={'x': 6.339294940049091}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 64}, intermediate_values={}, trial_id=64),\n",
" FrozenTrial(number=65, state=TrialState.COMPLETE, value=2.800061910268745, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 27, 868640), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 27, 994502), params={'x': 0.3266614478030021}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 65}, intermediate_values={}, trial_id=65),\n",
" FrozenTrial(number=66, state=TrialState.COMPLETE, value=0.6122505880055049, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 27, 996442), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 28, 122626), params={'x': 1.2175355675779858}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 66}, intermediate_values={}, trial_id=66),\n",
" FrozenTrial(number=67, state=TrialState.COMPLETE, value=3.1017288130935317, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 28, 123755), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 28, 251149), params={'x': 3.761172567664376}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 67}, intermediate_values={}, trial_id=67),\n",
" FrozenTrial(number=68, state=TrialState.COMPLETE, value=0.1348552768751693, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 28, 252365), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 28, 380001), params={'x': 2.3672264653795656}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 68}, intermediate_values={}, trial_id=68),\n",
" FrozenTrial(number=69, state=TrialState.COMPLETE, value=0.1749702679547182, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 28, 381191), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 28, 509908), params={'x': 1.581705524833619}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 69}, intermediate_values={}, trial_id=69),\n",
" FrozenTrial(number=70, state=TrialState.COMPLETE, value=0.8985187195128104, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 28, 511033), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 28, 642935), params={'x': 2.9479022731868567}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 70}, intermediate_values={}, trial_id=70),\n",
" FrozenTrial(number=71, state=TrialState.COMPLETE, value=0.07148757033510479, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 28, 644041), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 28, 769696), params={'x': 2.267371595976657}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 71}, intermediate_values={}, trial_id=71),\n",
" FrozenTrial(number=72, state=TrialState.COMPLETE, value=0.00044389015974030644, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 28, 770870), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 28, 896534), params={'x': 2.0210687009504693}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 72}, intermediate_values={}, trial_id=72),\n",
" FrozenTrial(number=73, state=TrialState.COMPLETE, value=7.369472360192736, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 28, 897742), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 29, 22256), params={'x': -0.7146772110497293}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 73}, intermediate_values={}, trial_id=73),\n",
" FrozenTrial(number=74, state=TrialState.COMPLETE, value=7.375726767614576, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 29, 23391), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 29, 146000), params={'x': 4.715828928267496}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 74}, intermediate_values={}, trial_id=74),\n",
" FrozenTrial(number=75, state=TrialState.COMPLETE, value=0.8727071514675215, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 29, 147205), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 29, 282663), params={'x': 1.0658120363291328}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 75}, intermediate_values={}, trial_id=75),\n",
" FrozenTrial(number=76, state=TrialState.COMPLETE, value=0.010382044004186347, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 29, 283846), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 29, 414378), params={'x': 1.8981076842731193}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 76}, intermediate_values={}, trial_id=76),\n",
" FrozenTrial(number=77, state=TrialState.COMPLETE, value=2.635184403387668, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 29, 415547), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 29, 543309), params={'x': 0.37667489288569556}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 77}, intermediate_values={}, trial_id=77),\n",
" FrozenTrial(number=78, state=TrialState.COMPLETE, value=2.9782091678318325, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 29, 544557), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 29, 672456), params={'x': 3.7257488716009153}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 78}, intermediate_values={}, trial_id=78),\n",
" FrozenTrial(number=79, state=TrialState.COMPLETE, value=4.729113750534535, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 29, 673581), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 29, 800043), params={'x': -0.17465255857907902}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 79}, intermediate_values={}, trial_id=79),\n",
" FrozenTrial(number=80, state=TrialState.COMPLETE, value=0.24403482002036858, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 29, 801421), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 29, 931119), params={'x': 1.5060011943128115}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 80}, intermediate_values={}, trial_id=80),\n",
" FrozenTrial(number=81, state=TrialState.COMPLETE, value=0.5486926135394343, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 29, 932295), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 30, 62769), params={'x': 2.740737884503982}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 81}, intermediate_values={}, trial_id=81),\n",
" FrozenTrial(number=82, state=TrialState.COMPLETE, value=0.00038306803768869984, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 30, 64038), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 30, 194203), params={'x': 2.0195721239953333}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 82}, intermediate_values={}, trial_id=82),\n",
" FrozenTrial(number=83, state=TrialState.COMPLETE, value=0.0027532224013859462, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 30, 196107), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 30, 327581), params={'x': 1.9475288421951074}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 83}, intermediate_values={}, trial_id=83),\n",
" FrozenTrial(number=84, state=TrialState.COMPLETE, value=1.1783329603299877, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 30, 328812), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 30, 459550), params={'x': 0.9144895392811787}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 84}, intermediate_values={}, trial_id=84),\n",
" FrozenTrial(number=85, state=TrialState.COMPLETE, value=0.11919566653881611, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 30, 460699), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 30, 590443), params={'x': 1.6547527457910547}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 85}, intermediate_values={}, trial_id=85),\n",
" FrozenTrial(number=86, state=TrialState.COMPLETE, value=1.6160801124778839, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 30, 591954), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 30, 721754), params={'x': 3.271251396254055}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 86}, intermediate_values={}, trial_id=86),\n",
" FrozenTrial(number=87, state=TrialState.COMPLETE, value=4.379138035153702, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 30, 722862), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 30, 848921), params={'x': 4.0926390121455976}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 87}, intermediate_values={}, trial_id=87),\n",
" FrozenTrial(number=88, state=TrialState.COMPLETE, value=0.3957440711432687, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 30, 850036), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 30, 980564), params={'x': 2.6290819272108115}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 88}, intermediate_values={}, trial_id=88),\n",
" FrozenTrial(number=89, state=TrialState.COMPLETE, value=10.860223820880375, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 30, 981762), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 31, 117924), params={'x': -1.2954853695442763}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 89}, intermediate_values={}, trial_id=89),\n",
" FrozenTrial(number=90, state=TrialState.COMPLETE, value=1.8220964007163718, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 31, 119099), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 31, 254383), params={'x': 0.6501494894928654}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 90}, intermediate_values={}, trial_id=90),\n",
" FrozenTrial(number=91, state=TrialState.COMPLETE, value=0.004661277092706195, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 31, 255587), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 31, 390833), params={'x': 1.9317264539319496}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 91}, intermediate_values={}, trial_id=91),\n",
" FrozenTrial(number=92, state=TrialState.COMPLETE, value=0.013725422022546807, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 31, 391977), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 31, 518462), params={'x': 2.117155546273093}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 92}, intermediate_values={}, trial_id=92),\n",
" FrozenTrial(number=93, state=TrialState.COMPLETE, value=1.366403346908549, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 31, 519621), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 31, 648078), params={'x': 3.16893256730598}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 93}, intermediate_values={}, trial_id=93),\n",
" FrozenTrial(number=94, state=TrialState.COMPLETE, value=0.03291340176895431, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 31, 649244), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 31, 776982), params={'x': 1.8185794891172602}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 94}, intermediate_values={}, trial_id=94),\n",
" FrozenTrial(number=95, state=TrialState.COMPLETE, value=0.6909278677197279, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 31, 778075), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 31, 905671), params={'x': 1.1687792906094507}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 95}, intermediate_values={}, trial_id=95),\n",
" FrozenTrial(number=96, state=TrialState.COMPLETE, value=5.633426645824235, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 31, 906845), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 32, 35471), params={'x': -0.3734840732189957}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 96}, intermediate_values={}, trial_id=96),\n",
" FrozenTrial(number=97, state=TrialState.COMPLETE, value=2.8777104748700775, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 32, 36652), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 32, 169710), params={'x': 0.3036184170800258}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 97}, intermediate_values={}, trial_id=97),\n",
" FrozenTrial(number=98, state=TrialState.COMPLETE, value=0.2727919794867617, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 32, 170882), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 32, 301313), params={'x': 2.522294916198465}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 98}, intermediate_values={}, trial_id=98),\n",
" FrozenTrial(number=99, state=TrialState.COMPLETE, value=2.566803190789853, datetime_start=datetime.datetime(2019, 11, 20, 8, 25, 32, 302558), datetime_complete=datetime.datetime(2019, 11, 20, 8, 25, 32, 434199), params={'x': 3.602124586538092}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={'_number': 99}, intermediate_values={}, trial_id=99)]"
]
},
"metadata": {
"tags": []
},
"execution_count": 34
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "Xi98WPMSl4Mb",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 33
},
"outputId": "5963b3d8-b20c-4c39-e524-3985131970f2"
},
"source": [
"len(study.trials)"
],
"execution_count": 35,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"100"
]
},
"metadata": {
"tags": []
},
"execution_count": 35
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "JLGkTBvvmc0h",
"colab_type": "text"
},
"source": [
"### 目的関数に引数を持たせる"
]
},
{
"cell_type": "code",
"metadata": {
"id": "fKR_oHqwl6mU",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 1000
},
"outputId": "01594e25-e3c2-494e-8ffa-cd74b19f71bb"
},
"source": [
"def objective_variable_degree(n):\n",
"\n",
" def objective(trial):\n",
" x = trial.suggest_uniform('x', -10, 10)\n",
" return (x - 2) ** n\n",
"\n",
" return objective\n",
"\n",
"study = optuna.create_study()\n",
"study.optimize(objective_variable_degree(4), n_trials=100)\n",
"\n",
"print(study.best_params)"
],
"execution_count": 36,
"outputs": [
{
"output_type": "stream",
"text": [
"\u001b[32m[I 2019-11-20 08:29:06,195]\u001b[0m Finished trial#0 resulted in value: 1.382677100723694. Current best value is 1.382677100723694 with parameters: {'x': 0.9156232619116729}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:06,317]\u001b[0m Finished trial#1 resulted in value: 0.0027927353806104405. Current best value is 0.0027927353806104405 with parameters: {'x': 2.22988331265665}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:06,447]\u001b[0m Finished trial#2 resulted in value: 1767.5178430022604. Current best value is 0.0027927353806104405 with parameters: {'x': 2.22988331265665}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:06,574]\u001b[0m Finished trial#3 resulted in value: 0.45737109818229366. Current best value is 0.0027927353806104405 with parameters: {'x': 2.22988331265665}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:06,697]\u001b[0m Finished trial#4 resulted in value: 279.2295295424507. Current best value is 0.0027927353806104405 with parameters: {'x': 2.22988331265665}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:06,818]\u001b[0m Finished trial#5 resulted in value: 340.03219048410745. Current best value is 0.0027927353806104405 with parameters: {'x': 2.22988331265665}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:06,940]\u001b[0m Finished trial#6 resulted in value: 0.0015502100369688052. Current best value is 0.0015502100369688052 with parameters: {'x': 2.198425569803568}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:07,068]\u001b[0m Finished trial#7 resulted in value: 1.995255616267234. Current best value is 0.0015502100369688052 with parameters: {'x': 2.198425569803568}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:07,190]\u001b[0m Finished trial#8 resulted in value: 0.07920181091560147. Current best value is 0.0015502100369688052 with parameters: {'x': 2.198425569803568}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:07,319]\u001b[0m Finished trial#9 resulted in value: 24.707138080221128. Current best value is 0.0015502100369688052 with parameters: {'x': 2.198425569803568}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:07,444]\u001b[0m Finished trial#10 resulted in value: 2843.160076371392. Current best value is 0.0015502100369688052 with parameters: {'x': 2.198425569803568}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:07,570]\u001b[0m Finished trial#11 resulted in value: 16416.413889770443. Current best value is 0.0015502100369688052 with parameters: {'x': 2.198425569803568}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:07,694]\u001b[0m Finished trial#12 resulted in value: 983.5432159568961. Current best value is 0.0015502100369688052 with parameters: {'x': 2.198425569803568}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:07,821]\u001b[0m Finished trial#13 resulted in value: 350.40627537554263. Current best value is 0.0015502100369688052 with parameters: {'x': 2.198425569803568}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:07,948]\u001b[0m Finished trial#14 resulted in value: 86.46014708598416. Current best value is 0.0015502100369688052 with parameters: {'x': 2.198425569803568}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:08,077]\u001b[0m Finished trial#15 resulted in value: 4319.798784751398. Current best value is 0.0015502100369688052 with parameters: {'x': 2.198425569803568}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:08,206]\u001b[0m Finished trial#16 resulted in value: 56.23296080672064. Current best value is 0.0015502100369688052 with parameters: {'x': 2.198425569803568}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:08,331]\u001b[0m Finished trial#17 resulted in value: 1.0451627501731169. Current best value is 0.0015502100369688052 with parameters: {'x': 2.198425569803568}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:08,462]\u001b[0m Finished trial#18 resulted in value: 859.1559981480319. Current best value is 0.0015502100369688052 with parameters: {'x': 2.198425569803568}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:08,587]\u001b[0m Finished trial#19 resulted in value: 96.4466234885817. Current best value is 0.0015502100369688052 with parameters: {'x': 2.198425569803568}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:08,720]\u001b[0m Finished trial#20 resulted in value: 1749.2755311874043. Current best value is 0.0015502100369688052 with parameters: {'x': 2.198425569803568}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:08,842]\u001b[0m Finished trial#21 resulted in value: 0.004392868938340773. Current best value is 0.0015502100369688052 with parameters: {'x': 2.198425569803568}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:08,981]\u001b[0m Finished trial#22 resulted in value: 1.2244245643351994. Current best value is 0.0015502100369688052 with parameters: {'x': 2.198425569803568}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:09,112]\u001b[0m Finished trial#23 resulted in value: 1.1831019776072788e-06. Current best value is 1.1831019776072788e-06 with parameters: {'x': 2.0329803716311354}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:09,244]\u001b[0m Finished trial#24 resulted in value: 64.76305283490193. Current best value is 1.1831019776072788e-06 with parameters: {'x': 2.0329803716311354}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:09,370]\u001b[0m Finished trial#25 resulted in value: 23.86645215072568. Current best value is 1.1831019776072788e-06 with parameters: {'x': 2.0329803716311354}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:09,501]\u001b[0m Finished trial#26 resulted in value: 434.1592378058126. Current best value is 1.1831019776072788e-06 with parameters: {'x': 2.0329803716311354}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:09,627]\u001b[0m Finished trial#27 resulted in value: 0.08666864924532226. Current best value is 1.1831019776072788e-06 with parameters: {'x': 2.0329803716311354}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:09,760]\u001b[0m Finished trial#28 resulted in value: 306.0980233107942. Current best value is 1.1831019776072788e-06 with parameters: {'x': 2.0329803716311354}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:09,886]\u001b[0m Finished trial#29 resulted in value: 7.296801932988714. Current best value is 1.1831019776072788e-06 with parameters: {'x': 2.0329803716311354}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:10,013]\u001b[0m Finished trial#30 resulted in value: 2954.3767686484675. Current best value is 1.1831019776072788e-06 with parameters: {'x': 2.0329803716311354}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:10,140]\u001b[0m Finished trial#31 resulted in value: 0.007764500804611206. Current best value is 1.1831019776072788e-06 with parameters: {'x': 2.0329803716311354}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:10,271]\u001b[0m Finished trial#32 resulted in value: 9.709668298289492e-05. Current best value is 1.1831019776072788e-06 with parameters: {'x': 2.0329803716311354}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:10,403]\u001b[0m Finished trial#33 resulted in value: 15.562490676553871. Current best value is 1.1831019776072788e-06 with parameters: {'x': 2.0329803716311354}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:10,537]\u001b[0m Finished trial#34 resulted in value: 3.1430347165590526. Current best value is 1.1831019776072788e-06 with parameters: {'x': 2.0329803716311354}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:10,666]\u001b[0m Finished trial#35 resulted in value: 0.0013187582296048045. Current best value is 1.1831019776072788e-06 with parameters: {'x': 2.0329803716311354}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:10,800]\u001b[0m Finished trial#36 resulted in value: 151.1750958492243. Current best value is 1.1831019776072788e-06 with parameters: {'x': 2.0329803716311354}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:10,928]\u001b[0m Finished trial#37 resulted in value: 9.653950996952547. Current best value is 1.1831019776072788e-06 with parameters: {'x': 2.0329803716311354}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:11,056]\u001b[0m Finished trial#38 resulted in value: 1.676901103446206. Current best value is 1.1831019776072788e-06 with parameters: {'x': 2.0329803716311354}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:11,182]\u001b[0m Finished trial#39 resulted in value: 179.290922350787. Current best value is 1.1831019776072788e-06 with parameters: {'x': 2.0329803716311354}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:11,312]\u001b[0m Finished trial#40 resulted in value: 1373.921829414544. Current best value is 1.1831019776072788e-06 with parameters: {'x': 2.0329803716311354}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:11,443]\u001b[0m Finished trial#41 resulted in value: 2.8382499997917145e-06. Current best value is 1.1831019776072788e-06 with parameters: {'x': 2.0329803716311354}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:11,568]\u001b[0m Finished trial#42 resulted in value: 0.09959199479269125. Current best value is 1.1831019776072788e-06 with parameters: {'x': 2.0329803716311354}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:11,696]\u001b[0m Finished trial#43 resulted in value: 7.554134411823142e-10. Current best value is 7.554134411823142e-10 with parameters: {'x': 1.994757406593011}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:11,823]\u001b[0m Finished trial#44 resulted in value: 19.344046741568047. Current best value is 7.554134411823142e-10 with parameters: {'x': 1.994757406593011}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:11,956]\u001b[0m Finished trial#45 resulted in value: 0.2706037340388827. Current best value is 7.554134411823142e-10 with parameters: {'x': 1.994757406593011}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:12,085]\u001b[0m Finished trial#46 resulted in value: 11.861129489623831. Current best value is 7.554134411823142e-10 with parameters: {'x': 1.994757406593011}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:12,214]\u001b[0m Finished trial#47 resulted in value: 848.4479475756694. Current best value is 7.554134411823142e-10 with parameters: {'x': 1.994757406593011}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:12,340]\u001b[0m Finished trial#48 resulted in value: 0.014167392057216333. Current best value is 7.554134411823142e-10 with parameters: {'x': 1.994757406593011}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:12,471]\u001b[0m Finished trial#49 resulted in value: 44.578365248529586. Current best value is 7.554134411823142e-10 with parameters: {'x': 1.994757406593011}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:12,595]\u001b[0m Finished trial#50 resulted in value: 42.09112259228404. Current best value is 7.554134411823142e-10 with parameters: {'x': 1.994757406593011}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:12,719]\u001b[0m Finished trial#51 resulted in value: 0.0012085691470911865. Current best value is 7.554134411823142e-10 with parameters: {'x': 1.994757406593011}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:12,845]\u001b[0m Finished trial#52 resulted in value: 0.06748067858602633. Current best value is 7.554134411823142e-10 with parameters: {'x': 1.994757406593011}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:12,993]\u001b[0m Finished trial#53 resulted in value: 2.0812847959579264. Current best value is 7.554134411823142e-10 with parameters: {'x': 1.994757406593011}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:13,127]\u001b[0m Finished trial#54 resulted in value: 1.4405714503901568. Current best value is 7.554134411823142e-10 with parameters: {'x': 1.994757406593011}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:13,255]\u001b[0m Finished trial#55 resulted in value: 213.3833012464252. Current best value is 7.554134411823142e-10 with parameters: {'x': 1.994757406593011}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:13,390]\u001b[0m Finished trial#56 resulted in value: 0.0007329974790669412. Current best value is 7.554134411823142e-10 with parameters: {'x': 1.994757406593011}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:13,517]\u001b[0m Finished trial#57 resulted in value: 5.364633615929887. Current best value is 7.554134411823142e-10 with parameters: {'x': 1.994757406593011}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:13,647]\u001b[0m Finished trial#58 resulted in value: 88.08330146330765. Current best value is 7.554134411823142e-10 with parameters: {'x': 1.994757406593011}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:13,774]\u001b[0m Finished trial#59 resulted in value: 2.2196121207911904e-06. Current best value is 7.554134411823142e-10 with parameters: {'x': 1.994757406593011}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:13,905]\u001b[0m Finished trial#60 resulted in value: 15.116782680438885. Current best value is 7.554134411823142e-10 with parameters: {'x': 1.994757406593011}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:14,035]\u001b[0m Finished trial#61 resulted in value: 3.330271327501411e-05. Current best value is 7.554134411823142e-10 with parameters: {'x': 1.994757406593011}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:14,163]\u001b[0m Finished trial#62 resulted in value: 0.29436194514469. Current best value is 7.554134411823142e-10 with parameters: {'x': 1.994757406593011}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:14,296]\u001b[0m Finished trial#63 resulted in value: 1.8033171821349794. Current best value is 7.554134411823142e-10 with parameters: {'x': 1.994757406593011}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:14,424]\u001b[0m Finished trial#64 resulted in value: 0.00011647331502580325. Current best value is 7.554134411823142e-10 with parameters: {'x': 1.994757406593011}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:14,551]\u001b[0m Finished trial#65 resulted in value: 35.16151783357178. Current best value is 7.554134411823142e-10 with parameters: {'x': 1.994757406593011}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:14,682]\u001b[0m Finished trial#66 resulted in value: 9.078208607765687. Current best value is 7.554134411823142e-10 with parameters: {'x': 1.994757406593011}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:14,811]\u001b[0m Finished trial#67 resulted in value: 96.30558807708644. Current best value is 7.554134411823142e-10 with parameters: {'x': 1.994757406593011}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:14,943]\u001b[0m Finished trial#68 resulted in value: 0.5074120373778825. Current best value is 7.554134411823142e-10 with parameters: {'x': 1.994757406593011}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:15,069]\u001b[0m Finished trial#69 resulted in value: 0.08638557259740251. Current best value is 7.554134411823142e-10 with parameters: {'x': 1.994757406593011}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:15,197]\u001b[0m Finished trial#70 resulted in value: 2.11370465484328. Current best value is 7.554134411823142e-10 with parameters: {'x': 1.994757406593011}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:15,327]\u001b[0m Finished trial#71 resulted in value: 1.3110855955186357e-06. Current best value is 7.554134411823142e-10 with parameters: {'x': 1.994757406593011}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:15,458]\u001b[0m Finished trial#72 resulted in value: 0.00018125185911226446. Current best value is 7.554134411823142e-10 with parameters: {'x': 1.994757406593011}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:15,590]\u001b[0m Finished trial#73 resulted in value: 5.864446179777471. Current best value is 7.554134411823142e-10 with parameters: {'x': 1.994757406593011}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:15,722]\u001b[0m Finished trial#74 resulted in value: 0.2203373606148161. Current best value is 7.554134411823142e-10 with parameters: {'x': 1.994757406593011}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:15,849]\u001b[0m Finished trial#75 resulted in value: 8.389607899698985. Current best value is 7.554134411823142e-10 with parameters: {'x': 1.994757406593011}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:15,972]\u001b[0m Finished trial#76 resulted in value: 0.3752822055733322. Current best value is 7.554134411823142e-10 with parameters: {'x': 1.994757406593011}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:16,101]\u001b[0m Finished trial#77 resulted in value: 26.415472265628605. Current best value is 7.554134411823142e-10 with parameters: {'x': 1.994757406593011}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:16,232]\u001b[0m Finished trial#78 resulted in value: 4.97441622449693e-08. Current best value is 7.554134411823142e-10 with parameters: {'x': 1.994757406593011}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:16,364]\u001b[0m Finished trial#79 resulted in value: 31.107916253902854. Current best value is 7.554134411823142e-10 with parameters: {'x': 1.994757406593011}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:16,497]\u001b[0m Finished trial#80 resulted in value: 1.4019561808997287. Current best value is 7.554134411823142e-10 with parameters: {'x': 1.994757406593011}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:16,624]\u001b[0m Finished trial#81 resulted in value: 0.011545573945175741. Current best value is 7.554134411823142e-10 with parameters: {'x': 1.994757406593011}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:16,750]\u001b[0m Finished trial#82 resulted in value: 0.005310412431597511. Current best value is 7.554134411823142e-10 with parameters: {'x': 1.994757406593011}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:16,882]\u001b[0m Finished trial#83 resulted in value: 3.650587118880552e-05. Current best value is 7.554134411823142e-10 with parameters: {'x': 1.994757406593011}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:17,014]\u001b[0m Finished trial#84 resulted in value: 0.42181231251911255. Current best value is 7.554134411823142e-10 with parameters: {'x': 1.994757406593011}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:17,145]\u001b[0m Finished trial#85 resulted in value: 2.724431205364563. Current best value is 7.554134411823142e-10 with parameters: {'x': 1.994757406593011}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:17,278]\u001b[0m Finished trial#86 resulted in value: 3.313763941231036. Current best value is 7.554134411823142e-10 with parameters: {'x': 1.994757406593011}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:17,409]\u001b[0m Finished trial#87 resulted in value: 1.957965575016234e-07. Current best value is 7.554134411823142e-10 with parameters: {'x': 1.994757406593011}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:17,540]\u001b[0m Finished trial#88 resulted in value: 84.62191186553488. Current best value is 7.554134411823142e-10 with parameters: {'x': 1.994757406593011}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:17,668]\u001b[0m Finished trial#89 resulted in value: 0.5201409130700528. Current best value is 7.554134411823142e-10 with parameters: {'x': 1.994757406593011}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:17,790]\u001b[0m Finished trial#90 resulted in value: 0.14875284513882667. Current best value is 7.554134411823142e-10 with parameters: {'x': 1.994757406593011}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:17,917]\u001b[0m Finished trial#91 resulted in value: 0.0527220762670679. Current best value is 7.554134411823142e-10 with parameters: {'x': 1.994757406593011}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:18,042]\u001b[0m Finished trial#92 resulted in value: 4.825097880769117e-05. Current best value is 7.554134411823142e-10 with parameters: {'x': 1.994757406593011}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:18,172]\u001b[0m Finished trial#93 resulted in value: 0.006415232399833745. Current best value is 7.554134411823142e-10 with parameters: {'x': 1.994757406593011}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:18,305]\u001b[0m Finished trial#94 resulted in value: 8.41479466111938. Current best value is 7.554134411823142e-10 with parameters: {'x': 1.994757406593011}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:18,434]\u001b[0m Finished trial#95 resulted in value: 5.336050645482353. Current best value is 7.554134411823142e-10 with parameters: {'x': 1.994757406593011}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:18,561]\u001b[0m Finished trial#96 resulted in value: 18.52223325179656. Current best value is 7.554134411823142e-10 with parameters: {'x': 1.994757406593011}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:18,689]\u001b[0m Finished trial#97 resulted in value: 1.5649987477151347e-11. Current best value is 1.5649987477151347e-11 with parameters: {'x': 1.9980110287819561}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:18,819]\u001b[0m Finished trial#98 resulted in value: 2.139358349196014. Current best value is 1.5649987477151347e-11 with parameters: {'x': 1.9980110287819561}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:29:18,948]\u001b[0m Finished trial#99 resulted in value: 0.5558213279470339. Current best value is 1.5649987477151347e-11 with parameters: {'x': 1.9980110287819561}.\u001b[0m\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"{'x': 1.9980110287819561}\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "0BBYBA6w7fRA",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 542
},
"outputId": "35344548-f0e6-4e91-bd04-ce22eda9e56d"
},
"source": [
"plot_optimization_history(study)"
],
"execution_count": 37,
"outputs": [
{
"output_type": "display_data",
"data": {
"text/html": [
"<html>\n",
"<head><meta charset=\"utf-8\" /></head>\n",
"<body>\n",
" <div>\n",
" <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax) {MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>\n",
" <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
" <script src=\"https://cdn.plot.ly/plotly-latest.min.js\"></script> \n",
" <div id=\"bc35e23b-1357-458a-b23e-95ddcbfd7660\" class=\"plotly-graph-div\" style=\"height:525px; width:100%;\"></div>\n",
" <script type=\"text/javascript\">\n",
" \n",
" window.PLOTLYENV=window.PLOTLYENV || {};\n",
" \n",
" if (document.getElementById(\"bc35e23b-1357-458a-b23e-95ddcbfd7660\")) {\n",
" Plotly.newPlot(\n",
" 'bc35e23b-1357-458a-b23e-95ddcbfd7660',\n",
" [{\"mode\": \"markers\", \"name\": \"Objective Value\", \"type\": \"scatter\", \"x\": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99], \"y\": [1.382677100723694, 0.0027927353806104405, 1767.5178430022604, 0.45737109818229366, 279.2295295424507, 340.03219048410745, 0.0015502100369688052, 1.995255616267234, 0.07920181091560147, 24.707138080221128, 2843.160076371392, 16416.413889770443, 983.5432159568961, 350.40627537554263, 86.46014708598416, 4319.798784751398, 56.23296080672064, 1.0451627501731169, 859.1559981480319, 96.4466234885817, 1749.2755311874043, 0.004392868938340773, 1.2244245643351994, 1.1831019776072788e-06, 64.76305283490193, 23.86645215072568, 434.1592378058126, 0.08666864924532226, 306.0980233107942, 7.296801932988714, 2954.3767686484675, 0.007764500804611206, 9.709668298289492e-05, 15.562490676553871, 3.1430347165590526, 0.0013187582296048045, 151.1750958492243, 9.653950996952547, 1.676901103446206, 179.290922350787, 1373.921829414544, 2.8382499997917145e-06, 0.09959199479269125, 7.554134411823142e-10, 19.344046741568047, 0.2706037340388827, 11.861129489623831, 848.4479475756694, 0.014167392057216333, 44.578365248529586, 42.09112259228404, 0.0012085691470911865, 0.06748067858602633, 2.0812847959579264, 1.4405714503901568, 213.3833012464252, 0.0007329974790669412, 5.364633615929887, 88.08330146330765, 2.2196121207911904e-06, 15.116782680438885, 3.330271327501411e-05, 0.29436194514469, 1.8033171821349794, 0.00011647331502580325, 35.16151783357178, 9.078208607765687, 96.30558807708644, 0.5074120373778825, 0.08638557259740251, 2.11370465484328, 1.3110855955186357e-06, 0.00018125185911226446, 5.864446179777471, 0.2203373606148161, 8.389607899698985, 0.3752822055733322, 26.415472265628605, 4.97441622449693e-08, 31.107916253902854, 1.4019561808997287, 0.011545573945175741, 0.005310412431597511, 3.650587118880552e-05, 0.42181231251911255, 2.724431205364563, 3.313763941231036, 1.957965575016234e-07, 84.62191186553488, 0.5201409130700528, 0.14875284513882667, 0.0527220762670679, 4.825097880769117e-05, 0.006415232399833745, 8.41479466111938, 5.336050645482353, 18.52223325179656, 1.5649987477151347e-11, 2.139358349196014, 0.5558213279470339]}, {\"name\": \"Best Value\", \"type\": \"scatter\", \"x\": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99], \"y\": [1.382677100723694, 0.0027927353806104405, 0.0027927353806104405, 0.0027927353806104405, 0.0027927353806104405, 0.0027927353806104405, 0.0015502100369688052, 0.0015502100369688052, 0.0015502100369688052, 0.0015502100369688052, 0.0015502100369688052, 0.0015502100369688052, 0.0015502100369688052, 0.0015502100369688052, 0.0015502100369688052, 0.0015502100369688052, 0.0015502100369688052, 0.0015502100369688052, 0.0015502100369688052, 0.0015502100369688052, 0.0015502100369688052, 0.0015502100369688052, 0.0015502100369688052, 1.1831019776072788e-06, 1.1831019776072788e-06, 1.1831019776072788e-06, 1.1831019776072788e-06, 1.1831019776072788e-06, 1.1831019776072788e-06, 1.1831019776072788e-06, 1.1831019776072788e-06, 1.1831019776072788e-06, 1.1831019776072788e-06, 1.1831019776072788e-06, 1.1831019776072788e-06, 1.1831019776072788e-06, 1.1831019776072788e-06, 1.1831019776072788e-06, 1.1831019776072788e-06, 1.1831019776072788e-06, 1.1831019776072788e-06, 1.1831019776072788e-06, 1.1831019776072788e-06, 7.554134411823142e-10, 7.554134411823142e-10, 7.554134411823142e-10, 7.554134411823142e-10, 7.554134411823142e-10, 7.554134411823142e-10, 7.554134411823142e-10, 7.554134411823142e-10, 7.554134411823142e-10, 7.554134411823142e-10, 7.554134411823142e-10, 7.554134411823142e-10, 7.554134411823142e-10, 7.554134411823142e-10, 7.554134411823142e-10, 7.554134411823142e-10, 7.554134411823142e-10, 7.554134411823142e-10, 7.554134411823142e-10, 7.554134411823142e-10, 7.554134411823142e-10, 7.554134411823142e-10, 7.554134411823142e-10, 7.554134411823142e-10, 7.554134411823142e-10, 7.554134411823142e-10, 7.554134411823142e-10, 7.554134411823142e-10, 7.554134411823142e-10, 7.554134411823142e-10, 7.554134411823142e-10, 7.554134411823142e-10, 7.554134411823142e-10, 7.554134411823142e-10, 7.554134411823142e-10, 7.554134411823142e-10, 7.554134411823142e-10, 7.554134411823142e-10, 7.554134411823142e-10, 7.554134411823142e-10, 7.554134411823142e-10, 7.554134411823142e-10, 7.554134411823142e-10, 7.554134411823142e-10, 7.554134411823142e-10, 7.554134411823142e-10, 7.554134411823142e-10, 7.554134411823142e-10, 7.554134411823142e-10, 7.554134411823142e-10, 7.554134411823142e-10, 7.554134411823142e-10, 7.554134411823142e-10, 7.554134411823142e-10, 1.5649987477151347e-11, 1.5649987477151347e-11, 1.5649987477151347e-11]}],\n",
" {\"template\": {\"data\": {\"bar\": [{\"error_x\": {\"color\": \"#2a3f5f\"}, \"error_y\": {\"color\": \"#2a3f5f\"}, \"marker\": {\"line\": {\"color\": \"#E5ECF6\", \"width\": 0.5}}, \"type\": \"bar\"}], \"barpolar\": [{\"marker\": {\"line\": {\"color\": \"#E5ECF6\", \"width\": 0.5}}, \"type\": \"barpolar\"}], \"carpet\": [{\"aaxis\": {\"endlinecolor\": \"#2a3f5f\", \"gridcolor\": \"white\", \"linecolor\": \"white\", \"minorgridcolor\": \"white\", \"startlinecolor\": \"#2a3f5f\"}, \"baxis\": {\"endlinecolor\": \"#2a3f5f\", \"gridcolor\": \"white\", \"linecolor\": \"white\", \"minorgridcolor\": \"white\", \"startlinecolor\": \"#2a3f5f\"}, \"type\": \"carpet\"}], \"choropleth\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"choropleth\"}], \"contour\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"contour\"}], \"contourcarpet\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"contourcarpet\"}], \"heatmap\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"heatmap\"}], \"heatmapgl\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"heatmapgl\"}], \"histogram\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"histogram\"}], \"histogram2d\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"histogram2d\"}], \"histogram2dcontour\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"histogram2dcontour\"}], \"mesh3d\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"mesh3d\"}], \"parcoords\": [{\"line\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"parcoords\"}], \"scatter\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatter\"}], \"scatter3d\": [{\"line\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatter3d\"}], \"scattercarpet\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattercarpet\"}], \"scattergeo\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattergeo\"}], \"scattergl\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattergl\"}], \"scattermapbox\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattermapbox\"}], \"scatterpolar\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterpolar\"}], \"scatterpolargl\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterpolargl\"}], \"scatterternary\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterternary\"}], \"surface\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"surface\"}], \"table\": [{\"cells\": {\"fill\": {\"color\": \"#EBF0F8\"}, \"line\": {\"color\": \"white\"}}, \"header\": {\"fill\": {\"color\": \"#C8D4E3\"}, \"line\": {\"color\": \"white\"}}, \"type\": \"table\"}]}, \"layout\": {\"annotationdefaults\": {\"arrowcolor\": \"#2a3f5f\", \"arrowhead\": 0, \"arrowwidth\": 1}, \"colorscale\": {\"diverging\": [[0, \"#8e0152\"], [0.1, \"#c51b7d\"], [0.2, \"#de77ae\"], [0.3, \"#f1b6da\"], [0.4, \"#fde0ef\"], [0.5, \"#f7f7f7\"], [0.6, \"#e6f5d0\"], [0.7, \"#b8e186\"], [0.8, \"#7fbc41\"], [0.9, \"#4d9221\"], [1, \"#276419\"]], \"sequential\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"sequentialminus\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]]}, \"colorway\": [\"#636efa\", \"#EF553B\", \"#00cc96\", \"#ab63fa\", \"#FFA15A\", \"#19d3f3\", \"#FF6692\", \"#B6E880\", \"#FF97FF\", \"#FECB52\"], \"font\": {\"color\": \"#2a3f5f\"}, \"geo\": {\"bgcolor\": \"white\", \"lakecolor\": \"white\", \"landcolor\": \"#E5ECF6\", \"showlakes\": true, \"showland\": true, \"subunitcolor\": \"white\"}, \"hoverlabel\": {\"align\": \"left\"}, \"hovermode\": \"closest\", \"mapbox\": {\"style\": \"light\"}, \"paper_bgcolor\": \"white\", \"plot_bgcolor\": \"#E5ECF6\", \"polar\": {\"angularaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"bgcolor\": \"#E5ECF6\", \"radialaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}}, \"scene\": {\"xaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}, \"yaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}, \"zaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}}, \"shapedefaults\": {\"line\": {\"color\": \"#2a3f5f\"}}, \"ternary\": {\"aaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"baxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"bgcolor\": \"#E5ECF6\", \"caxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}}, \"title\": {\"x\": 0.05}, \"xaxis\": {\"automargin\": true, \"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\", \"zerolinecolor\": \"white\", \"zerolinewidth\": 2}, \"yaxis\": {\"automargin\": true, \"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\", \"zerolinecolor\": \"white\", \"zerolinewidth\": 2}}}, \"title\": {\"text\": \"Optimization History Plot\"}, \"xaxis\": {\"title\": {\"text\": \"#Trials\"}}, \"yaxis\": {\"title\": {\"text\": \"Objective Value\"}}},\n",
" {\"responsive\": true}\n",
" ).then(function(){\n",
" \n",
"var gd = document.getElementById('bc35e23b-1357-458a-b23e-95ddcbfd7660');\n",
"var x = new MutationObserver(function (mutations, observer) {{\n",
" var display = window.getComputedStyle(gd).display;\n",
" if (!display || display === 'none') {{\n",
" console.log([gd, 'removed!']);\n",
" Plotly.purge(gd);\n",
" observer.disconnect();\n",
" }}\n",
"}});\n",
"\n",
"// Listen for the removal of the full notebook cells\n",
"var notebookContainer = gd.closest('#notebook-container');\n",
"if (notebookContainer) {{\n",
" x.observe(notebookContainer, {childList: true});\n",
"}}\n",
"\n",
"// Listen for the clearing of the current output cell\n",
"var outputEl = gd.closest('.output');\n",
"if (outputEl) {{\n",
" x.observe(outputEl, {childList: true});\n",
"}}\n",
"\n",
" })\n",
" };\n",
" \n",
" </script>\n",
" </div>\n",
"</body>\n",
"</html>"
]
},
"metadata": {
"tags": []
}
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "XE3ARfxUnnKs",
"colab_type": "text"
},
"source": [
"### SVMに使ってみる\n",
"\n",
"- https://optuna.readthedocs.io/en/latest/reference/trial.html\n",
"- https://qiita.com/KROYO/items/66d613356e2bf4cf9fed\n",
"- https://qiita.com/KROYO/items/608c15f9c550417b217f"
]
},
{
"cell_type": "code",
"metadata": {
"id": "oOYxUl3lmXDy",
"colab_type": "code",
"colab": {}
},
"source": [
"from functools import partial\n",
"from sklearn.model_selection import StratifiedKFold\n",
"from sklearn.model_selection import cross_validate\n",
"from sklearn.svm import SVC\n",
"from sklearn import datasets"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "fHyPy6ixmXGC",
"colab_type": "code",
"colab": {}
},
"source": [
"def objective(X, y, trial):\n",
" params = {\n",
" 'kernel': trial.suggest_categorical('kernel', ['rbf', 'sigmoid']),\n",
" 'C': trial.suggest_loguniform('C', 1e+0, 1e+2),\n",
" 'gamma': trial.suggest_loguniform('gamma', 1e-2, 1e+1),\n",
" }\n",
"\n",
" # モデルを作る\n",
" model = SVC(**params)\n",
"\n",
" # 5-Fold CV / Accuracy でモデルを評価する\n",
" # StratifiledKFold 関数は なるべくクラスラベルが均等に割り振られるように交差検証を行う\n",
" kf = StratifiedKFold(n_splits=5, shuffle=True, random_state=1)\n",
"\n",
" scores = cross_validate(model, X=X, y=y, cv=kf)\n",
" # 最小化なので 1.0 からスコアを引く\n",
" return 1.0 - scores['test_score'].mean()"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "ecDSxQMUmXIU",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 1000
},
"outputId": "ea5fed4a-5d49-4d88-d477-7f2b0dfbb29a"
},
"source": [
"# データセットを読み込み\n",
"dataset = datasets.load_breast_cancer()\n",
"X, y = dataset.data, dataset.target\n",
"\n",
"# 目的関数の一部引数を固定し、新たに関数を作る\n",
"f = partial(objective, X, y)\n",
"\n",
"study = optuna.create_study()\n",
"study.optimize(f, n_trials=100)\n",
"print('params:', study.best_params)"
],
"execution_count": 42,
"outputs": [
{
"output_type": "stream",
"text": [
"\u001b[32m[I 2019-11-20 08:32:12,039]\u001b[0m Finished trial#0 resulted in value: 0.37257406694882644. Current best value is 0.37257406694882644 with parameters: {'kernel': 'rbf', 'C': 79.64934443359876, 'gamma': 5.399723073927139}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:12,263]\u001b[0m Finished trial#1 resulted in value: 0.37257406694882644. Current best value is 0.37257406694882644 with parameters: {'kernel': 'rbf', 'C': 79.64934443359876, 'gamma': 5.399723073927139}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:12,492]\u001b[0m Finished trial#2 resulted in value: 0.37257406694882644. Current best value is 0.37257406694882644 with parameters: {'kernel': 'rbf', 'C': 79.64934443359876, 'gamma': 5.399723073927139}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:12,670]\u001b[0m Finished trial#3 resulted in value: 0.37257406694882644. Current best value is 0.37257406694882644 with parameters: {'kernel': 'rbf', 'C': 79.64934443359876, 'gamma': 5.399723073927139}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:12,851]\u001b[0m Finished trial#4 resulted in value: 0.37257406694882644. Current best value is 0.37257406694882644 with parameters: {'kernel': 'rbf', 'C': 79.64934443359876, 'gamma': 5.399723073927139}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:13,031]\u001b[0m Finished trial#5 resulted in value: 0.37257406694882644. Current best value is 0.37257406694882644 with parameters: {'kernel': 'rbf', 'C': 79.64934443359876, 'gamma': 5.399723073927139}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:13,254]\u001b[0m Finished trial#6 resulted in value: 0.37257406694882644. Current best value is 0.37257406694882644 with parameters: {'kernel': 'rbf', 'C': 79.64934443359876, 'gamma': 5.399723073927139}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:13,433]\u001b[0m Finished trial#7 resulted in value: 0.37257406694882644. Current best value is 0.37257406694882644 with parameters: {'kernel': 'rbf', 'C': 79.64934443359876, 'gamma': 5.399723073927139}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:13,656]\u001b[0m Finished trial#8 resulted in value: 0.37257406694882644. Current best value is 0.37257406694882644 with parameters: {'kernel': 'rbf', 'C': 79.64934443359876, 'gamma': 5.399723073927139}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:13,896]\u001b[0m Finished trial#9 resulted in value: 0.37257406694882644. Current best value is 0.37257406694882644 with parameters: {'kernel': 'rbf', 'C': 79.64934443359876, 'gamma': 5.399723073927139}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:14,142]\u001b[0m Finished trial#10 resulted in value: 0.3708041554444017. Current best value is 0.3708041554444017 with parameters: {'kernel': 'rbf', 'C': 6.258770343467455, 'gamma': 0.012743982872691633}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:14,385]\u001b[0m Finished trial#11 resulted in value: 0.37257406694882644. Current best value is 0.3708041554444017 with parameters: {'kernel': 'rbf', 'C': 6.258770343467455, 'gamma': 0.012743982872691633}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:14,625]\u001b[0m Finished trial#12 resulted in value: 0.36903424393997697. Current best value is 0.36903424393997697 with parameters: {'kernel': 'rbf', 'C': 8.326698687314426, 'gamma': 0.01219802874323995}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:14,864]\u001b[0m Finished trial#13 resulted in value: 0.3672951135051943. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:15,107]\u001b[0m Finished trial#14 resulted in value: 0.3672951135051943. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:15,352]\u001b[0m Finished trial#15 resulted in value: 0.37257406694882644. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:15,587]\u001b[0m Finished trial#16 resulted in value: 0.37257406694882644. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:15,826]\u001b[0m Finished trial#17 resulted in value: 0.37257406694882644. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:16,079]\u001b[0m Finished trial#18 resulted in value: 0.36903424393997697. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:16,312]\u001b[0m Finished trial#19 resulted in value: 0.37257406694882644. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:16,546]\u001b[0m Finished trial#20 resulted in value: 0.37257406694882644. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:16,792]\u001b[0m Finished trial#21 resulted in value: 0.36903424393997697. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:17,034]\u001b[0m Finished trial#22 resulted in value: 0.37257406694882644. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:17,273]\u001b[0m Finished trial#23 resulted in value: 0.37257406694882644. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:17,517]\u001b[0m Finished trial#24 resulted in value: 0.37257406694882644. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:17,768]\u001b[0m Finished trial#25 resulted in value: 0.36903424393997697. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:17,951]\u001b[0m Finished trial#26 resulted in value: 0.37257406694882644. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:18,199]\u001b[0m Finished trial#27 resulted in value: 0.36903424393997697. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:18,446]\u001b[0m Finished trial#28 resulted in value: 0.37257406694882644. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:18,683]\u001b[0m Finished trial#29 resulted in value: 0.37257406694882644. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:18,925]\u001b[0m Finished trial#30 resulted in value: 0.37257406694882644. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:19,177]\u001b[0m Finished trial#31 resulted in value: 0.3672951135051943. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:19,425]\u001b[0m Finished trial#32 resulted in value: 0.37431319738360913. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:19,676]\u001b[0m Finished trial#33 resulted in value: 0.3672951135051943. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:19,921]\u001b[0m Finished trial#34 resulted in value: 0.37257406694882644. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:20,161]\u001b[0m Finished trial#35 resulted in value: 0.37257406694882644. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:20,349]\u001b[0m Finished trial#36 resulted in value: 0.37257406694882644. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:20,592]\u001b[0m Finished trial#37 resulted in value: 0.37257406694882644. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:20,836]\u001b[0m Finished trial#38 resulted in value: 0.37257406694882644. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:21,026]\u001b[0m Finished trial#39 resulted in value: 0.37257406694882644. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:21,268]\u001b[0m Finished trial#40 resulted in value: 0.37257406694882644. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:21,521]\u001b[0m Finished trial#41 resulted in value: 0.3672951135051943. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:21,770]\u001b[0m Finished trial#42 resulted in value: 0.37431319738360913. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:22,017]\u001b[0m Finished trial#43 resulted in value: 0.37257406694882644. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:22,265]\u001b[0m Finished trial#44 resulted in value: 0.37257406694882644. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:22,516]\u001b[0m Finished trial#45 resulted in value: 0.37257406694882644. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:22,773]\u001b[0m Finished trial#46 resulted in value: 0.37431319738360913. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:23,017]\u001b[0m Finished trial#47 resulted in value: 0.37257406694882644. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:23,271]\u001b[0m Finished trial#48 resulted in value: 0.36903424393997697. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:23,460]\u001b[0m Finished trial#49 resulted in value: 0.37257406694882644. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:23,714]\u001b[0m Finished trial#50 resulted in value: 0.37431319738360913. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:23,965]\u001b[0m Finished trial#51 resulted in value: 0.3672951135051943. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:24,213]\u001b[0m Finished trial#52 resulted in value: 0.37257406694882644. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:24,467]\u001b[0m Finished trial#53 resulted in value: 0.3708041554444017. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:24,714]\u001b[0m Finished trial#54 resulted in value: 0.37257406694882644. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:24,970]\u001b[0m Finished trial#55 resulted in value: 0.36903424393997697. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:25,230]\u001b[0m Finished trial#56 resulted in value: 0.37257406694882644. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:25,481]\u001b[0m Finished trial#57 resulted in value: 0.37257406694882644. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:25,719]\u001b[0m Finished trial#58 resulted in value: 0.37257406694882644. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:25,974]\u001b[0m Finished trial#59 resulted in value: 0.3672951135051943. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:26,223]\u001b[0m Finished trial#60 resulted in value: 0.37257406694882644. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:26,480]\u001b[0m Finished trial#61 resulted in value: 0.36903424393997697. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:26,732]\u001b[0m Finished trial#62 resulted in value: 0.36903424393997697. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:26,982]\u001b[0m Finished trial#63 resulted in value: 0.37257406694882644. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:27,239]\u001b[0m Finished trial#64 resulted in value: 0.37431319738360913. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:27,486]\u001b[0m Finished trial#65 resulted in value: 0.3672951135051943. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:27,743]\u001b[0m Finished trial#66 resulted in value: 0.37257406694882644. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:27,998]\u001b[0m Finished trial#67 resulted in value: 0.36903424393997697. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:28,263]\u001b[0m Finished trial#68 resulted in value: 0.3672951135051943. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:28,516]\u001b[0m Finished trial#69 resulted in value: 0.37257406694882644. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:28,769]\u001b[0m Finished trial#70 resulted in value: 0.36903424393997697. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:29,021]\u001b[0m Finished trial#71 resulted in value: 0.37431319738360913. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:29,274]\u001b[0m Finished trial#72 resulted in value: 0.3672951135051943. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:29,527]\u001b[0m Finished trial#73 resulted in value: 0.37257406694882644. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:29,777]\u001b[0m Finished trial#74 resulted in value: 0.37257406694882644. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:29,968]\u001b[0m Finished trial#75 resulted in value: 0.37257406694882644. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:30,220]\u001b[0m Finished trial#76 resulted in value: 0.37257406694882644. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:30,480]\u001b[0m Finished trial#77 resulted in value: 0.37431319738360913. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:30,727]\u001b[0m Finished trial#78 resulted in value: 0.37257406694882644. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:30,977]\u001b[0m Finished trial#79 resulted in value: 0.3672951135051943. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:31,222]\u001b[0m Finished trial#80 resulted in value: 0.37257406694882644. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:31,483]\u001b[0m Finished trial#81 resulted in value: 0.3672951135051943. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:31,740]\u001b[0m Finished trial#82 resulted in value: 0.3708041554444017. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:31,996]\u001b[0m Finished trial#83 resulted in value: 0.3672951135051943. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:32,271]\u001b[0m Finished trial#84 resulted in value: 0.37257406694882644. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:32,529]\u001b[0m Finished trial#85 resulted in value: 0.37431319738360913. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:32,781]\u001b[0m Finished trial#86 resulted in value: 0.37257406694882644. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:33,036]\u001b[0m Finished trial#87 resulted in value: 0.37257406694882644. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:33,293]\u001b[0m Finished trial#88 resulted in value: 0.37257406694882644. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:33,558]\u001b[0m Finished trial#89 resulted in value: 0.3672951135051943. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:33,756]\u001b[0m Finished trial#90 resulted in value: 0.37257406694882644. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:34,011]\u001b[0m Finished trial#91 resulted in value: 0.36903424393997697. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:34,271]\u001b[0m Finished trial#92 resulted in value: 0.3672951135051943. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:34,528]\u001b[0m Finished trial#93 resulted in value: 0.3672951135051943. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:34,787]\u001b[0m Finished trial#94 resulted in value: 0.3672951135051943. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:35,049]\u001b[0m Finished trial#95 resulted in value: 0.37257406694882644. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:35,304]\u001b[0m Finished trial#96 resulted in value: 0.37257406694882644. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:35,566]\u001b[0m Finished trial#97 resulted in value: 0.37431319738360913. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:35,825]\u001b[0m Finished trial#98 resulted in value: 0.37257406694882644. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:32:36,081]\u001b[0m Finished trial#99 resulted in value: 0.37257406694882644. Current best value is 0.3672951135051943 with parameters: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}.\u001b[0m\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"params: {'kernel': 'rbf', 'C': 7.311607370345594, 'gamma': 0.010367272011210715}\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "cgy3uROh8Mdw",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 542
},
"outputId": "4c37535a-bccd-4995-c782-49ac6b6dcfed"
},
"source": [
"plot_optimization_history(study)"
],
"execution_count": 43,
"outputs": [
{
"output_type": "display_data",
"data": {
"text/html": [
"<html>\n",
"<head><meta charset=\"utf-8\" /></head>\n",
"<body>\n",
" <div>\n",
" <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax) {MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>\n",
" <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
" <script src=\"https://cdn.plot.ly/plotly-latest.min.js\"></script> \n",
" <div id=\"e06d4175-2a22-4215-97d4-160518d22d07\" class=\"plotly-graph-div\" style=\"height:525px; width:100%;\"></div>\n",
" <script type=\"text/javascript\">\n",
" \n",
" window.PLOTLYENV=window.PLOTLYENV || {};\n",
" \n",
" if (document.getElementById(\"e06d4175-2a22-4215-97d4-160518d22d07\")) {\n",
" Plotly.newPlot(\n",
" 'e06d4175-2a22-4215-97d4-160518d22d07',\n",
" [{\"mode\": \"markers\", \"name\": \"Objective Value\", \"type\": \"scatter\", \"x\": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99], \"y\": [0.37257406694882644, 0.37257406694882644, 0.37257406694882644, 0.37257406694882644, 0.37257406694882644, 0.37257406694882644, 0.37257406694882644, 0.37257406694882644, 0.37257406694882644, 0.37257406694882644, 0.3708041554444017, 0.37257406694882644, 0.36903424393997697, 0.3672951135051943, 0.3672951135051943, 0.37257406694882644, 0.37257406694882644, 0.37257406694882644, 0.36903424393997697, 0.37257406694882644, 0.37257406694882644, 0.36903424393997697, 0.37257406694882644, 0.37257406694882644, 0.37257406694882644, 0.36903424393997697, 0.37257406694882644, 0.36903424393997697, 0.37257406694882644, 0.37257406694882644, 0.37257406694882644, 0.3672951135051943, 0.37431319738360913, 0.3672951135051943, 0.37257406694882644, 0.37257406694882644, 0.37257406694882644, 0.37257406694882644, 0.37257406694882644, 0.37257406694882644, 0.37257406694882644, 0.3672951135051943, 0.37431319738360913, 0.37257406694882644, 0.37257406694882644, 0.37257406694882644, 0.37431319738360913, 0.37257406694882644, 0.36903424393997697, 0.37257406694882644, 0.37431319738360913, 0.3672951135051943, 0.37257406694882644, 0.3708041554444017, 0.37257406694882644, 0.36903424393997697, 0.37257406694882644, 0.37257406694882644, 0.37257406694882644, 0.3672951135051943, 0.37257406694882644, 0.36903424393997697, 0.36903424393997697, 0.37257406694882644, 0.37431319738360913, 0.3672951135051943, 0.37257406694882644, 0.36903424393997697, 0.3672951135051943, 0.37257406694882644, 0.36903424393997697, 0.37431319738360913, 0.3672951135051943, 0.37257406694882644, 0.37257406694882644, 0.37257406694882644, 0.37257406694882644, 0.37431319738360913, 0.37257406694882644, 0.3672951135051943, 0.37257406694882644, 0.3672951135051943, 0.3708041554444017, 0.3672951135051943, 0.37257406694882644, 0.37431319738360913, 0.37257406694882644, 0.37257406694882644, 0.37257406694882644, 0.3672951135051943, 0.37257406694882644, 0.36903424393997697, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.37257406694882644, 0.37257406694882644, 0.37431319738360913, 0.37257406694882644, 0.37257406694882644]}, {\"name\": \"Best Value\", \"type\": \"scatter\", \"x\": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99], \"y\": [0.37257406694882644, 0.37257406694882644, 0.37257406694882644, 0.37257406694882644, 0.37257406694882644, 0.37257406694882644, 0.37257406694882644, 0.37257406694882644, 0.37257406694882644, 0.37257406694882644, 0.3708041554444017, 0.3708041554444017, 0.36903424393997697, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943, 0.3672951135051943]}],\n",
" {\"template\": {\"data\": {\"bar\": [{\"error_x\": {\"color\": \"#2a3f5f\"}, \"error_y\": {\"color\": \"#2a3f5f\"}, \"marker\": {\"line\": {\"color\": \"#E5ECF6\", \"width\": 0.5}}, \"type\": \"bar\"}], \"barpolar\": [{\"marker\": {\"line\": {\"color\": \"#E5ECF6\", \"width\": 0.5}}, \"type\": \"barpolar\"}], \"carpet\": [{\"aaxis\": {\"endlinecolor\": \"#2a3f5f\", \"gridcolor\": \"white\", \"linecolor\": \"white\", \"minorgridcolor\": \"white\", \"startlinecolor\": \"#2a3f5f\"}, \"baxis\": {\"endlinecolor\": \"#2a3f5f\", \"gridcolor\": \"white\", \"linecolor\": \"white\", \"minorgridcolor\": \"white\", \"startlinecolor\": \"#2a3f5f\"}, \"type\": \"carpet\"}], \"choropleth\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"choropleth\"}], \"contour\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"contour\"}], \"contourcarpet\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"contourcarpet\"}], \"heatmap\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"heatmap\"}], \"heatmapgl\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"heatmapgl\"}], \"histogram\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"histogram\"}], \"histogram2d\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"histogram2d\"}], \"histogram2dcontour\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"histogram2dcontour\"}], \"mesh3d\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"mesh3d\"}], \"parcoords\": [{\"line\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"parcoords\"}], \"scatter\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatter\"}], \"scatter3d\": [{\"line\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatter3d\"}], \"scattercarpet\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattercarpet\"}], \"scattergeo\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattergeo\"}], \"scattergl\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattergl\"}], \"scattermapbox\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattermapbox\"}], \"scatterpolar\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterpolar\"}], \"scatterpolargl\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterpolargl\"}], \"scatterternary\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterternary\"}], \"surface\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"surface\"}], \"table\": [{\"cells\": {\"fill\": {\"color\": \"#EBF0F8\"}, \"line\": {\"color\": \"white\"}}, \"header\": {\"fill\": {\"color\": \"#C8D4E3\"}, \"line\": {\"color\": \"white\"}}, \"type\": \"table\"}]}, \"layout\": {\"annotationdefaults\": {\"arrowcolor\": \"#2a3f5f\", \"arrowhead\": 0, \"arrowwidth\": 1}, \"colorscale\": {\"diverging\": [[0, \"#8e0152\"], [0.1, \"#c51b7d\"], [0.2, \"#de77ae\"], [0.3, \"#f1b6da\"], [0.4, \"#fde0ef\"], [0.5, \"#f7f7f7\"], [0.6, \"#e6f5d0\"], [0.7, \"#b8e186\"], [0.8, \"#7fbc41\"], [0.9, \"#4d9221\"], [1, \"#276419\"]], \"sequential\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"sequentialminus\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]]}, \"colorway\": [\"#636efa\", \"#EF553B\", \"#00cc96\", \"#ab63fa\", \"#FFA15A\", \"#19d3f3\", \"#FF6692\", \"#B6E880\", \"#FF97FF\", \"#FECB52\"], \"font\": {\"color\": \"#2a3f5f\"}, \"geo\": {\"bgcolor\": \"white\", \"lakecolor\": \"white\", \"landcolor\": \"#E5ECF6\", \"showlakes\": true, \"showland\": true, \"subunitcolor\": \"white\"}, \"hoverlabel\": {\"align\": \"left\"}, \"hovermode\": \"closest\", \"mapbox\": {\"style\": \"light\"}, \"paper_bgcolor\": \"white\", \"plot_bgcolor\": \"#E5ECF6\", \"polar\": {\"angularaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"bgcolor\": \"#E5ECF6\", \"radialaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}}, \"scene\": {\"xaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}, \"yaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}, \"zaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}}, \"shapedefaults\": {\"line\": {\"color\": \"#2a3f5f\"}}, \"ternary\": {\"aaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"baxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"bgcolor\": \"#E5ECF6\", \"caxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}}, \"title\": {\"x\": 0.05}, \"xaxis\": {\"automargin\": true, \"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\", \"zerolinecolor\": \"white\", \"zerolinewidth\": 2}, \"yaxis\": {\"automargin\": true, \"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\", \"zerolinecolor\": \"white\", \"zerolinewidth\": 2}}}, \"title\": {\"text\": \"Optimization History Plot\"}, \"xaxis\": {\"title\": {\"text\": \"#Trials\"}}, \"yaxis\": {\"title\": {\"text\": \"Objective Value\"}}},\n",
" {\"responsive\": true}\n",
" ).then(function(){\n",
" \n",
"var gd = document.getElementById('e06d4175-2a22-4215-97d4-160518d22d07');\n",
"var x = new MutationObserver(function (mutations, observer) {{\n",
" var display = window.getComputedStyle(gd).display;\n",
" if (!display || display === 'none') {{\n",
" console.log([gd, 'removed!']);\n",
" Plotly.purge(gd);\n",
" observer.disconnect();\n",
" }}\n",
"}});\n",
"\n",
"// Listen for the removal of the full notebook cells\n",
"var notebookContainer = gd.closest('#notebook-container');\n",
"if (notebookContainer) {{\n",
" x.observe(notebookContainer, {childList: true});\n",
"}}\n",
"\n",
"// Listen for the clearing of the current output cell\n",
"var outputEl = gd.closest('.output');\n",
"if (outputEl) {{\n",
" x.observe(outputEl, {childList: true});\n",
"}}\n",
"\n",
" })\n",
" };\n",
" \n",
" </script>\n",
" </div>\n",
"</body>\n",
"</html>"
]
},
"metadata": {
"tags": []
}
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "0EtZyNDF8Rjb",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 542
},
"outputId": "284a2117-c777-452c-eb5e-916ee58de34a"
},
"source": [
"optuna.visualization.plot_contour(study, params=['C', 'gamma'])"
],
"execution_count": 44,
"outputs": [
{
"output_type": "display_data",
"data": {
"text/html": [
"<html>\n",
"<head><meta charset=\"utf-8\" /></head>\n",
"<body>\n",
" <div>\n",
" <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax) {MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>\n",
" <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
" <script src=\"https://cdn.plot.ly/plotly-latest.min.js\"></script> \n",
" <div id=\"833605eb-d62f-4fe7-b51a-0544396d17d3\" class=\"plotly-graph-div\" style=\"height:525px; width:100%;\"></div>\n",
" <script type=\"text/javascript\">\n",
" \n",
" window.PLOTLYENV=window.PLOTLYENV || {};\n",
" \n",
" if (document.getElementById(\"833605eb-d62f-4fe7-b51a-0544396d17d3\")) {\n",
" Plotly.newPlot(\n",
" '833605eb-d62f-4fe7-b51a-0544396d17d3',\n",
" [{\"colorbar\": {\"title\": {\"text\": \"Objective Value\"}}, \"colorscale\": [[0, \"rgb(220,220,220)\"], [0.30000000000000004, \"rgb(106,137,247)\"], [0.4, \"rgb(90,120,245)\"], [0.5, \"rgb(70,100,245)\"], [0.65, \"rgb(40,60,190)\"], [1, \"rgb(5,10,172)\"]], \"connectgaps\": true, \"contours\": {\"coloring\": \"heatmap\"}, \"hoverinfo\": \"none\", \"line\": {\"smoothing\": 1.3}, \"type\": \"contour\", \"x\": [1.0364259407057848, 1.7928291726192989, 2.1488793737648164, 2.726495736424455, 3.0667671896086555, 3.4804206931684702, 4.028795166441822, 4.333383271306546, 4.577380312153573, 5.270654063944633, 5.312475523538329, 5.475888006361376, 6.071509784969308, 6.258770343467455, 6.463252206016852, 6.59548603186849, 6.728783791132939, 7.164593428004696, 7.311607370345594, 7.457421101586715, 7.708257021834121, 7.802012408087914, 8.070842165552412, 8.144408896690026, 8.326698687314426, 8.689161022956705, 8.793222240489834, 9.357381551048777, 9.404892516059112, 9.45511322933084, 10.053849844278645, 10.081003808321345, 10.51883140371503, 10.672217997245705, 11.222001444726256, 11.575079298130873, 11.656067778540814, 11.733842338451383, 11.818560999854759, 11.881754984080617, 12.196297744559846, 12.219825630287904, 12.288339051724, 12.575875408821439, 12.912392743320206, 13.072999006792637, 13.121442696915041, 13.399227043667558, 13.55745322380066, 14.016443042463802, 14.412165901651688, 14.546934411287483, 14.617473466753554, 15.088505170004366, 15.399273596390648, 15.73297821818745, 15.890274932563436, 16.15002147050895, 16.635673716204487, 16.642743223943306, 16.887011792697106, 16.887937262848006, 17.083260417290607, 17.169011966715104, 17.927390835121223, 18.588480518175167, 18.749532606865312, 19.02112069050599, 19.805630741902817, 20.417398171784523, 20.423192414957683, 20.51980058584701, 20.58420115057008, 20.883885272154547, 21.059134872317962, 22.105007915574266, 22.380105896762746, 22.871092677585505, 23.669333700853663, 24.354353873743218, 24.77624890414656, 25.35247844359653, 26.114244232141015, 26.803108058987124, 27.168383562627497, 27.529515977238628, 28.529513503597805, 29.694186483877264, 31.37879241837265, 31.907866703399012, 35.31782020908816, 36.2128923808199, 39.085964637519886, 41.658003437692976, 44.15090837906444, 53.86863856756103, 64.55692047431424, 76.5148738216162, 79.64934443359876, 83.7087252076461], \"y\": [0.010028915990704388, 0.010036600697757206, 0.010054553946181898, 0.010071937656246286, 0.010090752812459905, 0.010109461334014715, 0.010116888600272258, 0.01012457978584005, 0.010135736256662968, 0.010136411647065087, 0.010367272011210715, 0.010367969675334852, 0.010424361328189536, 0.010450357778498806, 0.010727918525900592, 0.010849610091960463, 0.011096774495932063, 0.011141098481296744, 0.011225671456785833, 0.011372176516249811, 0.011629592078746885, 0.011705610470156957, 0.01172464659286905, 0.011830239019935133, 0.01192824508655751, 0.011947285984829052, 0.012026411092600987, 0.012134105568218544, 0.01219802874323995, 0.012229518171648322, 0.01225920905254933, 0.012502810078914335, 0.01258526514082497, 0.012743982872691633, 0.013197595166349677, 0.01319974069473757, 0.013690403734180435, 0.013758841973262248, 0.013922914688616596, 0.014486357064388477, 0.014634074799153874, 0.014786899373513816, 0.015155062527194858, 0.015339327593735191, 0.015753626646019127, 0.0158492807513776, 0.015852225708661124, 0.016698683928660366, 0.01670616862456558, 0.01731302856583356, 0.01731361108099454, 0.017404301619068006, 0.017736763163670583, 0.018479428937941283, 0.019134405052058, 0.01984122143703075, 0.019842643928322166, 0.020142739154817926, 0.021006661962631328, 0.02117838426773326, 0.021871059326281788, 0.023311105052183836, 0.02338790384407034, 0.023883010047480866, 0.02440514883830316, 0.025325684747907872, 0.026597684717906165, 0.02666931908588682, 0.02852307988282518, 0.029208306442871926, 0.031661489137024816, 0.03245450593517221, 0.033372076436506054, 0.037518721418360865, 0.037831794186643994, 0.041267757575813885, 0.04627827790962873, 0.04859118300683853, 0.05480673577931646, 0.06304466226275689, 0.07447643691334832, 0.08313057815396326, 0.11032788491965177, 0.12195754596739561, 0.12338245148829584, 0.2157393321962981, 0.2756679502246661, 0.35035040107927856, 0.6351768636984743, 0.6400834911801184, 0.7683505283618415, 1.1733232140790113, 1.36699184113158, 2.3227946915124122, 3.580824783584128, 5.399723073927139, 6.342312581779458, 8.272716604678905, 8.787214199967, 9.0932413396537], \"z\": [[null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.36903424393997697, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.36903424393997697, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.36903424393997697, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.36903424393997697, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.36903424393997697, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.37257406694882644, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.36903424393997697, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.36903424393997697, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.36903424393997697, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.36903424393997697, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.3672951135051943, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.37257406694882644, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.3672951135051943, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.3672951135051943, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.3672951135051943, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.3672951135051943, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.3672951135051943, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.3672951135051943, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.3672951135051943, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.3672951135051943, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.3672951135051943, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.3672951135051943, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.3672951135051943, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.3672951135051943, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.3672951135051943, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.3672951135051943, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.3672951135051943, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.3672951135051943, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.36903424393997697, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.36903424393997697, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.36903424393997697, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.3708041554444017, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.3708041554444017, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, 0.3708041554444017, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, 0.37431319738360913, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.37431319738360913, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.37431319738360913, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.37431319738360913, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.37257406694882644, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.37431319738360913, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.37431319738360913, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.37431319738360913, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.37431319738360913, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, 0.37431319738360913, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.37257406694882644, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.37257406694882644, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.37257406694882644, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, 0.37257406694882644, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.37257406694882644, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.37257406694882644, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.37257406694882644, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.37257406694882644, null, null, null, null, null, null, null, null, null], [null, null, 0.37257406694882644, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.37257406694882644, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.37257406694882644, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, 0.37257406694882644, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.37257406694882644, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.37257406694882644, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.37257406694882644, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.37257406694882644, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.37257406694882644, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.37257406694882644, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.37257406694882644, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [0.37257406694882644, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.37257406694882644, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.37257406694882644, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.37257406694882644, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.37257406694882644, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.37257406694882644, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.37257406694882644, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.37257406694882644, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.37257406694882644, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.37257406694882644, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, 0.37257406694882644, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.37257406694882644, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.37257406694882644, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.37257406694882644, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.37257406694882644], [null, null, null, null, null, null, null, null, null, null, null, 0.37257406694882644, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.37257406694882644, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.37257406694882644, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.37257406694882644, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.37257406694882644, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.37257406694882644, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.37257406694882644, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, 0.37257406694882644, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.37257406694882644, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.37257406694882644, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.37257406694882644, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, 0.37257406694882644, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.37257406694882644, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.37257406694882644, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, 0.37257406694882644, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, 0.37257406694882644, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.37257406694882644, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.37257406694882644, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.37257406694882644, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, 0.37257406694882644, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.37257406694882644, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.37257406694882644, null, null, null, null]]}, {\"marker\": {\"color\": \"black\"}, \"mode\": \"markers\", \"showlegend\": false, \"type\": \"scatter\", \"x\": [79.64934443359876, 76.5148738216162, 83.7087252076461, 14.412165901651688, 36.2128923808199, 22.871092677585505, 2.726495736424455, 1.7928291726192989, 3.4804206931684702, 53.86863856756103, 6.258770343467455, 6.59548603186849, 8.326698687314426, 7.311607370345594, 12.288339051724, 16.15002147050895, 4.333383271306546, 1.0364259407057848, 11.656067778540814, 22.105007915574266, 28.529513503597805, 11.818560999854759, 10.053849844278645, 5.270654063944633, 16.887011792697106, 7.708257021834121, 8.144408896690026, 11.733842338451383, 2.1488793737648164, 4.028795166441822, 39.085964637519886, 12.219825630287904, 8.689161022956705, 13.55745322380066, 20.423192414957683, 5.475888006361376, 14.617473466753554, 25.35247844359653, 31.907866703399012, 16.887937262848006, 10.51883140371503, 13.072999006792637, 13.121442696915041, 19.02112069050599, 6.463252206016852, 9.357381551048777, 41.658003437692976, 14.016443042463802, 64.55692047431424, 7.164593428004696, 3.0667671896086555, 7.802012408087914, 5.312475523538329, 10.081003808321345, 12.912392743320206, 18.749532606865312, 4.577380312153573, 15.399273596390648, 7.457421101586715, 23.669333700853663, 26.114244232141015, 11.222001444726256, 8.793222240489834, 22.380105896762746, 6.071509784969308, 12.196297744559846, 12.575875408821439, 9.45511322933084, 17.083260417290607, 17.927390835121223, 31.37879241837265, 14.546934411287483, 20.51980058584701, 10.672217997245705, 24.77624890414656, 8.070842165552412, 11.881754984080617, 6.728783791132939, 15.088505170004366, 16.635673716204487, 9.404892516059112, 20.883885272154547, 21.059134872317962, 19.805630741902817, 44.15090837906444, 29.694186483877264, 24.354353873743218, 35.31782020908816, 16.642743223943306, 18.588480518175167, 13.399227043667558, 15.890274932563436, 20.417398171784523, 27.529515977238628, 26.803108058987124, 27.168383562627497, 17.169011966715104, 11.575079298130873, 20.58420115057008, 15.73297821818745], \"y\": [5.399723073927139, 0.7683505283618415, 0.04859118300683853, 0.11032788491965177, 0.07447643691334832, 8.787214199967, 2.3227946915124122, 1.36699184113158, 8.272716604678905, 9.0932413396537, 0.012743982872691633, 0.015852225708661124, 0.01219802874323995, 0.010367272011210715, 0.010424361328189536, 0.03245450593517221, 0.2157393321962981, 0.023883010047480866, 0.010090752812459905, 0.2756679502246661, 0.12195754596739561, 0.010054553946181898, 0.02440514883830316, 0.037518721418360865, 0.020142739154817926, 0.010135736256662968, 0.06304466226275689, 0.010136411647065087, 0.017736763163670583, 0.6400834911801184, 0.033372076436506054, 0.010727918525900592, 0.015155062527194858, 0.011096774495932063, 0.025325684747907872, 0.05480673577931646, 0.12338245148829584, 0.041267757575813885, 0.01670616862456558, 0.010367969675334852, 0.08313057815396326, 0.010849610091960463, 0.013758841973262248, 3.580824783584128, 0.026597684717906165, 0.019134405052058, 0.01319974069473757, 0.6351768636984743, 0.010071937656246286, 0.04627827790962873, 0.013197595166349677, 0.010450357778498806, 0.01984122143703075, 0.01258526514082497, 0.031661489137024816, 0.01012457978584005, 0.016698683928660366, 0.023311105052183836, 1.1733232140790113, 0.011705610470156957, 0.029208306442871926, 0.010036600697757206, 0.01225920905254933, 0.021006661962631328, 0.015339327593735191, 0.012026411092600987, 0.015753626646019127, 0.010116888600272258, 0.01192824508655751, 0.019842643928322166, 0.012229518171648322, 0.014486357064388477, 0.011629592078746885, 0.01731302856583356, 0.02338790384407034, 0.010109461334014715, 0.037831794186643994, 0.014786899373513816, 0.02666931908588682, 0.011225671456785833, 0.35035040107927856, 0.011141098481296744, 0.012502810078914335, 0.011947285984829052, 0.018479428937941283, 0.014634074799153874, 6.342312581779458, 0.017404301619068006, 0.02117838426773326, 0.011372176516249811, 0.013922914688616596, 0.010028915990704388, 0.011830239019935133, 0.01172464659286905, 0.012134105568218544, 0.0158492807513776, 0.021871059326281788, 0.013690403734180435, 0.01731361108099454, 0.02852307988282518]}],\n",
" {\"template\": {\"data\": {\"bar\": [{\"error_x\": {\"color\": \"#2a3f5f\"}, \"error_y\": {\"color\": \"#2a3f5f\"}, \"marker\": {\"line\": {\"color\": \"#E5ECF6\", \"width\": 0.5}}, \"type\": \"bar\"}], \"barpolar\": [{\"marker\": {\"line\": {\"color\": \"#E5ECF6\", \"width\": 0.5}}, \"type\": \"barpolar\"}], \"carpet\": [{\"aaxis\": {\"endlinecolor\": \"#2a3f5f\", \"gridcolor\": \"white\", \"linecolor\": \"white\", \"minorgridcolor\": \"white\", \"startlinecolor\": \"#2a3f5f\"}, \"baxis\": {\"endlinecolor\": \"#2a3f5f\", \"gridcolor\": \"white\", \"linecolor\": \"white\", \"minorgridcolor\": \"white\", \"startlinecolor\": \"#2a3f5f\"}, \"type\": \"carpet\"}], \"choropleth\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"choropleth\"}], \"contour\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"contour\"}], \"contourcarpet\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"contourcarpet\"}], \"heatmap\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"heatmap\"}], \"heatmapgl\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"heatmapgl\"}], \"histogram\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"histogram\"}], \"histogram2d\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"histogram2d\"}], \"histogram2dcontour\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"histogram2dcontour\"}], \"mesh3d\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"mesh3d\"}], \"parcoords\": [{\"line\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"parcoords\"}], \"scatter\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatter\"}], \"scatter3d\": [{\"line\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatter3d\"}], \"scattercarpet\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattercarpet\"}], \"scattergeo\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattergeo\"}], \"scattergl\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattergl\"}], \"scattermapbox\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattermapbox\"}], \"scatterpolar\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterpolar\"}], \"scatterpolargl\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterpolargl\"}], \"scatterternary\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterternary\"}], \"surface\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"surface\"}], \"table\": [{\"cells\": {\"fill\": {\"color\": \"#EBF0F8\"}, \"line\": {\"color\": \"white\"}}, \"header\": {\"fill\": {\"color\": \"#C8D4E3\"}, \"line\": {\"color\": \"white\"}}, \"type\": \"table\"}]}, \"layout\": {\"annotationdefaults\": {\"arrowcolor\": \"#2a3f5f\", \"arrowhead\": 0, \"arrowwidth\": 1}, \"colorscale\": {\"diverging\": [[0, \"#8e0152\"], [0.1, \"#c51b7d\"], [0.2, \"#de77ae\"], [0.3, \"#f1b6da\"], [0.4, \"#fde0ef\"], [0.5, \"#f7f7f7\"], [0.6, \"#e6f5d0\"], [0.7, \"#b8e186\"], [0.8, \"#7fbc41\"], [0.9, \"#4d9221\"], [1, \"#276419\"]], \"sequential\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"sequentialminus\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]]}, \"colorway\": [\"#636efa\", \"#EF553B\", \"#00cc96\", \"#ab63fa\", \"#FFA15A\", \"#19d3f3\", \"#FF6692\", \"#B6E880\", \"#FF97FF\", \"#FECB52\"], \"font\": {\"color\": \"#2a3f5f\"}, \"geo\": {\"bgcolor\": \"white\", \"lakecolor\": \"white\", \"landcolor\": \"#E5ECF6\", \"showlakes\": true, \"showland\": true, \"subunitcolor\": \"white\"}, \"hoverlabel\": {\"align\": \"left\"}, \"hovermode\": \"closest\", \"mapbox\": {\"style\": \"light\"}, \"paper_bgcolor\": \"white\", \"plot_bgcolor\": \"#E5ECF6\", \"polar\": {\"angularaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"bgcolor\": \"#E5ECF6\", \"radialaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}}, \"scene\": {\"xaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}, \"yaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}, \"zaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}}, \"shapedefaults\": {\"line\": {\"color\": \"#2a3f5f\"}}, \"ternary\": {\"aaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"baxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"bgcolor\": \"#E5ECF6\", \"caxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}}, \"title\": {\"x\": 0.05}, \"xaxis\": {\"automargin\": true, \"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\", \"zerolinecolor\": \"white\", \"zerolinewidth\": 2}, \"yaxis\": {\"automargin\": true, \"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\", \"zerolinecolor\": \"white\", \"zerolinewidth\": 2}}}, \"xaxis\": {\"range\": [0.015538274406792452, 1.9227707281490765], \"title\": {\"text\": \"C\"}, \"type\": \"log\"}, \"yaxis\": {\"range\": [-1.998746006630368, 0.9587187176559598], \"title\": {\"text\": \"gamma\"}, \"type\": \"log\"}},\n",
" {\"responsive\": true}\n",
" ).then(function(){\n",
" \n",
"var gd = document.getElementById('833605eb-d62f-4fe7-b51a-0544396d17d3');\n",
"var x = new MutationObserver(function (mutations, observer) {{\n",
" var display = window.getComputedStyle(gd).display;\n",
" if (!display || display === 'none') {{\n",
" console.log([gd, 'removed!']);\n",
" Plotly.purge(gd);\n",
" observer.disconnect();\n",
" }}\n",
"}});\n",
"\n",
"// Listen for the removal of the full notebook cells\n",
"var notebookContainer = gd.closest('#notebook-container');\n",
"if (notebookContainer) {{\n",
" x.observe(notebookContainer, {childList: true});\n",
"}}\n",
"\n",
"// Listen for the clearing of the current output cell\n",
"var outputEl = gd.closest('.output');\n",
"if (outputEl) {{\n",
" x.observe(outputEl, {childList: true});\n",
"}}\n",
"\n",
" })\n",
" };\n",
" \n",
" </script>\n",
" </div>\n",
"</body>\n",
"</html>"
]
},
"metadata": {
"tags": []
}
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "TIXE3V9j2v8s",
"colab_type": "text"
},
"source": [
"### 複数のモデル"
]
},
{
"cell_type": "code",
"metadata": {
"id": "HRPURuhZ2vT8",
"colab_type": "code",
"colab": {}
},
"source": [
"from sklearn.ensemble import RandomForestClassifier"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "pDwT4IQn27mu",
"colab_type": "code",
"colab": {}
},
"source": [
"def objective(X, y, trial):\n",
" classifier = trial.suggest_categorical('classifier', ['SVC', 'RandomForestClassifier'])\n",
"\n",
" if classifier == 'SVC':\n",
" params = {\n",
" 'kernel': trial.suggest_categorical('kernel', ['rbf', 'sigmoid']),\n",
" 'C': trial.suggest_loguniform('C', 1e+0, 1e+2),\n",
" 'gamma': trial.suggest_loguniform('gamma', 1e-2, 1e+1),\n",
" }\n",
" model = SVC(**params)\n",
" else:\n",
" params = {\n",
" 'n_estimators': int(trial.suggest_loguniform('n_estimators', 1e+2, 1e+3)),\n",
" 'max_depth': int(trial.suggest_loguniform('max_depth', 2, 32)),\n",
" }\n",
" model = RandomForestClassifier(**params)\n",
"\n",
" kf = StratifiedKFold(n_splits=5, shuffle=True, random_state=1)\n",
" scores = cross_validate(model, X=X, y=y, cv=kf, n_jobs=-1)\n",
" return 1.0 - scores['test_score'].mean()"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "HxkIuCsv9c8_",
"colab_type": "text"
},
"source": [
"- https://qiita.com/higumachan725/items/9c25a061830eccf82c59\n",
"- https://poyo.hatenablog.jp/entry/2019/03/25/003519#%E6%9E%9D%E5%88%88%E3%82%8A%E3%81%A8%E3%81%AF\n",
"\n",
"> 深層学習や勾配ブースティングなど、反復アルゴリズムが学習に用いられる場合、学習曲線から、最終的な結果がどのぐらいうまくいきそうかを大まかに予測することができます。この予測を用いて、良い結果を残すことが見込まれない試行は、最後まで行うことなく早期に終了させてしまうことができます。これが、Optuna のもつ枝刈りの機能になります。"
]
},
{
"cell_type": "code",
"metadata": {
"id": "4SVzz9q827pm",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 1000
},
"outputId": "939bd797-37ca-40fb-b837-de2c8af78565"
},
"source": [
"`dataset = datasets.load_breast_cancer()\n",
"X, y = dataset.data, dataset.target\n",
"f = partial(objective, X, y)\n",
"# 枝刈りをされないようにする\n",
"optuna.create_study(pruner=optuna.pruners.MedianPruner(n_warmup_steps=10)) \n",
"study.optimize(f, n_trials=100)\n",
"print('params:', study.best_params)"
],
"execution_count": 49,
"outputs": [
{
"output_type": "stream",
"text": [
"\u001b[32m[I 2019-11-20 08:37:14,149]\u001b[0m Finished trial#58 resulted in value: 0.033351288957291225. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:37:18,770]\u001b[0m Finished trial#59 resulted in value: 0.03161215852250865. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:37:22,915]\u001b[0m Finished trial#60 resulted in value: 0.033351288957291336. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:37:25,774]\u001b[0m Finished trial#61 resulted in value: 0.03505963832243175. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:37:30,284]\u001b[0m Finished trial#62 resulted in value: 0.0368295498268566. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:37:33,876]\u001b[0m Finished trial#63 resulted in value: 0.03509041939207391. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:37:38,928]\u001b[0m Finished trial#64 resulted in value: 0.033351288957291336. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:37:44,586]\u001b[0m Finished trial#65 resulted in value: 0.0315813774528666. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:37:50,329]\u001b[0m Finished trial#66 resulted in value: 0.033320507887649065. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:37:57,477]\u001b[0m Finished trial#67 resulted in value: 0.033320507887649065. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:38:03,135]\u001b[0m Finished trial#68 resulted in value: 0.0333820700269335. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:38:04,569]\u001b[0m Finished trial#69 resulted in value: 0.37257406694882644. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:38:06,889]\u001b[0m Finished trial#70 resulted in value: 0.03866102347056555. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:38:09,983]\u001b[0m Finished trial#71 resulted in value: 0.04210850327048854. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:38:16,487]\u001b[0m Finished trial#72 resulted in value: 0.0368295498268566. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:38:18,903]\u001b[0m Finished trial#73 resulted in value: 0.03509041939207391. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:38:23,348]\u001b[0m Finished trial#74 resulted in value: 0.033320507887649065. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:38:27,602]\u001b[0m Finished trial#75 resulted in value: 0.03679876875721422. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:38:30,227]\u001b[0m Finished trial#76 resulted in value: 0.033351288957291336. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:38:35,301]\u001b[0m Finished trial#77 resulted in value: 0.033351288957291336. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:38:42,918]\u001b[0m Finished trial#78 resulted in value: 0.0315813774528666. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:38:48,311]\u001b[0m Finished trial#79 resulted in value: 0.033351288957291336. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:38:56,276]\u001b[0m Finished trial#80 resulted in value: 0.0368295498268566. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:39:04,730]\u001b[0m Finished trial#81 resulted in value: 0.0421392843401307. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:39:08,656]\u001b[0m Finished trial#82 resulted in value: 0.0315813774528666. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:39:11,340]\u001b[0m Finished trial#83 resulted in value: 0.038568680261639066. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:39:15,216]\u001b[0m Finished trial#84 resulted in value: 0.03505963832243175. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:39:18,538]\u001b[0m Finished trial#85 resulted in value: 0.03161215852250865. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:39:26,023]\u001b[0m Finished trial#86 resulted in value: 0.0368295498268566. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:39:32,066]\u001b[0m Finished trial#87 resulted in value: 0.02984224701808391. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:39:33,539]\u001b[0m Finished trial#88 resulted in value: 0.37257406694882644. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:39:39,646]\u001b[0m Finished trial#89 resulted in value: 0.03509041939207391. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:39:43,585]\u001b[0m Finished trial#90 resulted in value: 0.033320507887649065. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:39:50,294]\u001b[0m Finished trial#91 resulted in value: 0.038568680261639066. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:39:54,966]\u001b[0m Finished trial#92 resulted in value: 0.033320507887649065. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:39:58,678]\u001b[0m Finished trial#93 resulted in value: 0.0315813774528666. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:40:04,611]\u001b[0m Finished trial#94 resulted in value: 0.03505963832243175. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:40:10,040]\u001b[0m Finished trial#95 resulted in value: 0.0368295498268566. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:40:14,165]\u001b[0m Finished trial#96 resulted in value: 0.033351288957291336. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:40:21,377]\u001b[0m Finished trial#97 resulted in value: 0.03509041939207391. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:40:27,669]\u001b[0m Finished trial#98 resulted in value: 0.04210850327048876. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:40:31,346]\u001b[0m Finished trial#99 resulted in value: 0.03509041939207391. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:40:34,606]\u001b[0m Finished trial#100 resulted in value: 0.028103116583301335. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:40:38,030]\u001b[0m Finished trial#101 resulted in value: 0.03509041939207391. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:40:41,267]\u001b[0m Finished trial#102 resulted in value: 0.033351288957291336. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:40:45,648]\u001b[0m Finished trial#103 resulted in value: 0.0368295498268566. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:40:48,645]\u001b[0m Finished trial#104 resulted in value: 0.04030781069642175. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:40:50,885]\u001b[0m Finished trial#105 resulted in value: 0.0315813774528666. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:40:53,363]\u001b[0m Finished trial#106 resulted in value: 0.029903809157368344. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:40:55,839]\u001b[0m Finished trial#107 resulted in value: 0.029873028087726183. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:40:58,338]\u001b[0m Finished trial#108 resulted in value: 0.03512120046171607. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:41:00,673]\u001b[0m Finished trial#109 resulted in value: 0.038599461331281226. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:41:02,918]\u001b[0m Finished trial#110 resulted in value: 0.03863024240092339. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:41:04,400]\u001b[0m Finished trial#111 resulted in value: 0.37257406694882644. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:41:06,935]\u001b[0m Finished trial#112 resulted in value: 0.03161215852250865. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:41:09,345]\u001b[0m Finished trial#113 resulted in value: 0.033320507887649065. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:41:12,063]\u001b[0m Finished trial#114 resulted in value: 0.0333820700269335. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:41:14,372]\u001b[0m Finished trial#115 resulted in value: 0.028103116583301335. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:41:16,853]\u001b[0m Finished trial#116 resulted in value: 0.038599461331281226. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:41:19,765]\u001b[0m Finished trial#117 resulted in value: 0.02984224701808391. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:41:22,411]\u001b[0m Finished trial#118 resulted in value: 0.03509041939207391. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:41:25,552]\u001b[0m Finished trial#119 resulted in value: 0.03512120046171607. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:41:28,447]\u001b[0m Finished trial#120 resulted in value: 0.03509041939207391. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:41:31,236]\u001b[0m Finished trial#121 resulted in value: 0.03679876875721422. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:41:33,627]\u001b[0m Finished trial#122 resulted in value: 0.03679876875721422. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:41:36,784]\u001b[0m Finished trial#123 resulted in value: 0.03686033089649876. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:41:39,171]\u001b[0m Finished trial#124 resulted in value: 0.03686033089649854. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:41:42,085]\u001b[0m Finished trial#125 resulted in value: 0.03509041939207391. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:41:44,406]\u001b[0m Finished trial#126 resulted in value: 0.03682954982685638. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:41:48,076]\u001b[0m Finished trial#127 resulted in value: 0.0315813774528666. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:41:50,848]\u001b[0m Finished trial#128 resulted in value: 0.0315813774528666. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:41:53,376]\u001b[0m Finished trial#129 resulted in value: 0.038599461331281226. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:41:57,959]\u001b[0m Finished trial#130 resulted in value: 0.02981146594844175. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:42:02,196]\u001b[0m Finished trial#131 resulted in value: 0.038568680261639066. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:42:04,954]\u001b[0m Finished trial#132 resulted in value: 0.05611388995767608. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:42:09,883]\u001b[0m Finished trial#133 resulted in value: 0.033320507887649065. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:42:13,422]\u001b[0m Finished trial#134 resulted in value: 0.03509041939207391. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:42:17,957]\u001b[0m Finished trial#135 resulted in value: 0.0315813774528666. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:42:21,343]\u001b[0m Finished trial#136 resulted in value: 0.033351288957291336. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:42:25,443]\u001b[0m Finished trial#137 resulted in value: 0.03509041939207391. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:42:29,222]\u001b[0m Finished trial#138 resulted in value: 0.033320507887649065. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:42:33,640]\u001b[0m Finished trial#139 resulted in value: 0.031550596383224216. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:42:38,010]\u001b[0m Finished trial#140 resulted in value: 0.03509041939207391. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:42:42,084]\u001b[0m Finished trial#141 resulted in value: 0.033320507887649065. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:42:46,730]\u001b[0m Finished trial#142 resulted in value: 0.02984224701808391. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:42:51,313]\u001b[0m Finished trial#143 resulted in value: 0.03509041939207391. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:42:56,133]\u001b[0m Finished trial#144 resulted in value: 0.02984224701808391. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:43:00,402]\u001b[0m Finished trial#145 resulted in value: 0.03161215852250865. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:43:05,442]\u001b[0m Finished trial#146 resulted in value: 0.03679876875721422. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:43:10,176]\u001b[0m Finished trial#147 resulted in value: 0.0315813774528666. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:43:15,356]\u001b[0m Finished trial#148 resulted in value: 0.033320507887649065. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:43:19,851]\u001b[0m Finished trial#149 resulted in value: 0.033320507887649065. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:43:24,656]\u001b[0m Finished trial#150 resulted in value: 0.033351288957291336. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:43:26,101]\u001b[0m Finished trial#151 resulted in value: 0.37257406694882644. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:43:30,710]\u001b[0m Finished trial#152 resulted in value: 0.0368295498268566. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:43:33,286]\u001b[0m Finished trial#153 resulted in value: 0.029903809157368344. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:43:35,762]\u001b[0m Finished trial#154 resulted in value: 0.03161215852250865. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:43:38,380]\u001b[0m Finished trial#155 resulted in value: 0.0368295498268566. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:43:40,973]\u001b[0m Finished trial#156 resulted in value: 0.033351288957291336. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 08:43:43,282]\u001b[0m Finished trial#157 resulted in value: 0.02981146594844175. Current best value is 0.028072335513659064 with parameters: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}.\u001b[0m\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"params: {'classifier': 'RandomForestClassifier', 'n_estimators': 401.4209045886614, 'max_depth': 8.917942336290903}\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "a9unb17s271w",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 542
},
"outputId": "06bdf1ad-3494-48f4-f385-78da4853d56e"
},
"source": [
"optuna.visualization.plot_contour(study, params=['n_estimators', 'max_depth'])"
],
"execution_count": 51,
"outputs": [
{
"output_type": "display_data",
"data": {
"text/html": [
"<html>\n",
"<head><meta charset=\"utf-8\" /></head>\n",
"<body>\n",
" <div>\n",
" <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax) {MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>\n",
" <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
" <script src=\"https://cdn.plot.ly/plotly-latest.min.js\"></script> \n",
" <div id=\"649b2227-193f-4bf5-a5ee-d2e018ef0608\" class=\"plotly-graph-div\" style=\"height:525px; width:100%;\"></div>\n",
" <script type=\"text/javascript\">\n",
" \n",
" window.PLOTLYENV=window.PLOTLYENV || {};\n",
" \n",
" if (document.getElementById(\"649b2227-193f-4bf5-a5ee-d2e018ef0608\")) {\n",
" Plotly.newPlot(\n",
" '649b2227-193f-4bf5-a5ee-d2e018ef0608',\n",
" [{\"colorbar\": {\"title\": {\"text\": \"Objective Value\"}}, \"colorscale\": [[0, \"rgb(220,220,220)\"], [0.30000000000000004, \"rgb(106,137,247)\"], [0.4, \"rgb(90,120,245)\"], [0.5, \"rgb(70,100,245)\"], [0.65, \"rgb(40,60,190)\"], [1, \"rgb(5,10,172)\"]], \"connectgaps\": true, \"contours\": {\"coloring\": \"heatmap\"}, \"hoverinfo\": \"none\", \"line\": {\"smoothing\": 1.3}, \"type\": \"contour\", \"x\": [2.047478401554478, 2.4064292904008098, 2.6265083979140798, 2.8010877232808986, 2.99879480578323, 3.0401285807321834, 3.91960848355412, 4.210226494509033, 4.378904777572097, 4.992921956739448, 5.10553555469419, 5.223090214263294, 5.60213355083506, 5.9690456473131395, 6.072548405363887, 6.184457511928801, 6.221007269688926, 6.285963404198077, 6.540801783652664, 6.592663620836698, 6.636172307276976, 6.785308307608685, 6.974028577681278, 7.099253890186411, 7.137278745733164, 7.190985704509367, 7.355726174735931, 7.399298799301023, 7.461844221098603, 7.573747510168445, 7.635917092924144, 7.707937454182399, 7.7147575689621055, 7.790253384491404, 7.930840534773581, 7.9528111510089055, 8.0567157055174, 8.074330973905067, 8.094194628004278, 8.354274039975628, 8.3582229218975, 8.373853378131345, 8.470326875174424, 8.501234880991907, 8.67268399552629, 8.728957150910754, 8.752561301914994, 8.755277285407926, 8.917942336290903, 8.95437464392877, 8.997218889248828, 9.133063137846786, 9.194770811029073, 9.194783848297979, 9.226421982850697, 9.278869122560033, 9.518262531257742, 9.600986921079112, 9.61128344255445, 9.674863215524189, 9.678503275016899, 9.694120219281483, 9.925491336604733, 9.955216364549047, 9.968910601923488, 9.995105706859041, 10.002610182831672, 10.072402360846876, 10.215423433508498, 10.289288158176056, 10.325830237549027, 10.373596573495051, 10.37573735004665, 10.494236126866042, 10.498820505439046, 10.522936585027416, 10.583468902025444, 10.64461538560894, 10.828632048526348, 10.887143635017248, 10.932073225764057, 11.060362408241177, 11.163395362827751, 11.177646783602173, 11.189377261826511, 11.454797884570915, 11.458853452852555, 11.482754364342282, 11.512406968117618, 11.662387451910517, 11.688188303578295, 11.749108054762049, 11.79691976597307, 11.880552437609458, 11.892307511601363, 11.926277821395162, 11.937862413820499, 12.00014922678741, 12.03983274861621, 12.184463355170081, 12.267477679501317, 12.293898198580171, 12.31820795978171, 12.380818955414258, 12.390908705391187, 12.408834265757974, 12.509657170496965, 12.686428354208253, 12.688423502409304, 12.843162069471518, 12.918236501586357, 12.998139730316067, 13.12583134705724, 13.154069609964786, 13.421526337107505, 13.426179442740926, 13.439746717512097, 13.58147034483273, 13.761328732407303, 13.791074717542783, 13.930754193010792, 14.034896630464848, 14.084770020372133, 14.239459639822327, 14.304223080391752, 14.61145297674688, 14.837435149236986, 15.117056228798226, 15.290534820592978, 15.51593943467453, 16.007782552511546, 16.020886550439204, 16.105924238484405, 16.459067099516993, 17.176316904253056, 17.1789688271964, 18.17003708256162, 18.248095893161302, 20.351076070693924, 21.00902538979861, 24.647702694383405, 29.613809463679353, 30.685385014318143], \"y\": [100.21749500958887, 100.45193600131003, 101.98070794590456, 102.29044454802505, 108.98629300416444, 109.50900540721139, 112.6452254858218, 114.74231569637807, 115.98513965585452, 116.39179856236392, 122.3086706709285, 122.67339037382612, 122.88513245758456, 129.48188357575413, 131.18240074613016, 132.70725671144325, 134.97364874846355, 136.1944539445815, 136.7975310059347, 138.6120972749781, 139.13118292224263, 140.48360332877328, 144.67750143713724, 146.58163121355147, 147.7542608320675, 147.82789755010265, 150.03661636818424, 150.30292187560207, 152.8418749725504, 153.2281903374361, 157.5403739099849, 158.43045268997415, 163.5664876908483, 166.7270008800896, 172.12341914582498, 175.74542871258714, 178.65220421825168, 182.64279557490207, 188.1624398285308, 188.79254376344943, 192.9324303776738, 193.64446351271258, 199.23897572891207, 201.6598290571611, 205.55119892895365, 207.7925223808929, 214.98726832979264, 217.77168255712826, 220.70981397499204, 227.45737104225668, 227.8697090936671, 238.36746639880164, 239.71083568783868, 243.2529880338835, 248.47453077129546, 251.59901351347284, 253.4860019360312, 253.6043363892294, 256.82256059075337, 260.51103281719736, 269.2109143081002, 271.9889859440977, 275.90517801312456, 285.01853227351523, 287.0511923451615, 295.0548669745551, 299.12256221876197, 300.05935929076367, 301.8303136644955, 308.0222388567993, 308.4258783552839, 313.8492325071967, 319.722617874184, 324.61425582158387, 326.59644166422277, 334.67362241002274, 335.23934464938856, 339.7714075087774, 341.09293143172687, 346.9580393181343, 351.1634546484884, 358.42469879663884, 361.3585010526889, 363.82089391232097, 366.3607084482093, 371.6687844364495, 380.8335298738248, 380.9707720778235, 382.1286355222168, 383.2848458255742, 388.17858842700736, 393.72507729925275, 394.0447298507991, 401.4209045886614, 403.1416155393638, 403.2870994732765, 407.8165477243672, 412.9443528939518, 414.6855835765124, 416.36052185121906, 418.24429991339167, 425.1908211196507, 428.445801857055, 435.5854507138769, 439.42652713994755, 440.52171529759204, 440.5333006067582, 453.05880502185533, 456.5235462335618, 469.33695166779756, 470.49958757979135, 478.00122634339186, 490.1589877235784, 500.83243496887553, 502.3106158085951, 510.0992166050814, 517.2893020696505, 518.0246629079226, 544.6439913309417, 548.3237183054225, 552.835078252529, 559.134373753873, 583.970572535274, 599.7768040445576, 604.4008422747855, 608.5693246990731, 626.1459437348685, 631.2135492294358, 638.8598163308294, 649.8612885095688, 663.1388550450731, 687.9138020890699, 713.5525837579337, 746.8604846541687, 748.3379198559102, 781.5330571726804, 790.7602773011945, 811.0568429319754, 852.5532098736315, 878.9939920009782, 898.3901263289507, 960.0340363566861, 991.9439943881974], \"z\": [[null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.03161215852250865, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.038599461331281226, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.03161215852250865, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.03863024240092339, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.03682954982685638, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.0315813774528666, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.028103116583301335, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.03866102347056555, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.038599461331281226, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.02981146594844175, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.03686033089649854, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.03679876875721422, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.035182762601000506, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.033320507887649065, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, 0.03509041939207391, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.038599461331281226, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.03161215852250865, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.029873028087726183, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.029903809157368344, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.038599461331281226, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.03512120046171607, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, 0.038568680261639066, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.03161215852250865, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.033351288957291336, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, 0.04390919584455555, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.03686033089649854, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.02987302808772596, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.029903809157368344, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.0315813774528666, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.03509041939207391, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.0368295498268566, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.033351288957291336, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.038568680261639066, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.0333820700269335, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.0315813774528666, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.03679876875721422, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.03505963832243175, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.0368295498268566, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.03505963832243175, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.03509041939207391, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.03509041939207391, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.02984224701808391, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, 0.05611388995767608, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.03686033089649876, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.04030781069642175, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, 0.0473874567141207, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.03512120046171607, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.04210850327048854, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.03509041939207391, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [0.06139284340130824, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.03686033089649876, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.033351288957291336, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.028103116583301335, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.03161215852250865, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, 0.04387841477491339, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.033351288957291336, null, null, null, null, null], [null, null, 0.050865717583686076, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.03509041939207391, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.033351288957291336, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.03509041939207391, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.033351288957291225, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.03509041939207391, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.03509041939207391, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.03509041939207391, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.0315813774528666, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.03509041939207391, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.038599461331281226, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.0315813774528666, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.03509041939207391, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.033320507887649065, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.03155059638322444, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.03505963832243175, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.0315813774528666, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.03509041939207391, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.033320507887649065, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.033351288957291336, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.033320507887649065, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.029873028087726183, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.033351288957291336, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.03509041939207391, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.033351288957291336, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.03679876875721422, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.038568680261639066, null, null, null, null, null, null, null], [null, 0.05965371296652555, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.03161215852250865, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.033320507887649065, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.03509041939207391, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.031550596383224216, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.0368295498268566, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.03509041939207391, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.0315813774528666, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.033320507887649065, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.033320507887649065, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.028072335513659064, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.0315813774528666, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.0368295498268566, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.02981146594844175, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.03509041939207391, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.0368295498268566, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.02984224701808391, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.03161215852250865, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.033320507887649065, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.0315813774528666, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.033320507887649065, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.033351288957291336, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.02984224701808391, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.03509041939207391, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.033320507887649065, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.038599461331281226], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.03679876875721422, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.033351288957291336, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.033351288957291336, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.033320507887649065, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, 0.05968449403616771, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, 0.038568680261639066, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.033320507887649065, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.033351288957291336, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.0368295498268566, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.0315813774528666, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.0333820700269335, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.033351288957291225, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.033320507887649065, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.03505963832243175, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.02984224701808391, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.03509041939207391, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.03509041939207391, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.033320507887649065, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.028133897652943496, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.033320507887649065, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, 0.04210850327048876, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.0368295498268566, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.038568680261639066, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, 0.045648326279338236, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.03509041939207391, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.033320507887649065, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.03505963832243175, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.0368295498268566, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.0315813774528666, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.0368295498268566, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.033320507887649065, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.03509041939207391, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, 0.0421392843401307, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null], [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0.03509041939207391, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null]]}, {\"marker\": {\"color\": \"black\"}, \"mode\": \"markers\", \"showlegend\": false, \"type\": \"scatter\", \"x\": [2.8010877232808986, 3.0401285807321834, 10.215423433508498, 29.613809463679353, 2.6265083979140798, 2.4064292904008098, 3.91960848355412, 8.470326875174424, 24.647702694383405, 18.248095893161302, 10.64461538560894, 16.007782552511546, 14.034896630464848, 6.072548405363887, 5.9690456473131395, 17.176316904253056, 11.79691976597307, 12.00014922678741, 7.355726174735931, 9.995105706859041, 4.210226494509033, 6.540801783652664, 9.674863215524189, 12.31820795978171, 21.00902538979861, 8.917942336290903, 5.223090214263294, 8.3582229218975, 13.761328732407303, 8.0567157055174, 13.930754193010792, 8.752561301914994, 10.37573735004665, 11.662387451910517, 11.454797884570915, 7.137278745733164, 20.351076070693924, 4.992921956739448, 14.837435149236986, 12.686428354208253, 8.95437464392877, 9.226421982850697, 7.573747510168445, 2.047478401554478, 11.060362408241177, 30.685385014318143, 16.020886550439204, 9.694120219281483, 6.592663620836698, 12.267477679501317, 18.17003708256162, 9.600986921079112, 10.583468902025444, 8.728957150910754, 10.373596573495051, 12.998139730316067, 15.51593943467453, 7.930840534773581, 11.177646783602173, 6.974028577681278, 6.184457511928801, 5.60213355083506, 10.072402360846876, 8.755277285407926, 7.399298799301023, 11.512406968117618, 9.194783848297979, 13.58147034483273, 9.278869122560033, 4.378904777572097, 10.494236126866042, 8.094194628004278, 10.522936585027416, 12.390908705391187, 9.133063137846786, 7.790253384491404, 8.074330973905067, 6.785308307608685, 7.461844221098603, 11.749108054762049, 6.285963404198077, 9.925491336604733, 8.354274039975628, 10.828632048526348, 8.67268399552629, 5.10553555469419, 6.221007269688926, 13.12583134705724, 16.459067099516993, 15.117056228798226, 14.304223080391752, 13.154069609964786, 9.194770811029073, 11.892307511601363, 12.03983274861621, 11.937862413820499, 14.61145297674688, 11.163395362827751, 12.184463355170081, 13.426179442740926, 12.843162069471518, 9.955216364549047, 9.968910601923488, 7.635917092924144, 11.458853452852555, 7.707937454182399, 9.61128344255445, 7.099253890186411, 8.373853378131345, 11.189377261826511, 14.084770020372133, 8.997218889248828, 9.518262531257742, 7.7147575689621055, 10.498820505439046, 7.190985704509367, 12.380818955414258, 17.1789688271964, 2.99879480578323, 12.509657170496965, 10.289288158176056, 11.926277821395162, 7.9528111510089055, 10.932073225764057, 6.636172307276976, 13.421526337107505, 13.439746717512097, 15.290534820592978, 12.408834265757974, 12.688423502409304, 11.688188303578295, 14.239459639822327, 11.880552437609458, 12.918236501586357, 11.482754364342282, 13.791074717542783, 10.887143635017248, 16.105924238484405, 10.002610182831672, 9.678503275016899, 10.325830237549027, 12.293898198580171, 8.501234880991907], \"y\": [500.83243496887553, 248.47453077129546, 308.4258783552839, 201.6598290571611, 253.4860019360312, 363.82089391232097, 207.7925223808929, 220.70981397499204, 253.6043363892294, 251.59901351347284, 898.3901263289507, 101.98070794590456, 100.21749500958887, 100.45193600131003, 140.48360332877328, 781.5330571726804, 638.8598163308294, 150.03661636818424, 152.8418749725504, 388.17858842700736, 147.7542608320675, 147.82789755010265, 383.2848458255742, 510.0992166050814, 324.61425582158387, 401.4209045886614, 502.3106158085951, 626.1459437348685, 299.12256221876197, 435.5854507138769, 178.65220421825168, 440.5333006067582, 301.8303136644955, 631.2135492294358, 604.4008422747855, 878.9939920009782, 122.88513245758456, 713.5525837579337, 991.9439943881974, 182.64279557490207, 339.7714075087774, 334.67362241002274, 271.9889859440977, 227.45737104225668, 552.835078252529, 456.5235462335618, 371.6687844364495, 269.2109143081002, 418.24429991339167, 351.1634546484884, 188.1624398285308, 403.2870994732765, 285.01853227351523, 470.49958757979135, 544.6439913309417, 559.134373753873, 748.3379198559102, 548.3237183054225, 114.74231569637807, 217.77168255712826, 663.1388550450731, 131.18240074613016, 393.72507729925275, 358.42469879663884, 158.43045268997415, 478.00122634339186, 811.0568429319754, 517.2893020696505, 852.5532098736315, 960.0340363566861, 319.722617874184, 163.5664876908483, 313.8492325071967, 243.2529880338835, 790.7602773011945, 599.7768040445576, 608.5693246990731, 326.59644166422277, 687.9138020890699, 425.1908211196507, 300.05935929076367, 583.970572535274, 518.0246629079226, 341.09293143172687, 746.8604846541687, 649.8612885095688, 295.0548669745551, 239.71083568783868, 260.51103281719736, 238.36746639880164, 382.1286355222168, 205.55119892895365, 109.50900540721139, 136.7975310059347, 136.1944539445815, 139.13118292224263, 115.98513965585452, 102.29044454802505, 144.67750143713724, 129.48188357575413, 166.7270008800896, 112.6452254858218, 132.70725671144325, 193.64446351271258, 153.2281903374361, 214.98726832979264, 188.79254376344943, 175.74542871258714, 122.67339037382612, 227.8697090936671, 122.3086706709285, 192.9324303776738, 108.98629300416444, 287.0511923451615, 172.12341914582498, 138.6120972749781, 407.8165477243672, 361.3585010526889, 199.23897572891207, 453.05880502185533, 275.90517801312456, 403.1416155393638, 256.82256059075337, 346.9580393181343, 308.0222388567993, 380.9707720778235, 380.8335298738248, 335.23934464938856, 416.36052185121906, 412.9443528939518, 440.52171529759204, 366.3607084482093, 469.33695166779756, 428.445801857055, 490.1589877235784, 394.0447298507991, 439.42652713994755, 414.6855835765124, 150.30292187560207, 134.97364874846355, 157.5403739099849, 146.58163121355147, 116.39179856236392]}],\n",
" {\"template\": {\"data\": {\"bar\": [{\"error_x\": {\"color\": \"#2a3f5f\"}, \"error_y\": {\"color\": \"#2a3f5f\"}, \"marker\": {\"line\": {\"color\": \"#E5ECF6\", \"width\": 0.5}}, \"type\": \"bar\"}], \"barpolar\": [{\"marker\": {\"line\": {\"color\": \"#E5ECF6\", \"width\": 0.5}}, \"type\": \"barpolar\"}], \"carpet\": [{\"aaxis\": {\"endlinecolor\": \"#2a3f5f\", \"gridcolor\": \"white\", \"linecolor\": \"white\", \"minorgridcolor\": \"white\", \"startlinecolor\": \"#2a3f5f\"}, \"baxis\": {\"endlinecolor\": \"#2a3f5f\", \"gridcolor\": \"white\", \"linecolor\": \"white\", \"minorgridcolor\": \"white\", \"startlinecolor\": \"#2a3f5f\"}, \"type\": \"carpet\"}], \"choropleth\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"choropleth\"}], \"contour\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"contour\"}], \"contourcarpet\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"contourcarpet\"}], \"heatmap\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"heatmap\"}], \"heatmapgl\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"heatmapgl\"}], \"histogram\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"histogram\"}], \"histogram2d\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"histogram2d\"}], \"histogram2dcontour\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"histogram2dcontour\"}], \"mesh3d\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"mesh3d\"}], \"parcoords\": [{\"line\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"parcoords\"}], \"scatter\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatter\"}], \"scatter3d\": [{\"line\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatter3d\"}], \"scattercarpet\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattercarpet\"}], \"scattergeo\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattergeo\"}], \"scattergl\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattergl\"}], \"scattermapbox\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattermapbox\"}], \"scatterpolar\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterpolar\"}], \"scatterpolargl\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterpolargl\"}], \"scatterternary\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterternary\"}], \"surface\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"surface\"}], \"table\": [{\"cells\": {\"fill\": {\"color\": \"#EBF0F8\"}, \"line\": {\"color\": \"white\"}}, \"header\": {\"fill\": {\"color\": \"#C8D4E3\"}, \"line\": {\"color\": \"white\"}}, \"type\": \"table\"}]}, \"layout\": {\"annotationdefaults\": {\"arrowcolor\": \"#2a3f5f\", \"arrowhead\": 0, \"arrowwidth\": 1}, \"colorscale\": {\"diverging\": [[0, \"#8e0152\"], [0.1, \"#c51b7d\"], [0.2, \"#de77ae\"], [0.3, \"#f1b6da\"], [0.4, \"#fde0ef\"], [0.5, \"#f7f7f7\"], [0.6, \"#e6f5d0\"], [0.7, \"#b8e186\"], [0.8, \"#7fbc41\"], [0.9, \"#4d9221\"], [1, \"#276419\"]], \"sequential\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"sequentialminus\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]]}, \"colorway\": [\"#636efa\", \"#EF553B\", \"#00cc96\", \"#ab63fa\", \"#FFA15A\", \"#19d3f3\", \"#FF6692\", \"#B6E880\", \"#FF97FF\", \"#FECB52\"], \"font\": {\"color\": \"#2a3f5f\"}, \"geo\": {\"bgcolor\": \"white\", \"lakecolor\": \"white\", \"landcolor\": \"#E5ECF6\", \"showlakes\": true, \"showland\": true, \"subunitcolor\": \"white\"}, \"hoverlabel\": {\"align\": \"left\"}, \"hovermode\": \"closest\", \"mapbox\": {\"style\": \"light\"}, \"paper_bgcolor\": \"white\", \"plot_bgcolor\": \"#E5ECF6\", \"polar\": {\"angularaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"bgcolor\": \"#E5ECF6\", \"radialaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}}, \"scene\": {\"xaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}, \"yaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}, \"zaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}}, \"shapedefaults\": {\"line\": {\"color\": \"#2a3f5f\"}}, \"ternary\": {\"aaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"baxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"bgcolor\": \"#E5ECF6\", \"caxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}}, \"title\": {\"x\": 0.05}, \"xaxis\": {\"automargin\": true, \"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\", \"zerolinecolor\": \"white\", \"zerolinewidth\": 2}, \"yaxis\": {\"automargin\": true, \"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\", \"zerolinecolor\": \"white\", \"zerolinewidth\": 2}}}, \"xaxis\": {\"range\": [0.3112193291698519, 1.4869315768148281], \"title\": {\"text\": \"max_depth\"}, \"type\": \"log\"}, \"yaxis\": {\"range\": [2.000943543117004, 2.9964871523812047], \"title\": {\"text\": \"n_estimators\"}, \"type\": \"log\"}},\n",
" {\"responsive\": true}\n",
" ).then(function(){\n",
" \n",
"var gd = document.getElementById('649b2227-193f-4bf5-a5ee-d2e018ef0608');\n",
"var x = new MutationObserver(function (mutations, observer) {{\n",
" var display = window.getComputedStyle(gd).display;\n",
" if (!display || display === 'none') {{\n",
" console.log([gd, 'removed!']);\n",
" Plotly.purge(gd);\n",
" observer.disconnect();\n",
" }}\n",
"}});\n",
"\n",
"// Listen for the removal of the full notebook cells\n",
"var notebookContainer = gd.closest('#notebook-container');\n",
"if (notebookContainer) {{\n",
" x.observe(notebookContainer, {childList: true});\n",
"}}\n",
"\n",
"// Listen for the clearing of the current output cell\n",
"var outputEl = gd.closest('.output');\n",
"if (outputEl) {{\n",
" x.observe(outputEl, {childList: true});\n",
"}}\n",
"\n",
" })\n",
" };\n",
" \n",
" </script>\n",
" </div>\n",
"</body>\n",
"</html>"
]
},
"metadata": {
"tags": []
}
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "TtiZbOEeC-jn",
"colab_type": "text"
},
"source": [
"### Databaseも使えるらしい\n",
"\n",
"\n",
"- InMemoryStorage \n",
" - デフォルトで使用されるメモリ上のストレージ\n",
" - optuna.create_study() \n",
"- RDBStorage\n",
" - RDBをバックエンドにしたストレージ\n",
" - SQLAlchemyがサポートするRDBが使用可能 \n",
" - E.g., MySQL, PostgreSQL, SQLite \n",
" - SQLiteはファイルベースのRDB (標準ライブラリに組み込み)\n",
" - optuna.create_study(storage=RDB_URL) \n",
"\n",
"##### ストレージの使い分け \n",
"- とりあえずOptunaを動かしてみたい\n",
" - InMemoryStorage\n",
"- 分散最適化\n",
" - SQLite以外のRDBStorage (e.g., MySQL, PostgreSQL) \n",
" - SQLiteは、NFSと相性が悪く、スケールもしないので注意\n",
"- Studyを中断・再開したり、後から結果を分析したい\n",
" - RDBStorage 「とりあえずSQLiteに保存」"
]
},
{
"cell_type": "code",
"metadata": {
"id": "GSl3vJIf5WT6",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 378
},
"outputId": "bc9cbc15-2a58-435c-aaa6-603bc48fc7bb"
},
"source": [
"study_name = 'example-study'\n",
"study = optuna.create_study(study_name=study_name,\n",
" storage='sqlite:///optuna_study.db',\n",
" load_if_exists=True)\n",
"study.optimize(f, n_trials=20)"
],
"execution_count": 64,
"outputs": [
{
"output_type": "stream",
"text": [
"\u001b[32m[I 2019-11-20 09:01:37,363]\u001b[0m A new study created with name: example-study\u001b[0m\n",
"\u001b[32m[I 2019-11-20 09:01:39,239]\u001b[0m Finished trial#0 resulted in value: 0.37257406694882644. Current best value is 0.37257406694882644 with parameters: {'C': 10.904882712437471, 'classifier': 'SVC', 'gamma': 0.24448039340068206, 'kernel': 'rbf'}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 09:01:41,294]\u001b[0m Finished trial#1 resulted in value: 0.37257406694882644. Current best value is 0.37257406694882644 with parameters: {'C': 10.904882712437471, 'classifier': 'SVC', 'gamma': 0.24448039340068206, 'kernel': 'rbf'}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 09:01:48,383]\u001b[0m Finished trial#2 resulted in value: 0.03505963832243175. Current best value is 0.03505963832243175 with parameters: {'classifier': 'RandomForestClassifier', 'max_depth': 6.108798079653031, 'n_estimators': 651.6221066697033}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 09:01:53,491]\u001b[0m Finished trial#3 resulted in value: 0.04210850327048854. Current best value is 0.03505963832243175 with parameters: {'classifier': 'RandomForestClassifier', 'max_depth': 6.108798079653031, 'n_estimators': 651.6221066697033}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 09:02:00,111]\u001b[0m Finished trial#4 resulted in value: 0.03509041939207391. Current best value is 0.03505963832243175 with parameters: {'classifier': 'RandomForestClassifier', 'max_depth': 6.108798079653031, 'n_estimators': 651.6221066697033}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 09:02:04,932]\u001b[0m Finished trial#5 resulted in value: 0.038568680261639066. Current best value is 0.03505963832243175 with parameters: {'classifier': 'RandomForestClassifier', 'max_depth': 6.108798079653031, 'n_estimators': 651.6221066697033}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 09:02:13,759]\u001b[0m Finished trial#6 resulted in value: 0.0315813774528666. Current best value is 0.0315813774528666 with parameters: {'classifier': 'RandomForestClassifier', 'max_depth': 11.101753501515459, 'n_estimators': 827.235808620839}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 09:02:15,755]\u001b[0m Finished trial#7 resulted in value: 0.37257406694882644. Current best value is 0.0315813774528666 with parameters: {'classifier': 'RandomForestClassifier', 'max_depth': 11.101753501515459, 'n_estimators': 827.235808620839}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 09:02:17,775]\u001b[0m Finished trial#8 resulted in value: 0.37257406694882644. Current best value is 0.0315813774528666 with parameters: {'classifier': 'RandomForestClassifier', 'max_depth': 11.101753501515459, 'n_estimators': 827.235808620839}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 09:02:24,022]\u001b[0m Finished trial#9 resulted in value: 0.033320507887649065. Current best value is 0.0315813774528666 with parameters: {'classifier': 'RandomForestClassifier', 'max_depth': 11.101753501515459, 'n_estimators': 827.235808620839}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 09:02:26,866]\u001b[0m Finished trial#10 resulted in value: 0.038568680261639066. Current best value is 0.0315813774528666 with parameters: {'classifier': 'RandomForestClassifier', 'max_depth': 11.101753501515459, 'n_estimators': 827.235808620839}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 09:02:31,074]\u001b[0m Finished trial#11 resulted in value: 0.057914582531742864. Current best value is 0.0315813774528666 with parameters: {'classifier': 'RandomForestClassifier', 'max_depth': 11.101753501515459, 'n_estimators': 827.235808620839}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 09:02:39,220]\u001b[0m Finished trial#12 resulted in value: 0.03679876875721422. Current best value is 0.0315813774528666 with parameters: {'classifier': 'RandomForestClassifier', 'max_depth': 11.101753501515459, 'n_estimators': 827.235808620839}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 09:02:42,617]\u001b[0m Finished trial#13 resulted in value: 0.038599461331281226. Current best value is 0.0315813774528666 with parameters: {'classifier': 'RandomForestClassifier', 'max_depth': 11.101753501515459, 'n_estimators': 827.235808620839}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 09:02:52,460]\u001b[0m Finished trial#14 resulted in value: 0.033320507887649065. Current best value is 0.0315813774528666 with parameters: {'classifier': 'RandomForestClassifier', 'max_depth': 11.101753501515459, 'n_estimators': 827.235808620839}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 09:03:02,333]\u001b[0m Finished trial#15 resulted in value: 0.0368295498268566. Current best value is 0.0315813774528666 with parameters: {'classifier': 'RandomForestClassifier', 'max_depth': 11.101753501515459, 'n_estimators': 827.235808620839}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 09:03:08,389]\u001b[0m Finished trial#16 resulted in value: 0.03682954982685638. Current best value is 0.0315813774528666 with parameters: {'classifier': 'RandomForestClassifier', 'max_depth': 11.101753501515459, 'n_estimators': 827.235808620839}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 09:03:12,002]\u001b[0m Finished trial#17 resulted in value: 0.033320507887649065. Current best value is 0.0315813774528666 with parameters: {'classifier': 'RandomForestClassifier', 'max_depth': 11.101753501515459, 'n_estimators': 827.235808620839}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 09:03:20,139]\u001b[0m Finished trial#18 resulted in value: 0.033351288957291336. Current best value is 0.0315813774528666 with parameters: {'classifier': 'RandomForestClassifier', 'max_depth': 11.101753501515459, 'n_estimators': 827.235808620839}.\u001b[0m\n",
"\u001b[32m[I 2019-11-20 09:03:25,940]\u001b[0m Finished trial#19 resulted in value: 0.03509041939207391. Current best value is 0.0315813774528666 with parameters: {'classifier': 'RandomForestClassifier', 'max_depth': 11.101753501515459, 'n_estimators': 827.235808620839}.\u001b[0m\n"
],
"name": "stderr"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "zyiF3vA2HdaL",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 542
},
"outputId": "675ff68f-5d7e-4f79-a8c0-70f4f7f1fbcc"
},
"source": [
"plot_optimization_history(study)"
],
"execution_count": 71,
"outputs": [
{
"output_type": "display_data",
"data": {
"text/html": [
"<html>\n",
"<head><meta charset=\"utf-8\" /></head>\n",
"<body>\n",
" <div>\n",
" <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax) {MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>\n",
" <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
" <script src=\"https://cdn.plot.ly/plotly-latest.min.js\"></script> \n",
" <div id=\"e514d4b1-5ed2-4ed7-8963-15ce6cf4761f\" class=\"plotly-graph-div\" style=\"height:525px; width:100%;\"></div>\n",
" <script type=\"text/javascript\">\n",
" \n",
" window.PLOTLYENV=window.PLOTLYENV || {};\n",
" \n",
" if (document.getElementById(\"e514d4b1-5ed2-4ed7-8963-15ce6cf4761f\")) {\n",
" Plotly.newPlot(\n",
" 'e514d4b1-5ed2-4ed7-8963-15ce6cf4761f',\n",
" [{\"mode\": \"markers\", \"name\": \"Objective Value\", \"type\": \"scatter\", \"x\": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19], \"y\": [0.37257406694882644, 0.37257406694882644, 0.03505963832243175, 0.04210850327048854, 0.03509041939207391, 0.038568680261639066, 0.0315813774528666, 0.37257406694882644, 0.37257406694882644, 0.033320507887649065, 0.038568680261639066, 0.057914582531742864, 0.03679876875721422, 0.038599461331281226, 0.033320507887649065, 0.0368295498268566, 0.03682954982685638, 0.033320507887649065, 0.033351288957291336, 0.03509041939207391]}, {\"name\": \"Best Value\", \"type\": \"scatter\", \"x\": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19], \"y\": [0.37257406694882644, 0.37257406694882644, 0.03505963832243175, 0.03505963832243175, 0.03505963832243175, 0.03505963832243175, 0.0315813774528666, 0.0315813774528666, 0.0315813774528666, 0.0315813774528666, 0.0315813774528666, 0.0315813774528666, 0.0315813774528666, 0.0315813774528666, 0.0315813774528666, 0.0315813774528666, 0.0315813774528666, 0.0315813774528666, 0.0315813774528666, 0.0315813774528666]}],\n",
" {\"template\": {\"data\": {\"bar\": [{\"error_x\": {\"color\": \"#2a3f5f\"}, \"error_y\": {\"color\": \"#2a3f5f\"}, \"marker\": {\"line\": {\"color\": \"#E5ECF6\", \"width\": 0.5}}, \"type\": \"bar\"}], \"barpolar\": [{\"marker\": {\"line\": {\"color\": \"#E5ECF6\", \"width\": 0.5}}, \"type\": \"barpolar\"}], \"carpet\": [{\"aaxis\": {\"endlinecolor\": \"#2a3f5f\", \"gridcolor\": \"white\", \"linecolor\": \"white\", \"minorgridcolor\": \"white\", \"startlinecolor\": \"#2a3f5f\"}, \"baxis\": {\"endlinecolor\": \"#2a3f5f\", \"gridcolor\": \"white\", \"linecolor\": \"white\", \"minorgridcolor\": \"white\", \"startlinecolor\": \"#2a3f5f\"}, \"type\": \"carpet\"}], \"choropleth\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"choropleth\"}], \"contour\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"contour\"}], \"contourcarpet\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"contourcarpet\"}], \"heatmap\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"heatmap\"}], \"heatmapgl\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"heatmapgl\"}], \"histogram\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"histogram\"}], \"histogram2d\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"histogram2d\"}], \"histogram2dcontour\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"histogram2dcontour\"}], \"mesh3d\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"mesh3d\"}], \"parcoords\": [{\"line\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"parcoords\"}], \"scatter\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatter\"}], \"scatter3d\": [{\"line\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatter3d\"}], \"scattercarpet\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattercarpet\"}], \"scattergeo\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattergeo\"}], \"scattergl\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattergl\"}], \"scattermapbox\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattermapbox\"}], \"scatterpolar\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterpolar\"}], \"scatterpolargl\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterpolargl\"}], \"scatterternary\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterternary\"}], \"surface\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"surface\"}], \"table\": [{\"cells\": {\"fill\": {\"color\": \"#EBF0F8\"}, \"line\": {\"color\": \"white\"}}, \"header\": {\"fill\": {\"color\": \"#C8D4E3\"}, \"line\": {\"color\": \"white\"}}, \"type\": \"table\"}]}, \"layout\": {\"annotationdefaults\": {\"arrowcolor\": \"#2a3f5f\", \"arrowhead\": 0, \"arrowwidth\": 1}, \"colorscale\": {\"diverging\": [[0, \"#8e0152\"], [0.1, \"#c51b7d\"], [0.2, \"#de77ae\"], [0.3, \"#f1b6da\"], [0.4, \"#fde0ef\"], [0.5, \"#f7f7f7\"], [0.6, \"#e6f5d0\"], [0.7, \"#b8e186\"], [0.8, \"#7fbc41\"], [0.9, \"#4d9221\"], [1, \"#276419\"]], \"sequential\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"sequentialminus\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]]}, \"colorway\": [\"#636efa\", \"#EF553B\", \"#00cc96\", \"#ab63fa\", \"#FFA15A\", \"#19d3f3\", \"#FF6692\", \"#B6E880\", \"#FF97FF\", \"#FECB52\"], \"font\": {\"color\": \"#2a3f5f\"}, \"geo\": {\"bgcolor\": \"white\", \"lakecolor\": \"white\", \"landcolor\": \"#E5ECF6\", \"showlakes\": true, \"showland\": true, \"subunitcolor\": \"white\"}, \"hoverlabel\": {\"align\": \"left\"}, \"hovermode\": \"closest\", \"mapbox\": {\"style\": \"light\"}, \"paper_bgcolor\": \"white\", \"plot_bgcolor\": \"#E5ECF6\", \"polar\": {\"angularaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"bgcolor\": \"#E5ECF6\", \"radialaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}}, \"scene\": {\"xaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}, \"yaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}, \"zaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}}, \"shapedefaults\": {\"line\": {\"color\": \"#2a3f5f\"}}, \"ternary\": {\"aaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"baxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"bgcolor\": \"#E5ECF6\", \"caxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}}, \"title\": {\"x\": 0.05}, \"xaxis\": {\"automargin\": true, \"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\", \"zerolinecolor\": \"white\", \"zerolinewidth\": 2}, \"yaxis\": {\"automargin\": true, \"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\", \"zerolinecolor\": \"white\", \"zerolinewidth\": 2}}}, \"title\": {\"text\": \"Optimization History Plot\"}, \"xaxis\": {\"title\": {\"text\": \"#Trials\"}}, \"yaxis\": {\"title\": {\"text\": \"Objective Value\"}}},\n",
" {\"responsive\": true}\n",
" ).then(function(){\n",
" \n",
"var gd = document.getElementById('e514d4b1-5ed2-4ed7-8963-15ce6cf4761f');\n",
"var x = new MutationObserver(function (mutations, observer) {{\n",
" var display = window.getComputedStyle(gd).display;\n",
" if (!display || display === 'none') {{\n",
" console.log([gd, 'removed!']);\n",
" Plotly.purge(gd);\n",
" observer.disconnect();\n",
" }}\n",
"}});\n",
"\n",
"// Listen for the removal of the full notebook cells\n",
"var notebookContainer = gd.closest('#notebook-container');\n",
"if (notebookContainer) {{\n",
" x.observe(notebookContainer, {childList: true});\n",
"}}\n",
"\n",
"// Listen for the clearing of the current output cell\n",
"var outputEl = gd.closest('.output');\n",
"if (outputEl) {{\n",
" x.observe(outputEl, {childList: true});\n",
"}}\n",
"\n",
" })\n",
" };\n",
" \n",
" </script>\n",
" </div>\n",
"</body>\n",
"</html>"
]
},
"metadata": {
"tags": []
}
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "D1DjzDOH5WZc",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 391
},
"outputId": "04346de3-b7eb-4542-be67-65dc6c93ea4f"
},
"source": [
"!apt-get install sqlite3 libsqlite3-dev"
],
"execution_count": 58,
"outputs": [
{
"output_type": "stream",
"text": [
"Reading package lists... Done\n",
"Building dependency tree \n",
"Reading state information... Done\n",
"Suggested packages:\n",
" sqlite3-doc\n",
"The following NEW packages will be installed:\n",
" libsqlite3-dev sqlite3\n",
"0 upgraded, 2 newly installed, 0 to remove and 29 not upgraded.\n",
"Need to get 1,385 kB of archives.\n",
"After this operation, 4,615 kB of additional disk space will be used.\n",
"Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libsqlite3-dev amd64 3.22.0-1ubuntu0.1 [633 kB]\n",
"Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 sqlite3 amd64 3.22.0-1ubuntu0.1 [753 kB]\n",
"Fetched 1,385 kB in 1s (1,014 kB/s)\n",
"Selecting previously unselected package libsqlite3-dev:amd64.\n",
"(Reading database ... 134923 files and directories currently installed.)\n",
"Preparing to unpack .../libsqlite3-dev_3.22.0-1ubuntu0.1_amd64.deb ...\n",
"Unpacking libsqlite3-dev:amd64 (3.22.0-1ubuntu0.1) ...\n",
"Selecting previously unselected package sqlite3.\n",
"Preparing to unpack .../sqlite3_3.22.0-1ubuntu0.1_amd64.deb ...\n",
"Unpacking sqlite3 (3.22.0-1ubuntu0.1) ...\n",
"Setting up libsqlite3-dev:amd64 (3.22.0-1ubuntu0.1) ...\n",
"Setting up sqlite3 (3.22.0-1ubuntu0.1) ...\n",
"Processing triggers for man-db (2.8.3-2ubuntu0.1) ...\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "9QVAm0FA5WW0",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 82
},
"outputId": "6dd6a6ce-4a97-4ca4-bc37-698af4ae1ead"
},
"source": [
"!sqlite3 optuna_study.db '.tables'"
],
"execution_count": 65,
"outputs": [
{
"output_type": "stream",
"text": [
"alembic_version trial_params trials \n",
"studies trial_system_attributes version_info \n",
"study_system_attributes trial_user_attributes \n",
"study_user_attributes trial_values \n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "Ntt62wby5Wb_",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 33
},
"outputId": "a4604c1c-80bc-49c7-c1d7-cfb626f584e1"
},
"source": [
"!sqlite3 optuna_study.db 'SELECT * FROM studies'"
],
"execution_count": 68,
"outputs": [
{
"output_type": "stream",
"text": [
"1|example-study|MINIMIZE\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "VB1AAJtM5We7",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 33
},
"outputId": "c2026850-e590-4a98-81e4-e636eb8e23fe"
},
"source": [
"!sqlite3 optuna_study.db 'SELECT COUNT(1) AS count FROM trials'"
],
"execution_count": 70,
"outputs": [
{
"output_type": "stream",
"text": [
"20\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "WU-FJp4nD4g2",
"colab_type": "text"
},
"source": [
"### 発展\n",
"\n",
"- https://blog.amedama.jp/entry/2018/12/06/015217\n",
"\n",
"mysqlも使えるらしい\n",
"\n",
"ちなみに再度optimizeしてもtrialにあるデータは消えることなく、蓄積されて行く"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "aYcQ012xmXU_",
"colab_type": "text"
},
"source": [
"## Reference\n",
"\n",
"- https://tech.515hikaru.net/2019-06-26-optuna-have-arg/\n",
"- https://blog.amedama.jp/entry/2018/12/06/015217\n",
"- https://www.slideshare.net/hoxo_m/ss-77421091\n",
"- https://www.slideshare.net/pfi/pydatatokyo-meetup-21-optuna"
]
}
]
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment