sile/report.md Secret

## report.md

      
    Raw
  

              report.md
            
          
    Benchmark Result Report

Kurobako Version: 0.2.6
Number of Solvers: 2
Number of Problems: 42
Metrics Precedence: best value -> AUC

Please refer to "A Strategy for Ranking Optimizers using Multiple Criteria" for the ranking strategy used in this report.
Table of Contents

Overall Results
Individual Results
Solvers
Problems
Studies

Overall Results


Solver
Borda
Firsts


Optuna
1
38


TPE
4
41


Individual Results
(1) Problem: sigopt/evalset/Xor(dim=9)


Ranking
Solver
Best (avg +- sd)
AUC (avg +- sd)
Elapsed (avg +- sd)


1
Optuna (study)
0.993390 +- 0.005154
79.062 +- 0.223
2.099 +- 0.272


1
TPE (study)
0.989686 +- 0.007573
78.846 +- 0.655
0.154 +- 0.024


(2) Problem: sigopt/evalset/CarromTable(dim=2)


Ranking
Solver
Best (avg +- sd)
AUC (avg +- sd)
Elapsed (avg +- sd)


1
TPE (study)
-22.872949 +- 1.005172
-1652.571 +- 190.729
0.052 +- 0.010


2
Optuna (study)
-22.194564 +- 3.416137
-1348.154 +- 352.782
0.514 +- 0.071


(3) Problem: sigopt/evalset/DeflectedCorrugatedSpring(dim=4)


Ranking
Solver
Best (avg +- sd)
AUC (avg +- sd)
Elapsed (avg +- sd)


1
Optuna (study)
-0.717780 +- 0.182056
-22.876 +- 16.804
0.974 +- 0.108


1
TPE (study)
-0.727596 +- 0.196942
-29.251 +- 19.134
0.082 +- 0.017


(4) Problem: sigopt/evalset/McCourt07(dim=6, res=12)


Ranking
Solver
Best (avg +- sd)
AUC (avg +- sd)
Elapsed (avg +- sd)


1
TPE (study)
-0.275000 +- 0.068550
-5.367 +- 6.368
0.108 +- 0.018


2
Optuna (study)
-0.227778 +- 0.064310
-0.981 +- 8.374
1.447 +- 0.150


(5) Problem: sigopt/evalset/Trid(dim=6)


Ranking
Solver
Best (avg +- sd)
AUC (avg +- sd)
Elapsed (avg +- sd)


1
Optuna (study)
-39.322946 +- 6.127839
-972.224 +- 852.826
1.390 +- 0.141


1
TPE (study)
-41.375532 +- 5.999828
-1136.604 +- 1061.727
0.114 +- 0.012


(6) Problem: sigopt/evalset/Hartmann3(dim=3)


Ranking
Solver
Best (avg +- sd)
AUC (avg +- sd)
Elapsed (avg +- sd)


1
Optuna (study)
-3.794874 +- 0.068767
-268.923 +- 16.981
0.714 +- 0.084


1
TPE (study)
-3.771731 +- 0.143582
-266.521 +- 16.429
0.067 +- 0.015


(7) Problem: sigopt/evalset/Rastrigin(dim=8)


Ranking
Solver
Best (avg +- sd)
AUC (avg +- sd)
Elapsed (avg +- sd)


1
Optuna (study)
51.567655 +- 9.603779
5092.247 +- 559.509
1.890 +- 0.163


1
TPE (study)
47.929794 +- 8.504059
5011.361 +- 569.197
0.131 +- 0.008


(8) Problem: sigopt/evalset/CarromTable(dim=2, int=[0])


Ranking
Solver
Best (avg +- sd)
AUC (avg +- sd)
Elapsed (avg +- sd)


1
TPE (study)
-19.382277 +- 2.139465
-1398.042 +- 240.191
0.048 +- 0.009


2
Optuna (study)
-18.558809 +- 2.898384
-1124.117 +- 355.133
0.546 +- 0.097


(9) Problem: sigopt/evalset/Branin02(dim=2, int=[0])


Ranking
Solver
Best (avg +- sd)
AUC (avg +- sd)
Elapsed (avg +- sd)


1
Optuna (study)
8.322043 +- 1.121514
906.763 +- 163.107
0.543 +- 0.086


1
TPE (study)
8.176882 +- 0.891681
898.070 +- 197.586
0.047 +- 0.008


(10) Problem: sigopt/evalset/Exponential(dim=6)


Ranking
Solver
Best (avg +- sd)
AUC (avg +- sd)
Elapsed (avg +- sd)


1
Optuna (study)
-0.988995 +- 0.009684
-74.520 +- 1.445
1.411 +- 0.160


2
TPE (study)
-0.985751 +- 0.009745
-73.532 +- 1.698
0.113 +- 0.014


(11) Problem: sigopt/evalset/McCourt27(dim=3)


Ranking
Solver
Best (avg +- sd)
AUC (avg +- sd)
Elapsed (avg +- sd)


1
Optuna (study)
-1.115265 +- 0.290447
-29.629 +- 27.596
0.736 +- 0.074


1
TPE (study)
-1.181162 +- 0.392760
-28.515 +- 20.999
0.065 +- 0.008


(12) Problem: sigopt/evalset/Easom(dim=4)


Ranking
Solver
Best (avg +- sd)
AUC (avg +- sd)
Elapsed (avg +- sd)


1
Optuna (study)
17.207166 +- 4.150281
1532.213 +- 153.864
0.965 +- 0.118


1
TPE (study)
16.640832 +- 4.458301
1513.019 +- 152.896
0.081 +- 0.015


(13) Problem: sigopt/evalset/Michalewicz(dim=4)


Ranking
Solver
Best (avg +- sd)
AUC (avg +- sd)
Elapsed (avg +- sd)


1
Optuna (study)
-2.456921 +- 0.409687
-154.184 +- 24.965
1.017 +- 0.103


1
TPE (study)
-2.429811 +- 0.344868
-151.232 +- 24.882
0.079 +- 0.012


(14) Problem: sigopt/evalset/Sphere(dim=7)


Ranking
Solver
Best (avg +- sd)
AUC (avg +- sd)
Elapsed (avg +- sd)


1
Optuna (study)
2.584901 +- 1.340599
653.296 +- 206.452
1.631 +- 0.149


1
TPE (study)
2.519518 +- 1.088015
712.224 +- 149.846
0.129 +- 0.017


(15) Problem: sigopt/evalset/Tripod(dim=2)


Ranking
Solver
Best (avg +- sd)
AUC (avg +- sd)
Elapsed (avg +- sd)


1
Optuna (study)
5.096237 +- 2.344442
1012.417 +- 305.085
0.494 +- 0.058


1
TPE (study)
4.703087 +- 1.725078
955.442 +- 243.257
0.050 +- 0.008


(16) Problem: sigopt/evalset/Shekel05(dim=4)


Ranking
Solver
Best (avg +- sd)
AUC (avg +- sd)
Elapsed (avg +- sd)


1
Optuna (study)
-1.589587 +- 0.735620
-69.877 +- 19.731
0.974 +- 0.084


1
TPE (study)
-2.369999 +- 1.654154
-102.969 +- 69.721
0.086 +- 0.016


(17) Problem: sigopt/evalset/Ackley(dim=3)


Ranking
Solver
Best (avg +- sd)
AUC (avg +- sd)
Elapsed (avg +- sd)


1
Optuna (study)
5.059500 +- 1.552897
710.526 +- 117.211
0.738 +- 0.102


1
TPE (study)
5.024110 +- 1.502778
744.546 +- 142.450
0.067 +- 0.021


(18) Problem: sigopt/evalset/LennardJones6(dim=6)


Ranking
Solver
Best (avg +- sd)
AUC (avg +- sd)
Elapsed (avg +- sd)


1
Optuna (study)
-0.978761 +- 0.045789
-50.342 +- 9.442
1.409 +- 0.147


1
TPE (study)
-0.964420 +- 0.065015
-51.227 +- 10.873
0.108 +- 0.014


(19) Problem: sigopt/evalset/McCourt22(dim=5)


Ranking
Solver
Best (avg +- sd)
AUC (avg +- sd)
Elapsed (avg +- sd)


1
Optuna (study)
-2.578230 +- 0.242046
-165.000 +- 23.266
1.210 +- 0.120


1
TPE (study)
-2.707553 +- 0.254801
-166.977 +- 15.933
0.099 +- 0.023


(20) Problem: sigopt/evalset/Ackley(dim=11, int=[0, 1, 2])


Ranking
Solver
Best (avg +- sd)
AUC (avg +- sd)
Elapsed (avg +- sd)


1
Optuna (study)
13.936461 +- 1.516013
1300.225 +- 72.129
2.650 +- 0.300


1
TPE (study)
14.140112 +- 1.929461
1290.064 +- 103.408
0.181 +- 0.019


(21) Problem: sigopt/evalset/Mishra06(dim=2)


Ranking
Solver
Best (avg +- sd)
AUC (avg +- sd)
Elapsed (avg +- sd)


1
Optuna (study)
-1.856294 +- 0.287699
-100.549 +- 21.900
0.502 +- 0.045


1
TPE (study)
-1.944638 +- 0.251441
-100.954 +- 23.131
0.049 +- 0.008


(22) Problem: sigopt/evalset/Weierstrass(dim=3)


Ranking
Solver
Best (avg +- sd)
AUC (avg +- sd)
Elapsed (avg +- sd)


1
Optuna (study)
13.312590 +- 0.432109
1113.177 +- 31.581
0.705 +- 0.088


1
TPE (study)
13.169722 +- 0.328912
1099.970 +- 24.815
0.066 +- 0.009


(23) Problem: sigopt/evalset/Ackley(dim=5)


Ranking
Solver
Best (avg +- sd)
AUC (avg +- sd)
Elapsed (avg +- sd)


1
Optuna (study)
8.113488 +- 1.606887
980.545 +- 158.876
1.223 +- 0.169


1
TPE (study)
8.236194 +- 1.903449
1001.801 +- 126.397
0.091 +- 0.017


(24) Problem: sigopt/evalset/Bukin06(dim=2, int=[0])


Ranking
Solver
Best (avg +- sd)
AUC (avg +- sd)
Elapsed (avg +- sd)


1
Optuna (study)
9.722000 +- 5.270652
1883.877 +- 525.400
0.551 +- 0.063


1
TPE (study)
8.347149 +- 3.900352
1778.613 +- 446.735
0.050 +- 0.009


(25) Problem: sigopt/evalset/Schwefel20(dim=2, int=[0])


Ranking
Solver
Best (avg +- sd)
AUC (avg +- sd)
Elapsed (avg +- sd)


1
Optuna (study)
2.805165 +- 1.868455
955.988 +- 269.244
0.614 +- 0.071


1
TPE (study)
3.147916 +- 1.906583
940.110 +- 311.947
0.049 +- 0.009


(26) Problem: sigopt/evalset/Deb02(dim=6)


Ranking
Solver
Best (avg +- sd)
AUC (avg +- sd)
Elapsed (avg +- sd)


1
Optuna (study)
-0.745466 +- 0.058212
-51.715 +- 3.635
1.418 +- 0.148


1
TPE (study)
-0.731667 +- 0.071744
-51.128 +- 4.275
0.107 +- 0.014


(27) Problem: sigopt/evalset/Ackley(dim=11)


Ranking
Solver
Best (avg +- sd)
AUC (avg +- sd)
Elapsed (avg +- sd)


1
Optuna (study)
14.277443 +- 1.794100
1315.809 +- 91.249
2.556 +- 0.265


1
TPE (study)
14.521596 +- 1.513862
1305.979 +- 98.917
0.179 +- 0.015


(28) Problem: sigopt/evalset/Sphere(dim=7, int=[0, 1, 2, 3, 4])


Ranking
Solver
Best (avg +- sd)
AUC (avg +- sd)
Elapsed (avg +- sd)


1
Optuna (study)
2.103150 +- 1.342227
698.066 +- 210.207
1.755 +- 0.152


1
TPE (study)
2.689469 +- 1.195182
795.176 +- 214.213
0.119 +- 0.012


(29) Problem: sigopt/evalset/StyblinskiTang(dim=5)


Ranking
Solver
Best (avg +- sd)
AUC (avg +- sd)
Elapsed (avg +- sd)


1
Optuna (study)
-157.061698 +- 13.683493
-10443.805 +- 860.943
1.159 +- 0.112


1
TPE (study)
-159.368114 +- 13.823497
-10983.855 +- 1194.895
0.091 +- 0.016


(30) Problem: sigopt/evalset/Ned01(dim=2)


Ranking
Solver
Best (avg +- sd)
AUC (avg +- sd)
Elapsed (avg +- sd)


1
Optuna (study)
0.076205 +- 0.070345
14.860 +- 5.824
0.501 +- 0.046


1
TPE (study)
0.038726 +- 0.088347
13.520 +- 7.614
0.048 +- 0.008


(31) Problem: sigopt/evalset/YaoLiu(dim=5)


Ranking
Solver
Best (avg +- sd)
AUC (avg +- sd)
Elapsed (avg +- sd)


1
Optuna (study)
20.587560 +- 5.389727
2416.369 +- 411.935
1.181 +- 0.137


1
TPE (study)
22.129447 +- 6.635147
2599.144 +- 393.332
0.099 +- 0.021


(32) Problem: sigopt/evalset/Easom(dim=5)


Ranking
Solver
Best (avg +- sd)
AUC (avg +- sd)
Elapsed (avg +- sd)


1
Optuna (study)
19.598449 +- 2.335792
1618.950 +- 77.894
1.178 +- 0.134


1
TPE (study)
19.675417 +- 2.527447
1599.711 +- 140.754
0.099 +- 0.019


(33) Problem: sigopt/evalset/Ackley(dim=3, res=1)


Ranking
Solver
Best (avg +- sd)
AUC (avg +- sd)
Elapsed (avg +- sd)


1
Optuna (study)
4.300000 +- 1.417745
658.700 +- 91.811
0.742 +- 0.118


1
TPE (study)
3.900000 +- 1.011599
595.333 +- 128.060
0.065 +- 0.009


(34) Problem: sigopt/evalset/McCourt19(dim=2)


Ranking
Solver
Best (avg +- sd)
AUC (avg +- sd)
Elapsed (avg +- sd)


1
Optuna (study)
-8.461423 +- 0.131150
-594.724 +- 32.279
0.531 +- 0.071


1
TPE (study)
-8.506519 +- 0.097196
-584.351 +- 34.321
0.050 +- 0.009


(35) Problem: sigopt/evalset/Plateau(dim=2)


Ranking
Solver
Best (avg +- sd)
AUC (avg +- sd)
Elapsed (avg +- sd)


1
Optuna (study)
30.000000 +- 0.000000
2386.167 +- 8.513
0.506 +- 0.047


1
TPE (study)
30.000000 +- 0.000000
2384.400 +- 8.333
0.049 +- 0.008


(36) Problem: sigopt/evalset/McCourt01(dim=7, res=10)


Ranking
Solver
Best (avg +- sd)
AUC (avg +- sd)
Elapsed (avg +- sd)


1
Optuna (study)
0.026667 +- 0.092856
21.297 +- 10.298
1.623 +- 0.157


1
TPE (study)
0.013333 +- 0.123108
21.433 +- 8.740
0.129 +- 0.026


(37) Problem: sigopt/evalset/Shekel07(dim=4)


Ranking
Solver
Best (avg +- sd)
AUC (avg +- sd)
Elapsed (avg +- sd)


1
Optuna (study)
-2.223391 +- 0.891697
-91.441 +- 20.872
0.978 +- 0.083


1
TPE (study)
-2.352949 +- 1.194168
-95.508 +- 33.456
0.084 +- 0.015


(38) Problem: sigopt/evalset/McCourt06(dim=5, res=12)


Ranking
Solver
Best (avg +- sd)
AUC (avg +- sd)
Elapsed (avg +- sd)


1
Optuna (study)
2.847222 +- 0.048512
234.444 +- 4.336
1.185 +- 0.126


1
TPE (study)
2.844444 +- 0.041574
234.839 +- 5.059
0.097 +- 0.014


(39) Problem: sigopt/evalset/Sargan(dim=2, int=[0])


Ranking
Solver
Best (avg +- sd)
AUC (avg +- sd)
Elapsed (avg +- sd)


1
Optuna (study)
0.000881 +- 0.002711
73.963 +- 37.908
0.616 +- 0.071


1
TPE (study)
0.056329 +- 0.301516
79.598 +- 53.628
0.048 +- 0.013


(40) Problem: sigopt/evalset/McCourt02(dim=7)


Ranking
Solver
Best (avg +- sd)
AUC (avg +- sd)
Elapsed (avg +- sd)


1
Optuna (study)
-2.686687 +- 0.026472
-203.463 +- 4.287
1.677 +- 0.165


1
TPE (study)
-2.691230 +- 0.027726
-204.052 +- 3.832
0.125 +- 0.018


(41) Problem: sigopt/evalset/Rastrigin(dim=8, res=0.1)


Ranking
Solver
Best (avg +- sd)
AUC (avg +- sd)
Elapsed (avg +- sd)


1
TPE (study)
40.000000 +- 9.309493
4412.000 +- 698.185
0.141 +- 0.017


2
Optuna (study)
46.000000 +- 10.198039
4801.333 +- 713.787
1.896 +- 0.163


(42) Problem: sigopt/evalset/Schwefel20(dim=2)


Ranking
Solver
Best (avg +- sd)
AUC (avg +- sd)
Elapsed (avg +- sd)


1
Optuna (study)
3.066478 +- 1.381306
952.353 +- 255.962
0.583 +- 0.072


1
TPE (study)
2.518236 +- 1.449889
1103.380 +- 314.851
0.045 +- 0.005


Solvers
ID: 487e04184b5eeb61450126dfd6552b09ffc08402cf3183770d239fb094201cf2
recipe:
{
  "optuna": {}
}
specification:
{
  "name": "Optuna",
  "attrs": {
    "github": "https://github.com/optuna/optuna",
    "paper": "Akiba, Takuya, et al. \"Optuna: A next-generation hyperparameter optimization framework.\" Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. ACM, 2019.",
    "version": "optuna=2.1.0, kurobako-py=0.1.8"
  },
  "capabilities": [
    "UNIFORM_CONTINUOUS",
    "UNIFORM_DISCRETE",
    "LOG_UNIFORM_CONTINUOUS",
    "CATEGORICAL",
    "CONDITIONAL",
    "CONCURRENT"
  ]
}
ID: cd5a17022fd1fec0554d9192671f1901113ace2492e20d59ac0417f3666c4618
recipe:
{
  "command": {
    "path": "cargo",
    "args": [
      "run",
      "--release",
      "--example",
      "tpe-solver"
    ]
  }
}
specification:
{
  "name": "TPE",
  "attrs": {},
  "capabilities": [
    "UNIFORM_CONTINUOUS",
    "UNIFORM_DISCRETE",
    "LOG_UNIFORM_CONTINUOUS",
    "CATEGORICAL",
    "CONCURRENT"
  ]
}
Problems
ID: 762d606fd111289ffacaa9410ef618348513d70d9fe903e5d71a60978c28d213
recipe:
{
  "sigopt": {
    "name": "ACKLEY",
    "dim": 11
  }
}
specification:
{
  "name": "sigopt/evalset/Ackley(dim=11)",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "CONTINUOUS",
        "low": -10.0,
        "high": 30.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": -10.0,
        "high": 30.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p2",
      "range": {
        "type": "CONTINUOUS",
        "low": -10.0,
        "high": 30.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p3",
      "range": {
        "type": "CONTINUOUS",
        "low": -10.0,
        "high": 30.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p4",
      "range": {
        "type": "CONTINUOUS",
        "low": -10.0,
        "high": 30.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p5",
      "range": {
        "type": "CONTINUOUS",
        "low": -10.0,
        "high": 30.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p6",
      "range": {
        "type": "CONTINUOUS",
        "low": -10.0,
        "high": 30.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p7",
      "range": {
        "type": "CONTINUOUS",
        "low": -10.0,
        "high": 30.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p8",
      "range": {
        "type": "CONTINUOUS",
        "low": -10.0,
        "high": 30.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p9",
      "range": {
        "type": "CONTINUOUS",
        "low": -10.0,
        "high": 30.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p10",
      "range": {
        "type": "CONTINUOUS",
        "low": -10.0,
        "high": 30.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}
ID: 5711ea99766d928a6218dbeaadd93bd28e82a9c3b19b0eac5b64e01b7f252fac
recipe:
{
  "sigopt": {
    "name": "ACKLEY",
    "dim": 11,
    "int": [
      0,
      1,
      2
    ]
  }
}
specification:
{
  "name": "sigopt/evalset/Ackley(dim=11, int=[0, 1, 2])",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "DISCRETE",
        "low": -10,
        "high": 30
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "DISCRETE",
        "low": -10,
        "high": 30
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p2",
      "range": {
        "type": "DISCRETE",
        "low": -10,
        "high": 30
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p3",
      "range": {
        "type": "CONTINUOUS",
        "low": -10.0,
        "high": 30.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p4",
      "range": {
        "type": "CONTINUOUS",
        "low": -10.0,
        "high": 30.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p5",
      "range": {
        "type": "CONTINUOUS",
        "low": -10.0,
        "high": 30.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p6",
      "range": {
        "type": "CONTINUOUS",
        "low": -10.0,
        "high": 30.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p7",
      "range": {
        "type": "CONTINUOUS",
        "low": -10.0,
        "high": 30.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p8",
      "range": {
        "type": "CONTINUOUS",
        "low": -10.0,
        "high": 30.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p9",
      "range": {
        "type": "CONTINUOUS",
        "low": -10.0,
        "high": 30.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p10",
      "range": {
        "type": "CONTINUOUS",
        "low": -10.0,
        "high": 30.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}
ID: 48bc49268474110e0c2a181bb78990c6dbd331d61fb6e082d3bd095d4de815e2
recipe:
{
  "sigopt": {
    "name": "ACKLEY",
    "dim": 3
  }
}
specification:
{
  "name": "sigopt/evalset/Ackley(dim=3)",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "CONTINUOUS",
        "low": -10.0,
        "high": 30.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": -10.0,
        "high": 30.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p2",
      "range": {
        "type": "CONTINUOUS",
        "low": -10.0,
        "high": 30.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}
ID: bc80da85f8186f371a2a127d82ff7136379712e311358c60678085830effc226
recipe:
{
  "sigopt": {
    "name": "ACKLEY",
    "dim": 3,
    "res": 1.0
  }
}
specification:
{
  "name": "sigopt/evalset/Ackley(dim=3, res=1)",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "CONTINUOUS",
        "low": -10.0,
        "high": 30.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": -10.0,
        "high": 30.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p2",
      "range": {
        "type": "CONTINUOUS",
        "low": -10.0,
        "high": 30.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}
ID: 66cda488c56779bfd9d3bb1203160dce3d784c9bc293daabacf0509790e43aed
recipe:
{
  "sigopt": {
    "name": "ACKLEY",
    "dim": 5
  }
}
specification:
{
  "name": "sigopt/evalset/Ackley(dim=5)",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "CONTINUOUS",
        "low": -10.0,
        "high": 30.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": -10.0,
        "high": 30.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p2",
      "range": {
        "type": "CONTINUOUS",
        "low": -10.0,
        "high": 30.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p3",
      "range": {
        "type": "CONTINUOUS",
        "low": -10.0,
        "high": 30.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p4",
      "range": {
        "type": "CONTINUOUS",
        "low": -10.0,
        "high": 30.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}
ID: 2318f2882267c2f27d174954c9976ade13be9e978006ce076c6dac22fa7c8463
recipe:
{
  "sigopt": {
    "name": "BRANIN02",
    "dim": 2,
    "int": [
      0
    ]
  }
}
specification:
{
  "name": "sigopt/evalset/Branin02(dim=2, int=[0])",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "DISCRETE",
        "low": -5,
        "high": 15
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": -5.0,
        "high": 15.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}
ID: 6ab293911d0424ebb16aa03a9185f16e62c9023da1c0ccbf4353b0433fa42d10
recipe:
{
  "sigopt": {
    "name": "BUKIN06",
    "dim": 2,
    "int": [
      0
    ]
  }
}
specification:
{
  "name": "sigopt/evalset/Bukin06(dim=2, int=[0])",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "DISCRETE",
        "low": -15,
        "high": -5
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": -3.0,
        "high": 3.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}
ID: 0dcb70363493ed100a24a44a12d3d347aa1a0903a18e1ed8abe003df7a74355b
recipe:
{
  "sigopt": {
    "name": "CARROM_TABLE",
    "dim": 2
  }
}
specification:
{
  "name": "sigopt/evalset/CarromTable(dim=2)",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "CONTINUOUS",
        "low": -10.0,
        "high": 10.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": -10.0,
        "high": 10.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}
ID: 20c7917b2603e9d2e6928ad861cc2b8edf702d96f5759ef27ca85c11335edf16
recipe:
{
  "sigopt": {
    "name": "CARROM_TABLE",
    "dim": 2,
    "int": [
      0
    ]
  }
}
specification:
{
  "name": "sigopt/evalset/CarromTable(dim=2, int=[0])",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "DISCRETE",
        "low": -10,
        "high": 10
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": -10.0,
        "high": 10.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}
ID: 6fd3e8ef60eec3daf630366f99491d35b6f05a4163ebb6d0b4db74926bc42ce2
recipe:
{
  "sigopt": {
    "name": "DEB02",
    "dim": 6
  }
}
specification:
{
  "name": "sigopt/evalset/Deb02(dim=6)",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p2",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p3",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p4",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p5",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}
ID: 1210ccbbf66a139a961e1392f8cc35351922a8f089a440ccd54af4577e5f368b
recipe:
{
  "sigopt": {
    "name": "DEFLECTED_CORRUGATED_SPRING",
    "dim": 4
  }
}
specification:
{
  "name": "sigopt/evalset/DeflectedCorrugatedSpring(dim=4)",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 7.5
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 7.5
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p2",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 7.5
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p3",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 7.5
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}
ID: 30f46f4f59c9a6b9ba33e0dadd611080e606ff70a844b2c68bfeb66a7a07e405
recipe:
{
  "sigopt": {
    "name": "EASOM",
    "dim": 4
  }
}
specification:
{
  "name": "sigopt/evalset/Easom(dim=4)",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "CONTINUOUS",
        "low": -100.0,
        "high": 20.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": -100.0,
        "high": 20.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p2",
      "range": {
        "type": "CONTINUOUS",
        "low": -100.0,
        "high": 20.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p3",
      "range": {
        "type": "CONTINUOUS",
        "low": -100.0,
        "high": 20.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}
ID: a39b57cec0151c892256bc8386f43dc15e83c2a985c9761fbf46f0abc541307e
recipe:
{
  "sigopt": {
    "name": "EASOM",
    "dim": 5
  }
}
specification:
{
  "name": "sigopt/evalset/Easom(dim=5)",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "CONTINUOUS",
        "low": -100.0,
        "high": 20.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": -100.0,
        "high": 20.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p2",
      "range": {
        "type": "CONTINUOUS",
        "low": -100.0,
        "high": 20.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p3",
      "range": {
        "type": "CONTINUOUS",
        "low": -100.0,
        "high": 20.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p4",
      "range": {
        "type": "CONTINUOUS",
        "low": -100.0,
        "high": 20.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}
ID: 29849239fd4b7b5d4543cec4bc2cbb857fa41fa8b4147a765850b43409317dc1
recipe:
{
  "sigopt": {
    "name": "EXPONENTIAL",
    "dim": 6
  }
}
specification:
{
  "name": "sigopt/evalset/Exponential(dim=6)",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "CONTINUOUS",
        "low": -0.7,
        "high": 0.2
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": -0.7,
        "high": 0.2
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p2",
      "range": {
        "type": "CONTINUOUS",
        "low": -0.7,
        "high": 0.2
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p3",
      "range": {
        "type": "CONTINUOUS",
        "low": -0.7,
        "high": 0.2
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p4",
      "range": {
        "type": "CONTINUOUS",
        "low": -0.7,
        "high": 0.2
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p5",
      "range": {
        "type": "CONTINUOUS",
        "low": -0.7,
        "high": 0.2
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}
ID: 1e08466d41a46890e37413c5f989e30c6f2155b3e3e99721997e6d9ed29148d5
recipe:
{
  "sigopt": {
    "name": "HARTMANN3",
    "dim": 3
  }
}
specification:
{
  "name": "sigopt/evalset/Hartmann3(dim=3)",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p2",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}
ID: 4cc33f4e2ff5625738795af29a4d94fa607d5f5949ece8ad44b73b1e63b63339
recipe:
{
  "sigopt": {
    "name": "LENNARD_JONES6",
    "dim": 6
  }
}
specification:
{
  "name": "sigopt/evalset/LennardJones6(dim=6)",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "CONTINUOUS",
        "low": -3.0,
        "high": 3.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": -3.0,
        "high": 3.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p2",
      "range": {
        "type": "CONTINUOUS",
        "low": -3.0,
        "high": 3.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p3",
      "range": {
        "type": "CONTINUOUS",
        "low": -3.0,
        "high": 3.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p4",
      "range": {
        "type": "CONTINUOUS",
        "low": -3.0,
        "high": 3.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p5",
      "range": {
        "type": "CONTINUOUS",
        "low": -3.0,
        "high": 3.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}
ID: ccf73084ef5dbe3746e1d5252d6bfaa643bc1ece3358a4c77decf923d8e0facb
recipe:
{
  "sigopt": {
    "name": "MC_COURT01",
    "dim": 7,
    "res": 10.0
  }
}
specification:
{
  "name": "sigopt/evalset/McCourt01(dim=7, res=10)",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p2",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p3",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p4",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p5",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p6",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}
ID: e7895b7bb5dac94288be96e61ad121d23e185003eaefd87ba5e1023a6132104e
recipe:
{
  "sigopt": {
    "name": "MC_COURT02",
    "dim": 7
  }
}
specification:
{
  "name": "sigopt/evalset/McCourt02(dim=7)",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p2",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p3",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p4",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p5",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p6",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}
ID: cff13c7e9a01ac1b8cce1a957f7caa7cc6d438ced9a8c25b658215794acc6d92
recipe:
{
  "sigopt": {
    "name": "MC_COURT06",
    "dim": 5,
    "res": 12.0
  }
}
specification:
{
  "name": "sigopt/evalset/McCourt06(dim=5, res=12)",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p2",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p3",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p4",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}
ID: 1464bdea4ea2f345666eae6ecdda641b6a196281174f5ba8007a8e8a2bec532e
recipe:
{
  "sigopt": {
    "name": "MC_COURT07",
    "dim": 6,
    "res": 12.0
  }
}
specification:
{
  "name": "sigopt/evalset/McCourt07(dim=6, res=12)",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p2",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p3",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p4",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p5",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}
ID: c402d6ef019d823cfda5127b120f2bd5eb3e021edc112118ae0c9f07e72de789
recipe:
{
  "sigopt": {
    "name": "MC_COURT19",
    "dim": 2
  }
}
specification:
{
  "name": "sigopt/evalset/McCourt19(dim=2)",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}
ID: 541f33a97b45aa4a10f3197b56751eb385ded69f15d40453f8b7dee5ba7882a8
recipe:
{
  "sigopt": {
    "name": "MC_COURT22",
    "dim": 5
  }
}
specification:
{
  "name": "sigopt/evalset/McCourt22(dim=5)",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p2",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p3",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p4",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}
ID: 2fef32d714633c85c412e13e086a1ec8df60117de097a4209b590376b1787729
recipe:
{
  "sigopt": {
    "name": "MC_COURT27",
    "dim": 3
  }
}
specification:
{
  "name": "sigopt/evalset/McCourt27(dim=3)",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p2",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}
ID: 3902a3f083b3cb4c95db2c0166a0a337063e4555d7a2f9d2d31c367d32758f4e
recipe:
{
  "sigopt": {
    "name": "MICHALEWICZ",
    "dim": 4
  }
}
specification:
{
  "name": "sigopt/evalset/Michalewicz(dim=4)",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 3.141592653589793
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 3.141592653589793
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p2",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 3.141592653589793
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p3",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 3.141592653589793
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}
ID: 59c429b4a300b87b138a006f1128ff1d16e2c50a9c909a88ef514a10e3c80148
recipe:
{
  "sigopt": {
    "name": "MISHRA06",
    "dim": 2
  }
}
specification:
{
  "name": "sigopt/evalset/Mishra06(dim=2)",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "CONTINUOUS",
        "low": -10.0,
        "high": 10.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": -10.0,
        "high": 10.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}
ID: 862877d4e1f14c9bea4f924a5c6366938cc8e2aadd70df465af0586d67a99f8b
recipe:
{
  "sigopt": {
    "name": "NED01",
    "dim": 2
  }
}
specification:
{
  "name": "sigopt/evalset/Ned01(dim=2)",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "CONTINUOUS",
        "low": -10.0,
        "high": 10.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": -10.0,
        "high": 10.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}
ID: c6e62d2a584679ad6fcdec6abc3bab8ab394e333a01e662053a28c2837ef57a9
recipe:
{
  "sigopt": {
    "name": "PLATEAU",
    "dim": 2
  }
}
specification:
{
  "name": "sigopt/evalset/Plateau(dim=2)",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "CONTINUOUS",
        "low": -2.34,
        "high": 5.12
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": -2.34,
        "high": 5.12
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}
ID: 1eb7796f1b2d5f244bfd8eefd6b3d97091d460e9ab56c61d71a29ad15e7fa0da
recipe:
{
  "sigopt": {
    "name": "RASTRIGIN",
    "dim": 8
  }
}
specification:
{
  "name": "sigopt/evalset/Rastrigin(dim=8)",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "CONTINUOUS",
        "low": -5.0,
        "high": 2.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": -5.0,
        "high": 2.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p2",
      "range": {
        "type": "CONTINUOUS",
        "low": -2.0,
        "high": 5.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p3",
      "range": {
        "type": "CONTINUOUS",
        "low": -2.0,
        "high": 5.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p4",
      "range": {
        "type": "CONTINUOUS",
        "low": -5.0,
        "high": 2.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p5",
      "range": {
        "type": "CONTINUOUS",
        "low": -5.0,
        "high": 2.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p6",
      "range": {
        "type": "CONTINUOUS",
        "low": -2.0,
        "high": 5.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p7",
      "range": {
        "type": "CONTINUOUS",
        "low": -2.0,
        "high": 5.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}
ID: ec359d6b8722b456204bb2f41274079d5c5c0e08a6b3db98a762d3501a380e96
recipe:
{
  "sigopt": {
    "name": "RASTRIGIN",
    "dim": 8,
    "res": 0.1
  }
}
specification:
{
  "name": "sigopt/evalset/Rastrigin(dim=8, res=0.1)",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "CONTINUOUS",
        "low": -5.0,
        "high": 2.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": -5.0,
        "high": 2.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p2",
      "range": {
        "type": "CONTINUOUS",
        "low": -2.0,
        "high": 5.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p3",
      "range": {
        "type": "CONTINUOUS",
        "low": -2.0,
        "high": 5.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p4",
      "range": {
        "type": "CONTINUOUS",
        "low": -5.0,
        "high": 2.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p5",
      "range": {
        "type": "CONTINUOUS",
        "low": -5.0,
        "high": 2.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p6",
      "range": {
        "type": "CONTINUOUS",
        "low": -2.0,
        "high": 5.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p7",
      "range": {
        "type": "CONTINUOUS",
        "low": -2.0,
        "high": 5.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}
ID: e014fb18e27703930d93dd7e7ee8178ce5ffad97027d013fac90c263049c70d7
recipe:
{
  "sigopt": {
    "name": "SARGAN",
    "dim": 2,
    "int": [
      0
    ]
  }
}
specification:
{
  "name": "sigopt/evalset/Sargan(dim=2, int=[0])",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "DISCRETE",
        "low": -2,
        "high": 4
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": -2.0,
        "high": 4.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}
ID: fcbd60ae002525d084f788a792305d491f32a61ac99ac711a9cfe9d1860f8d34
recipe:
{
  "sigopt": {
    "name": "SCHWEFEL20",
    "dim": 2
  }
}
specification:
{
  "name": "sigopt/evalset/Schwefel20(dim=2)",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "CONTINUOUS",
        "low": -60.0,
        "high": 100.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": -60.0,
        "high": 100.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}
ID: 6de30105fa47b8ecbda30c3676ee059bcfcc221b82a83736bb8b479f4559da6a
recipe:
{
  "sigopt": {
    "name": "SCHWEFEL20",
    "dim": 2,
    "int": [
      0
    ]
  }
}
specification:
{
  "name": "sigopt/evalset/Schwefel20(dim=2, int=[0])",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "DISCRETE",
        "low": -60,
        "high": 100
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": -60.0,
        "high": 100.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}
ID: 47fe6e07816771591c8ea2640dea44bce93ff9171060ac4c3ece8b1c71a32aa2
recipe:
{
  "sigopt": {
    "name": "SHEKEL05",
    "dim": 4
  }
}
specification:
{
  "name": "sigopt/evalset/Shekel05(dim=4)",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "CONTINUOUS",
        "low": -0.0,
        "high": 10.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": -0.0,
        "high": 10.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p2",
      "range": {
        "type": "CONTINUOUS",
        "low": -0.0,
        "high": 10.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p3",
      "range": {
        "type": "CONTINUOUS",
        "low": -0.0,
        "high": 10.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}
ID: cd5ee5f8e7c0328b28837d220987f3ed318887b201e2dccd24559937f37c215a
recipe:
{
  "sigopt": {
    "name": "SHEKEL07",
    "dim": 4
  }
}
specification:
{
  "name": "sigopt/evalset/Shekel07(dim=4)",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "CONTINUOUS",
        "low": -0.0,
        "high": 10.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": -0.0,
        "high": 10.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p2",
      "range": {
        "type": "CONTINUOUS",
        "low": -0.0,
        "high": 10.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p3",
      "range": {
        "type": "CONTINUOUS",
        "low": -0.0,
        "high": 10.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}
ID: 429f7ada6a8f5648aee7930fdf1f3af0807ba052ca921cac32dc102c3d6d32f2
recipe:
{
  "sigopt": {
    "name": "SPHERE",
    "dim": 7
  }
}
specification:
{
  "name": "sigopt/evalset/Sphere(dim=7)",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "CONTINUOUS",
        "low": -5.12,
        "high": 2.12
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": -5.12,
        "high": 2.12
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p2",
      "range": {
        "type": "CONTINUOUS",
        "low": -5.12,
        "high": 2.12
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p3",
      "range": {
        "type": "CONTINUOUS",
        "low": -5.12,
        "high": 2.12
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p4",
      "range": {
        "type": "CONTINUOUS",
        "low": -5.12,
        "high": 2.12
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p5",
      "range": {
        "type": "CONTINUOUS",
        "low": -5.12,
        "high": 2.12
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p6",
      "range": {
        "type": "CONTINUOUS",
        "low": -5.12,
        "high": 2.12
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}
ID: 7a9b62682ddee5ac08c93669255954f154b4bae095dc877c2b478b7d20f9b45d
recipe:
{
  "sigopt": {
    "name": "SPHERE",
    "dim": 7,
    "int": [
      0,
      1,
      2,
      3,
      4
    ]
  }
}
specification:
{
  "name": "sigopt/evalset/Sphere(dim=7, int=[0, 1, 2, 3, 4])",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "DISCRETE",
        "low": -5,
        "high": 2
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "DISCRETE",
        "low": -5,
        "high": 2
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p2",
      "range": {
        "type": "DISCRETE",
        "low": -5,
        "high": 2
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p3",
      "range": {
        "type": "DISCRETE",
        "low": -5,
        "high": 2
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p4",
      "range": {
        "type": "DISCRETE",
        "low": -5,
        "high": 2
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p5",
      "range": {
        "type": "CONTINUOUS",
        "low": -5.12,
        "high": 2.12
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p6",
      "range": {
        "type": "CONTINUOUS",
        "low": -5.12,
        "high": 2.12
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}
ID: 847114b901d551887b3bb6e0686324852e2cc8c4fe1ac75e8ff018da13b1023e
recipe:
{
  "sigopt": {
    "name": "STYBLINSKI_TANG",
    "dim": 5
  }
}
specification:
{
  "name": "sigopt/evalset/StyblinskiTang(dim=5)",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "CONTINUOUS",
        "low": -5.0,
        "high": 5.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": -5.0,
        "high": 5.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p2",
      "range": {
        "type": "CONTINUOUS",
        "low": -5.0,
        "high": 5.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p3",
      "range": {
        "type": "CONTINUOUS",
        "low": -5.0,
        "high": 5.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p4",
      "range": {
        "type": "CONTINUOUS",
        "low": -5.0,
        "high": 5.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}
ID: 15ca748208253e4a2e35adf06a8f7feb49188210b02e1cd08702d54c9267a4f6
recipe:
{
  "sigopt": {
    "name": "TRID",
    "dim": 6
  }
}
specification:
{
  "name": "sigopt/evalset/Trid(dim=6)",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 20.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 20.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p2",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 20.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p3",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 20.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p4",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 20.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p5",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 20.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}
ID: 458dd11a95505ddb3a8d96d492dbc35b0de444d84fe0bf3fef739d02baeaf425
recipe:
{
  "sigopt": {
    "name": "TRIPOD",
    "dim": 2
  }
}
specification:
{
  "name": "sigopt/evalset/Tripod(dim=2)",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "CONTINUOUS",
        "low": -100.0,
        "high": 100.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": -100.0,
        "high": 100.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}
ID: 6396ba82fa5de7995ce66e0eed9448e5882508432c8b324d52c5f11a79f16a78
recipe:
{
  "sigopt": {
    "name": "WEIERSTRASS",
    "dim": 3
  }
}
specification:
{
  "name": "sigopt/evalset/Weierstrass(dim=3)",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "CONTINUOUS",
        "low": -0.5,
        "high": 0.2
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": -0.5,
        "high": 0.2
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p2",
      "range": {
        "type": "CONTINUOUS",
        "low": -0.5,
        "high": 0.2
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}
ID: 099c4ef27c777d65a669d88cd599c9e679f956bd04859ed85d2c9403ec82200c
recipe:
{
  "sigopt": {
    "name": "XOR",
    "dim": 9
  }
}
specification:
{
  "name": "sigopt/evalset/Xor(dim=9)",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "CONTINUOUS",
        "low": -1.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": -1.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p2",
      "range": {
        "type": "CONTINUOUS",
        "low": -1.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p3",
      "range": {
        "type": "CONTINUOUS",
        "low": -1.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p4",
      "range": {
        "type": "CONTINUOUS",
        "low": -1.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p5",
      "range": {
        "type": "CONTINUOUS",
        "low": -1.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p6",
      "range": {
        "type": "CONTINUOUS",
        "low": -1.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p7",
      "range": {
        "type": "CONTINUOUS",
        "low": -1.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p8",
      "range": {
        "type": "CONTINUOUS",
        "low": -1.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}
ID: a0d7aa2cea146419075791ef31a3e4b50dcefbca3e9b2c52d766f0b000607e67
recipe:
{
  "sigopt": {
    "name": "YAO_LIU",
    "dim": 5
  }
}
specification:
{
  "name": "sigopt/evalset/YaoLiu(dim=5)",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "CONTINUOUS",
        "low": -5.12,
        "high": 2.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": -5.12,
        "high": 2.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p2",
      "range": {
        "type": "CONTINUOUS",
        "low": -5.12,
        "high": 2.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p3",
      "range": {
        "type": "CONTINUOUS",
        "low": -5.12,
        "high": 2.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p4",
      "range": {
        "type": "CONTINUOUS",
        "low": -5.12,
        "high": 2.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}
Studies
ID: fbc2ddeb5d704422b08bce6880cb7846605a47643fb54981497602f9ac841d8a

problem: sigopt/evalset/Ackley(dim=11)
solver: Optuna
budget: 80
repeats: 30
concurrency: 1

ID: ba15b8dac88bcacef7d931cbbc0e78c34cfbbd1dee733ab492eb02164d602544

problem: sigopt/evalset/Ackley(dim=11)
solver: TPE
budget: 80
repeats: 30
concurrency: 1

ID: 266df7c20ab06d059b7148e715d6114b3b999a99a72b20cf50743d6a050f284a

problem: sigopt/evalset/Ackley(dim=11, int=[0, 1, 2])
solver: Optuna
budget: 80
repeats: 30
concurrency: 1

ID: c53a30cf85d11e6f300fd7a542807a272713a199726baacf2dced62d9864c4e7

problem: sigopt/evalset/Ackley(dim=11, int=[0, 1, 2])
solver: TPE
budget: 80
repeats: 30
concurrency: 1

ID: 62f6dd2b2d5f7230ad8e9414f3cb5c6b9b1e95e6eb0fadc08ce7bf5d597033c2

problem: sigopt/evalset/Ackley(dim=3)
solver: Optuna
budget: 80
repeats: 30
concurrency: 1

ID: 718b5e99f75ee31c2e31da79af467600fb753bb14bbac429ac41fab12f3ebaaa

problem: sigopt/evalset/Ackley(dim=3)
solver: TPE
budget: 80
repeats: 30
concurrency: 1

ID: 6bba607f548c2dba45f37c50ee76c66361ccc86e294f8efcaf1098b401dc96b5

problem: sigopt/evalset/Ackley(dim=3, res=1)
solver: Optuna
budget: 80
repeats: 30
concurrency: 1

ID: ce060de815ae7f252a2ad4fb0b897acc32c89c3ca71b241d869f50851ed8acf1

problem: sigopt/evalset/Ackley(dim=3, res=1)
solver: TPE
budget: 80
repeats: 30
concurrency: 1

ID: 2309623e53c7582bf49fcbba8ce8ccf8043730725d8537a8717ccb1ae9b7f061

problem: sigopt/evalset/Ackley(dim=5)
solver: Optuna
budget: 80
repeats: 30
concurrency: 1

ID: ca345964f1864c0c20a02638655c9eaf3ff65f5c490c81ed23cdd208ee45d2b8

problem: sigopt/evalset/Ackley(dim=5)
solver: TPE
budget: 80
repeats: 30
concurrency: 1

ID: c3e387959b4b0679916f6468ec6f8ff15692f6240a75d5735ce6c762f0452e96

problem: sigopt/evalset/Branin02(dim=2, int=[0])
solver: Optuna
budget: 80
repeats: 30
concurrency: 1

ID: 864721f31b1c405aec41d8b96455aa16bf6c17f875d7ded242798d3b47a20f66

problem: sigopt/evalset/Branin02(dim=2, int=[0])
solver: TPE
budget: 80
repeats: 30
concurrency: 1

ID: f95ae1a5b1536be196364428c452ca6deb4eb0f94bb7f9c591df983f922e8c39

problem: sigopt/evalset/Bukin06(dim=2, int=[0])
solver: Optuna
budget: 80
repeats: 30
concurrency: 1

ID: 4e772b3309d174340c7086d87533f4d527269b26814d94fdc2d7b494583a226c

problem: sigopt/evalset/Bukin06(dim=2, int=[0])
solver: TPE
budget: 80
repeats: 30
concurrency: 1

ID: d7670eedc4a9ba469ebb8a9e552fe392d331ca2241d11892f48f1d5fce4365d8

problem: sigopt/evalset/CarromTable(dim=2)
solver: Optuna
budget: 80
repeats: 30
concurrency: 1

ID: 0cfbe87abeab8c28a03a6f1b4f3f04c7d5d885ec919cfa82e98686e38ec357bc

problem: sigopt/evalset/CarromTable(dim=2)
solver: TPE
budget: 80
repeats: 30
concurrency: 1

ID: 5edee370b13ca497831aba9541e67c40963dc92568c6b31664320cc37d9f416f

problem: sigopt/evalset/CarromTable(dim=2, int=[0])
solver: Optuna
budget: 80
repeats: 30
concurrency: 1

ID: 64558e857bab80a017c8c4c7fd8b4d09f57a8c29c0a46a5a5519e6161ff8070b

problem: sigopt/evalset/CarromTable(dim=2, int=[0])
solver: TPE
budget: 80
repeats: 30
concurrency: 1

ID: 72ab536fb038676aaf065c9e1e4e65a4ab6530aa11134b823dd88bbbfa1deebb

problem: sigopt/evalset/Deb02(dim=6)
solver: Optuna
budget: 80
repeats: 30
concurrency: 1

ID: 29db2d0a0b10e6e1f45e270f9ecec8443b58ef7c9557398e1afe0365819b6537

problem: sigopt/evalset/Deb02(dim=6)
solver: TPE
budget: 80
repeats: 30
concurrency: 1

ID: fdc6266cb554357582d89eaabaff61afda2033708b6afe2af41ef7deee03ccfb

problem: sigopt/evalset/DeflectedCorrugatedSpring(dim=4)
solver: Optuna
budget: 80
repeats: 30
concurrency: 1

ID: 1af22482a0c849a5ab996bc5811347021d5a3352d14a4f50b80b43ae7fa54148

problem: sigopt/evalset/DeflectedCorrugatedSpring(dim=4)
solver: TPE
budget: 80
repeats: 30
concurrency: 1

ID: e2fcf7160f1f004ae3bf3d08427bac659d306dd078a7942f26f9b71d45f9476a

problem: sigopt/evalset/Easom(dim=4)
solver: Optuna
budget: 80
repeats: 30
concurrency: 1

ID: 66cafdadd66fa95cb1091980b4508e81ef165e42538acb3db9e560d3d05ce48c

problem: sigopt/evalset/Easom(dim=4)
solver: TPE
budget: 80
repeats: 30
concurrency: 1

ID: b5087592050024f2d1b5fe2b584a37009e1bb14062943987a8bd892c92e00689

problem: sigopt/evalset/Easom(dim=5)
solver: Optuna
budget: 80
repeats: 30
concurrency: 1

ID: dd65dd8919b645f06649fd4f3b4354a8ed1f98ebcee25c9f24ba1d1a6d1f764f

problem: sigopt/evalset/Easom(dim=5)
solver: TPE
budget: 80
repeats: 30
concurrency: 1

ID: 026cf006db02205edafefa0373d35c3d21f9bf5f73aaf9b0140ad8b7ea667dac

problem: sigopt/evalset/Exponential(dim=6)
solver: Optuna
budget: 80
repeats: 30
concurrency: 1

ID: d375e2076e80276b5f2e1e08767e8bc581c22d9249671af3301934a309dd08f9

problem: sigopt/evalset/Exponential(dim=6)
solver: TPE
budget: 80
repeats: 30
concurrency: 1

ID: 9ed0c9fa5fa1212cf8363517613dead2b412c6255fb4df0c4d577dee87a67dad

problem: sigopt/evalset/Hartmann3(dim=3)
solver: Optuna
budget: 80
repeats: 30
concurrency: 1

ID: 4dcf232ef87b9334534d5e82a45676ac37ff6ad22a7a5e7ec8c955e19aa243e4

problem: sigopt/evalset/Hartmann3(dim=3)
solver: TPE
budget: 80
repeats: 30
concurrency: 1

ID: f4d9d6052823cc44a52b6309b4dfdcf391b9d2118644e934cdead54394327196

problem: sigopt/evalset/LennardJones6(dim=6)
solver: Optuna
budget: 80
repeats: 30
concurrency: 1

ID: c4920d5bb8f157889bf1fd452f440bf027e50b9a9a2462b243f66f6981f25aa2

problem: sigopt/evalset/LennardJones6(dim=6)
solver: TPE
budget: 80
repeats: 30
concurrency: 1

ID: 69425670f741004875a1b3cfa90d2b57e0432e009214465d2f8172edacd8f79c

problem: sigopt/evalset/McCourt01(dim=7, res=10)
solver: Optuna
budget: 80
repeats: 30
concurrency: 1

ID: 1eb90774e8680c24522fd46f93f5ae39dbff60ca5759a94a988e97ef89449a6b

problem: sigopt/evalset/McCourt01(dim=7, res=10)
solver: TPE
budget: 80
repeats: 30
concurrency: 1

ID: ea03ac7ecf4c4cc8ed9913e359db54d805dcee67fbc0b9d8bca723f8c8839856

problem: sigopt/evalset/McCourt02(dim=7)
solver: Optuna
budget: 80
repeats: 30
concurrency: 1

ID: 306fc33c8ef4f94b223a850316c6f59551bddb5f05fed877d341d712c6d4a683

problem: sigopt/evalset/McCourt02(dim=7)
solver: TPE
budget: 80
repeats: 30
concurrency: 1

ID: ad9e6c1993a0232118d33bbb31a712e56766dc308ff424744313fdd729cc7987

problem: sigopt/evalset/McCourt06(dim=5, res=12)
solver: Optuna
budget: 80
repeats: 30
concurrency: 1

ID: f64db0f5f225d9dd15d19bcf8fe0f3c9f5184fd15926e4efc686a7194f85f005

problem: sigopt/evalset/McCourt06(dim=5, res=12)
solver: TPE
budget: 80
repeats: 30
concurrency: 1

ID: 08b8555de27d6e481943fe427ca3bd548e3febaf37311680a7b272c70e3710c2

problem: sigopt/evalset/McCourt07(dim=6, res=12)
solver: Optuna
budget: 80
repeats: 30
concurrency: 1

ID: bd31ea059a308f9daca40989c25139a56dae4a077c87f63003514c10fff604d5

problem: sigopt/evalset/McCourt07(dim=6, res=12)
solver: TPE
budget: 80
repeats: 30
concurrency: 1

ID: 43fe49690322ba402c8512d02278478019be3b542a3a8cb76ce7d828c9cc8b74

problem: sigopt/evalset/McCourt19(dim=2)
solver: Optuna
budget: 80
repeats: 30
concurrency: 1

ID: e8f7b9aca5c3129c5d48e1cb0c9aefd7bb8637f565070276f45ee523e47d48dc

problem: sigopt/evalset/McCourt19(dim=2)
solver: TPE
budget: 80
repeats: 30
concurrency: 1

ID: a7be5cdfec8412e2be11fd6d074af5adc0fd65102742e24b02347d49afb5ab69

problem: sigopt/evalset/McCourt22(dim=5)
solver: Optuna
budget: 80
repeats: 30
concurrency: 1

ID: 4b7f5baef55e5881f15fae9f06ccb9e6b41fb0847ea63ef038d4ad67cd22dad5

problem: sigopt/evalset/McCourt22(dim=5)
solver: TPE
budget: 80
repeats: 30
concurrency: 1

ID: 37797eb69370ee72e477422ed1963680255db0d64419937695766041fb0765e2

problem: sigopt/evalset/McCourt27(dim=3)
solver: Optuna
budget: 80
repeats: 30
concurrency: 1

ID: acce21aa5529dab14f15c43c757d89ec9fb45c2d4dec4eee376a046047e32b8c

problem: sigopt/evalset/McCourt27(dim=3)
solver: TPE
budget: 80
repeats: 30
concurrency: 1

ID: 0d891783e7095abbccb7482be7fa62e6895784b943f1c0d14f0be4fe77178bff

problem: sigopt/evalset/Michalewicz(dim=4)
solver: Optuna
budget: 80
repeats: 30
concurrency: 1

ID: 5915ad3448a2a2ab278194a439856b15a49c27ee5b187ef42f9c4200e7782171

problem: sigopt/evalset/Michalewicz(dim=4)
solver: TPE
budget: 80
repeats: 30
concurrency: 1

ID: 882afc4f976f3805433954c5ab29410fe979ee8a44342e5c62a115ae1af01bf3

problem: sigopt/evalset/Mishra06(dim=2)
solver: Optuna
budget: 80
repeats: 30
concurrency: 1

ID: 2243e1792beca103bc02555253233e6233b72c418dac2348a37761bf802f994e

problem: sigopt/evalset/Mishra06(dim=2)
solver: TPE
budget: 80
repeats: 30
concurrency: 1

ID: 78c00be95fb6dd5e1784950fd52c80cb58e49dfa6b63ffea0a49a9a7ba314461

problem: sigopt/evalset/Ned01(dim=2)
solver: Optuna
budget: 80
repeats: 30
concurrency: 1

ID: 0737bbed2e55bfb529e5a2b9a6bbb71946c3f433294ec0a154769426826148ec

problem: sigopt/evalset/Ned01(dim=2)
solver: TPE
budget: 80
repeats: 30
concurrency: 1

ID: 20b0e223a47eb7b1925d3cec82659ee9489d03be3fd99e4ae07dd48d80a056c9

problem: sigopt/evalset/Plateau(dim=2)
solver: Optuna
budget: 80
repeats: 30
concurrency: 1

ID: c54cb9323c118f67c087054e7f5dfa2b9a50f796f7cb8d12203fe3664512cf74

problem: sigopt/evalset/Plateau(dim=2)
solver: TPE
budget: 80
repeats: 30
concurrency: 1

ID: e3a02959393250bbfb7dcca21ec305ecc1969b0835e9942fd24deda98d98302a

problem: sigopt/evalset/Rastrigin(dim=8)
solver: Optuna
budget: 80
repeats: 30
concurrency: 1

ID: 85259cf4fb9177475d5a6228f82ced84c6074accaccee41fb1f45c1d1c0e5de8

problem: sigopt/evalset/Rastrigin(dim=8)
solver: TPE
budget: 80
repeats: 30
concurrency: 1

ID: a6eb383b7fd8da1bd614a548a3666c76600e7bdd8afb3d67d436c5ff6ad6d24d

problem: sigopt/evalset/Rastrigin(dim=8, res=0.1)
solver: Optuna
budget: 80
repeats: 30
concurrency: 1

ID: 60b4a33d8fed5653aac44cec59c1f37310378f30660d7563944b89389ee96957

problem: sigopt/evalset/Rastrigin(dim=8, res=0.1)
solver: TPE
budget: 80
repeats: 30
concurrency: 1

ID: 5df8a6762384448d08b062d3307ad01f85855509008ea8935a82f2a0f1a9a5c9

problem: sigopt/evalset/Sargan(dim=2, int=[0])
solver: Optuna
budget: 80
repeats: 30
concurrency: 1

ID: 72028594becc17af1f68e37512930c5af2bf5930129450b07b836b32b1adcae8

problem: sigopt/evalset/Sargan(dim=2, int=[0])
solver: TPE
budget: 80
repeats: 30
concurrency: 1

ID: c643a2192933fcf05692233098dccab54405faa23830adf3310cb2431a10359c

problem: sigopt/evalset/Schwefel20(dim=2)
solver: Optuna
budget: 80
repeats: 30
concurrency: 1

ID: d9b64d593a07ce986390ae55de660fb982e4ad9f879029ee5af34c231a22a9a0

problem: sigopt/evalset/Schwefel20(dim=2)
solver: TPE
budget: 80
repeats: 30
concurrency: 1

ID: 317c2fafbadb5979c56e237b46b4b11b7fdab4ed72caf7084d0551969c4fc3f3

problem: sigopt/evalset/Schwefel20(dim=2, int=[0])
solver: Optuna
budget: 80
repeats: 30
concurrency: 1

ID: f23036896292fe1b2573a8ae11720e63e4f619aa9985f92bb11d02295e5f99c9

problem: sigopt/evalset/Schwefel20(dim=2, int=[0])
solver: TPE
budget: 80
repeats: 30
concurrency: 1

ID: d7e1ee134d37455ff3eca3d41b4d83dbca364e5026a60c36e73c71e9c9da9621

problem: sigopt/evalset/Shekel05(dim=4)
solver: Optuna
budget: 80
repeats: 30
concurrency: 1

ID: ab6073ebc902618a8a025fd5e668746b59bb853f533a3cdbd7e417039fe5528c

problem: sigopt/evalset/Shekel05(dim=4)
solver: TPE
budget: 80
repeats: 30
concurrency: 1

ID: 69289af9474d8811e36c078810894fdbfe71eec5e64d1e69bd61b59f2c5a4b6d

problem: sigopt/evalset/Shekel07(dim=4)
solver: Optuna
budget: 80
repeats: 30
concurrency: 1

ID: 62dbb33e0bf3b3cf74c3699625f91b49f46186644d0f09bc7d67bd7b0201a21f

problem: sigopt/evalset/Shekel07(dim=4)
solver: TPE
budget: 80
repeats: 30
concurrency: 1

ID: 49338d0a7613afce6f09a487ea78dfb0dc87fd1dcf18aa76adf78433e03cd3dc

problem: sigopt/evalset/Sphere(dim=7)
solver: Optuna
budget: 80
repeats: 30
concurrency: 1

ID: 5bd4135f33cbec499f330f86ab5453360b13b85bf6d79913688bedd606b608b8

problem: sigopt/evalset/Sphere(dim=7)
solver: TPE
budget: 80
repeats: 30
concurrency: 1

ID: 93b128d2407b4388e51ebe19e9eadc07443bab174bf60e484ccbe66fe6044316

problem: sigopt/evalset/Sphere(dim=7, int=[0, 1, 2, 3, 4])
solver: Optuna
budget: 80
repeats: 30
concurrency: 1

ID: ccb569defd50efee4eafa191d836ac577a1a0c1a655987faaa00a503abde74ea

problem: sigopt/evalset/Sphere(dim=7, int=[0, 1, 2, 3, 4])
solver: TPE
budget: 80
repeats: 30
concurrency: 1

ID: 8f1ec9242c699bf1a5e2dbc22c861c16610f08945f7b0acdd8dfc5408ec2fd84

problem: sigopt/evalset/StyblinskiTang(dim=5)
solver: Optuna
budget: 80
repeats: 30
concurrency: 1

ID: 45e74efb75dc734c4ac0198a47c0155bdbebca05b837c09482f04d2d51905f75

problem: sigopt/evalset/StyblinskiTang(dim=5)
solver: TPE
budget: 80
repeats: 30
concurrency: 1

ID: f3b041e8630d5e7f5625f2a1767ec7acd8f5d238fb85218e7d2640dac471f6b3

problem: sigopt/evalset/Trid(dim=6)
solver: Optuna
budget: 80
repeats: 30
concurrency: 1

ID: 11f2f5fbdc24ea60fad3d5c285c93856b35142e4b9454d1d0dae668dd44b879f

problem: sigopt/evalset/Trid(dim=6)
solver: TPE
budget: 80
repeats: 30
concurrency: 1

ID: c06a343948dfbc04a30e0370894e7d603928d32a73465990a618fd45db78c6bf

problem: sigopt/evalset/Tripod(dim=2)
solver: Optuna
budget: 80
repeats: 30
concurrency: 1

ID: 38ad73a9f2a6fbbec8cbd3a2dcbd2ec704c3dc938dd2327cf8bc8ffab84d88b1

problem: sigopt/evalset/Tripod(dim=2)
solver: TPE
budget: 80
repeats: 30
concurrency: 1

ID: 629dcac5341c2fd9fc160fca4a1739f0caac080cd2483223e1edd2fd91387dd4

problem: sigopt/evalset/Weierstrass(dim=3)
solver: Optuna
budget: 80
repeats: 30
concurrency: 1

ID: e49c40b858250805da5a2c737998902974e595b3ed7bec471383b27bd463744b

problem: sigopt/evalset/Weierstrass(dim=3)
solver: TPE
budget: 80
repeats: 30
concurrency: 1

ID: 1cfe6da4632231dec7f8f3281d600a9453da184c045f04d9ff79aca6b90154ae

problem: sigopt/evalset/Xor(dim=9)
solver: Optuna
budget: 80
repeats: 30
concurrency: 1

ID: 21f47fa1d011900fcd818ec3e529cf71b0e92a491b8fee271db18a52f34d3d3c

problem: sigopt/evalset/Xor(dim=9)
solver: TPE
budget: 80
repeats: 30
concurrency: 1

ID: 366f5787b13521ee799bc560d165b88ab919ee584084ea6ef722e40c006befa7

problem: sigopt/evalset/YaoLiu(dim=5)
solver: Optuna
budget: 80
repeats: 30
concurrency: 1

ID: 8aa911aade6af58b4d3c21f721f32d5f3139ecac108f63ef6c4194f495c28f9a

problem: sigopt/evalset/YaoLiu(dim=5)
solver: TPE
budget: 80
repeats: 30
concurrency: 1
Ranking	Solver	Best (avg +- sd)	AUC (avg +- sd)	Elapsed (avg +- sd)
1	Optuna (study)	0.993390 +- 0.005154	79.062 +- 0.223	2.099 +- 0.272
1	TPE (study)	0.989686 +- 0.007573	78.846 +- 0.655	0.154 +- 0.024
Ranking	Solver	Best (avg +- sd)	AUC (avg +- sd)	Elapsed (avg +- sd)
1	TPE (study)	-22.872949 +- 1.005172	-1652.571 +- 190.729	0.052 +- 0.010
2	Optuna (study)	-22.194564 +- 3.416137	-1348.154 +- 352.782	0.514 +- 0.071
Ranking	Solver	Best (avg +- sd)	AUC (avg +- sd)	Elapsed (avg +- sd)
1	Optuna (study)	-0.717780 +- 0.182056	-22.876 +- 16.804	0.974 +- 0.108
1	TPE (study)	-0.727596 +- 0.196942	-29.251 +- 19.134	0.082 +- 0.017
Ranking	Solver	Best (avg +- sd)	AUC (avg +- sd)	Elapsed (avg +- sd)
1	TPE (study)	-0.275000 +- 0.068550	-5.367 +- 6.368	0.108 +- 0.018
2	Optuna (study)	-0.227778 +- 0.064310	-0.981 +- 8.374	1.447 +- 0.150
Ranking	Solver	Best (avg +- sd)	AUC (avg +- sd)	Elapsed (avg +- sd)
1	Optuna (study)	-39.322946 +- 6.127839	-972.224 +- 852.826	1.390 +- 0.141
1	TPE (study)	-41.375532 +- 5.999828	-1136.604 +- 1061.727	0.114 +- 0.012
Ranking	Solver	Best (avg +- sd)	AUC (avg +- sd)	Elapsed (avg +- sd)
1	Optuna (study)	-3.794874 +- 0.068767	-268.923 +- 16.981	0.714 +- 0.084
1	TPE (study)	-3.771731 +- 0.143582	-266.521 +- 16.429	0.067 +- 0.015
Ranking	Solver	Best (avg +- sd)	AUC (avg +- sd)	Elapsed (avg +- sd)
1	Optuna (study)	51.567655 +- 9.603779	5092.247 +- 559.509	1.890 +- 0.163
1	TPE (study)	47.929794 +- 8.504059	5011.361 +- 569.197	0.131 +- 0.008
Ranking	Solver	Best (avg +- sd)	AUC (avg +- sd)	Elapsed (avg +- sd)
1	TPE (study)	-19.382277 +- 2.139465	-1398.042 +- 240.191	0.048 +- 0.009
2	Optuna (study)	-18.558809 +- 2.898384	-1124.117 +- 355.133	0.546 +- 0.097
Ranking	Solver	Best (avg +- sd)	AUC (avg +- sd)	Elapsed (avg +- sd)
1	Optuna (study)	8.322043 +- 1.121514	906.763 +- 163.107	0.543 +- 0.086
1	TPE (study)	8.176882 +- 0.891681	898.070 +- 197.586	0.047 +- 0.008
Ranking	Solver	Best (avg +- sd)	AUC (avg +- sd)	Elapsed (avg +- sd)
1	Optuna (study)	-0.988995 +- 0.009684	-74.520 +- 1.445	1.411 +- 0.160
2	TPE (study)	-0.985751 +- 0.009745	-73.532 +- 1.698	0.113 +- 0.014