Skip to content

Instantly share code, notes, and snippets.

@sile

sile/report.md Secret

Created September 27, 2020 08:15
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save sile/f720168c85810757524e17c94978792a to your computer and use it in GitHub Desktop.
Save sile/f720168c85810757524e17c94978792a to your computer and use it in GitHub Desktop.
Download ZIP
kurobako benchmark example result
Raw
report.md

Benchmark Result Report

  • Kurobako Version: 0.2.6
  • Number of Solvers: 2
  • Number of Problems: 42
  • Metrics Precedence: best value -> AUC

Please refer to "A Strategy for Ranking Optimizers using Multiple Criteria" for the ranking strategy used in this report.

Table of Contents

  1. Overall Results
  2. Individual Results
  3. Solvers
  4. Problems
  5. Studies

Overall Results

Solver Borda Firsts
Optuna 1 38
TPE 4 41

Individual Results

(1) Problem: sigopt/evalset/Xor(dim=9)

Ranking Solver Best (avg +- sd) AUC (avg +- sd) Elapsed (avg +- sd)
1 Optuna (study) 0.993390 +- 0.005154 79.062 +- 0.223 2.099 +- 0.272
1 TPE (study) 0.989686 +- 0.007573 78.846 +- 0.655 0.154 +- 0.024

(2) Problem: sigopt/evalset/CarromTable(dim=2)

Ranking Solver Best (avg +- sd) AUC (avg +- sd) Elapsed (avg +- sd)
1 TPE (study) -22.872949 +- 1.005172 -1652.571 +- 190.729 0.052 +- 0.010
2 Optuna (study) -22.194564 +- 3.416137 -1348.154 +- 352.782 0.514 +- 0.071

(3) Problem: sigopt/evalset/DeflectedCorrugatedSpring(dim=4)

Ranking Solver Best (avg +- sd) AUC (avg +- sd) Elapsed (avg +- sd)
1 Optuna (study) -0.717780 +- 0.182056 -22.876 +- 16.804 0.974 +- 0.108
1 TPE (study) -0.727596 +- 0.196942 -29.251 +- 19.134 0.082 +- 0.017

(4) Problem: sigopt/evalset/McCourt07(dim=6, res=12)

Ranking Solver Best (avg +- sd) AUC (avg +- sd) Elapsed (avg +- sd)
1 TPE (study) -0.275000 +- 0.068550 -5.367 +- 6.368 0.108 +- 0.018
2 Optuna (study) -0.227778 +- 0.064310 -0.981 +- 8.374 1.447 +- 0.150

(5) Problem: sigopt/evalset/Trid(dim=6)

Ranking Solver Best (avg +- sd) AUC (avg +- sd) Elapsed (avg +- sd)
1 Optuna (study) -39.322946 +- 6.127839 -972.224 +- 852.826 1.390 +- 0.141
1 TPE (study) -41.375532 +- 5.999828 -1136.604 +- 1061.727 0.114 +- 0.012

(6) Problem: sigopt/evalset/Hartmann3(dim=3)

Ranking Solver Best (avg +- sd) AUC (avg +- sd) Elapsed (avg +- sd)
1 Optuna (study) -3.794874 +- 0.068767 -268.923 +- 16.981 0.714 +- 0.084
1 TPE (study) -3.771731 +- 0.143582 -266.521 +- 16.429 0.067 +- 0.015

(7) Problem: sigopt/evalset/Rastrigin(dim=8)

Ranking Solver Best (avg +- sd) AUC (avg +- sd) Elapsed (avg +- sd)
1 Optuna (study) 51.567655 +- 9.603779 5092.247 +- 559.509 1.890 +- 0.163
1 TPE (study) 47.929794 +- 8.504059 5011.361 +- 569.197 0.131 +- 0.008

(8) Problem: sigopt/evalset/CarromTable(dim=2, int=[0])

Ranking Solver Best (avg +- sd) AUC (avg +- sd) Elapsed (avg +- sd)
1 TPE (study) -19.382277 +- 2.139465 -1398.042 +- 240.191 0.048 +- 0.009
2 Optuna (study) -18.558809 +- 2.898384 -1124.117 +- 355.133 0.546 +- 0.097

(9) Problem: sigopt/evalset/Branin02(dim=2, int=[0])

Ranking Solver Best (avg +- sd) AUC (avg +- sd) Elapsed (avg +- sd)
1 Optuna (study) 8.322043 +- 1.121514 906.763 +- 163.107 0.543 +- 0.086
1 TPE (study) 8.176882 +- 0.891681 898.070 +- 197.586 0.047 +- 0.008

(10) Problem: sigopt/evalset/Exponential(dim=6)

Ranking Solver Best (avg +- sd) AUC (avg +- sd) Elapsed (avg +- sd)
1 Optuna (study) -0.988995 +- 0.009684 -74.520 +- 1.445 1.411 +- 0.160
2 TPE (study) -0.985751 +- 0.009745 -73.532 +- 1.698 0.113 +- 0.014

(11) Problem: sigopt/evalset/McCourt27(dim=3)

Ranking Solver Best (avg +- sd) AUC (avg +- sd) Elapsed (avg +- sd)
1 Optuna (study) -1.115265 +- 0.290447 -29.629 +- 27.596 0.736 +- 0.074
1 TPE (study) -1.181162 +- 0.392760 -28.515 +- 20.999 0.065 +- 0.008

(12) Problem: sigopt/evalset/Easom(dim=4)

Ranking Solver Best (avg +- sd) AUC (avg +- sd) Elapsed (avg +- sd)
1 Optuna (study) 17.207166 +- 4.150281 1532.213 +- 153.864 0.965 +- 0.118
1 TPE (study) 16.640832 +- 4.458301 1513.019 +- 152.896 0.081 +- 0.015

(13) Problem: sigopt/evalset/Michalewicz(dim=4)

Ranking Solver Best (avg +- sd) AUC (avg +- sd) Elapsed (avg +- sd)
1 Optuna (study) -2.456921 +- 0.409687 -154.184 +- 24.965 1.017 +- 0.103
1 TPE (study) -2.429811 +- 0.344868 -151.232 +- 24.882 0.079 +- 0.012

(14) Problem: sigopt/evalset/Sphere(dim=7)

Ranking Solver Best (avg +- sd) AUC (avg +- sd) Elapsed (avg +- sd)
1 Optuna (study) 2.584901 +- 1.340599 653.296 +- 206.452 1.631 +- 0.149
1 TPE (study) 2.519518 +- 1.088015 712.224 +- 149.846 0.129 +- 0.017

(15) Problem: sigopt/evalset/Tripod(dim=2)

Ranking Solver Best (avg +- sd) AUC (avg +- sd) Elapsed (avg +- sd)
1 Optuna (study) 5.096237 +- 2.344442 1012.417 +- 305.085 0.494 +- 0.058
1 TPE (study) 4.703087 +- 1.725078 955.442 +- 243.257 0.050 +- 0.008

(16) Problem: sigopt/evalset/Shekel05(dim=4)

Ranking Solver Best (avg +- sd) AUC (avg +- sd) Elapsed (avg +- sd)
1 Optuna (study) -1.589587 +- 0.735620 -69.877 +- 19.731 0.974 +- 0.084
1 TPE (study) -2.369999 +- 1.654154 -102.969 +- 69.721 0.086 +- 0.016

(17) Problem: sigopt/evalset/Ackley(dim=3)

Ranking Solver Best (avg +- sd) AUC (avg +- sd) Elapsed (avg +- sd)
1 Optuna (study) 5.059500 +- 1.552897 710.526 +- 117.211 0.738 +- 0.102
1 TPE (study) 5.024110 +- 1.502778 744.546 +- 142.450 0.067 +- 0.021

(18) Problem: sigopt/evalset/LennardJones6(dim=6)

Ranking Solver Best (avg +- sd) AUC (avg +- sd) Elapsed (avg +- sd)
1 Optuna (study) -0.978761 +- 0.045789 -50.342 +- 9.442 1.409 +- 0.147
1 TPE (study) -0.964420 +- 0.065015 -51.227 +- 10.873 0.108 +- 0.014

(19) Problem: sigopt/evalset/McCourt22(dim=5)

Ranking Solver Best (avg +- sd) AUC (avg +- sd) Elapsed (avg +- sd)
1 Optuna (study) -2.578230 +- 0.242046 -165.000 +- 23.266 1.210 +- 0.120
1 TPE (study) -2.707553 +- 0.254801 -166.977 +- 15.933 0.099 +- 0.023

(20) Problem: sigopt/evalset/Ackley(dim=11, int=[0, 1, 2])

Ranking Solver Best (avg +- sd) AUC (avg +- sd) Elapsed (avg +- sd)
1 Optuna (study) 13.936461 +- 1.516013 1300.225 +- 72.129 2.650 +- 0.300
1 TPE (study) 14.140112 +- 1.929461 1290.064 +- 103.408 0.181 +- 0.019

(21) Problem: sigopt/evalset/Mishra06(dim=2)

Ranking Solver Best (avg +- sd) AUC (avg +- sd) Elapsed (avg +- sd)
1 Optuna (study) -1.856294 +- 0.287699 -100.549 +- 21.900 0.502 +- 0.045
1 TPE (study) -1.944638 +- 0.251441 -100.954 +- 23.131 0.049 +- 0.008

(22) Problem: sigopt/evalset/Weierstrass(dim=3)

Ranking Solver Best (avg +- sd) AUC (avg +- sd) Elapsed (avg +- sd)
1 Optuna (study) 13.312590 +- 0.432109 1113.177 +- 31.581 0.705 +- 0.088
1 TPE (study) 13.169722 +- 0.328912 1099.970 +- 24.815 0.066 +- 0.009

(23) Problem: sigopt/evalset/Ackley(dim=5)

Ranking Solver Best (avg +- sd) AUC (avg +- sd) Elapsed (avg +- sd)
1 Optuna (study) 8.113488 +- 1.606887 980.545 +- 158.876 1.223 +- 0.169
1 TPE (study) 8.236194 +- 1.903449 1001.801 +- 126.397 0.091 +- 0.017

(24) Problem: sigopt/evalset/Bukin06(dim=2, int=[0])

Ranking Solver Best (avg +- sd) AUC (avg +- sd) Elapsed (avg +- sd)
1 Optuna (study) 9.722000 +- 5.270652 1883.877 +- 525.400 0.551 +- 0.063
1 TPE (study) 8.347149 +- 3.900352 1778.613 +- 446.735 0.050 +- 0.009

(25) Problem: sigopt/evalset/Schwefel20(dim=2, int=[0])

Ranking Solver Best (avg +- sd) AUC (avg +- sd) Elapsed (avg +- sd)
1 Optuna (study) 2.805165 +- 1.868455 955.988 +- 269.244 0.614 +- 0.071
1 TPE (study) 3.147916 +- 1.906583 940.110 +- 311.947 0.049 +- 0.009

(26) Problem: sigopt/evalset/Deb02(dim=6)

Ranking Solver Best (avg +- sd) AUC (avg +- sd) Elapsed (avg +- sd)
1 Optuna (study) -0.745466 +- 0.058212 -51.715 +- 3.635 1.418 +- 0.148
1 TPE (study) -0.731667 +- 0.071744 -51.128 +- 4.275 0.107 +- 0.014

(27) Problem: sigopt/evalset/Ackley(dim=11)

Ranking Solver Best (avg +- sd) AUC (avg +- sd) Elapsed (avg +- sd)
1 Optuna (study) 14.277443 +- 1.794100 1315.809 +- 91.249 2.556 +- 0.265
1 TPE (study) 14.521596 +- 1.513862 1305.979 +- 98.917 0.179 +- 0.015

(28) Problem: sigopt/evalset/Sphere(dim=7, int=[0, 1, 2, 3, 4])

Ranking Solver Best (avg +- sd) AUC (avg +- sd) Elapsed (avg +- sd)
1 Optuna (study) 2.103150 +- 1.342227 698.066 +- 210.207 1.755 +- 0.152
1 TPE (study) 2.689469 +- 1.195182 795.176 +- 214.213 0.119 +- 0.012

(29) Problem: sigopt/evalset/StyblinskiTang(dim=5)

Ranking Solver Best (avg +- sd) AUC (avg +- sd) Elapsed (avg +- sd)
1 Optuna (study) -157.061698 +- 13.683493 -10443.805 +- 860.943 1.159 +- 0.112
1 TPE (study) -159.368114 +- 13.823497 -10983.855 +- 1194.895 0.091 +- 0.016

(30) Problem: sigopt/evalset/Ned01(dim=2)

Ranking Solver Best (avg +- sd) AUC (avg +- sd) Elapsed (avg +- sd)
1 Optuna (study) 0.076205 +- 0.070345 14.860 +- 5.824 0.501 +- 0.046
1 TPE (study) 0.038726 +- 0.088347 13.520 +- 7.614 0.048 +- 0.008

(31) Problem: sigopt/evalset/YaoLiu(dim=5)

Ranking Solver Best (avg +- sd) AUC (avg +- sd) Elapsed (avg +- sd)
1 Optuna (study) 20.587560 +- 5.389727 2416.369 +- 411.935 1.181 +- 0.137
1 TPE (study) 22.129447 +- 6.635147 2599.144 +- 393.332 0.099 +- 0.021

(32) Problem: sigopt/evalset/Easom(dim=5)

Ranking Solver Best (avg +- sd) AUC (avg +- sd) Elapsed (avg +- sd)
1 Optuna (study) 19.598449 +- 2.335792 1618.950 +- 77.894 1.178 +- 0.134
1 TPE (study) 19.675417 +- 2.527447 1599.711 +- 140.754 0.099 +- 0.019

(33) Problem: sigopt/evalset/Ackley(dim=3, res=1)

Ranking Solver Best (avg +- sd) AUC (avg +- sd) Elapsed (avg +- sd)
1 Optuna (study) 4.300000 +- 1.417745 658.700 +- 91.811 0.742 +- 0.118
1 TPE (study) 3.900000 +- 1.011599 595.333 +- 128.060 0.065 +- 0.009

(34) Problem: sigopt/evalset/McCourt19(dim=2)

Ranking Solver Best (avg +- sd) AUC (avg +- sd) Elapsed (avg +- sd)
1 Optuna (study) -8.461423 +- 0.131150 -594.724 +- 32.279 0.531 +- 0.071
1 TPE (study) -8.506519 +- 0.097196 -584.351 +- 34.321 0.050 +- 0.009

(35) Problem: sigopt/evalset/Plateau(dim=2)

Ranking Solver Best (avg +- sd) AUC (avg +- sd) Elapsed (avg +- sd)
1 Optuna (study) 30.000000 +- 0.000000 2386.167 +- 8.513 0.506 +- 0.047
1 TPE (study) 30.000000 +- 0.000000 2384.400 +- 8.333 0.049 +- 0.008

(36) Problem: sigopt/evalset/McCourt01(dim=7, res=10)

Ranking Solver Best (avg +- sd) AUC (avg +- sd) Elapsed (avg +- sd)
1 Optuna (study) 0.026667 +- 0.092856 21.297 +- 10.298 1.623 +- 0.157
1 TPE (study) 0.013333 +- 0.123108 21.433 +- 8.740 0.129 +- 0.026

(37) Problem: sigopt/evalset/Shekel07(dim=4)

Ranking Solver Best (avg +- sd) AUC (avg +- sd) Elapsed (avg +- sd)
1 Optuna (study) -2.223391 +- 0.891697 -91.441 +- 20.872 0.978 +- 0.083
1 TPE (study) -2.352949 +- 1.194168 -95.508 +- 33.456 0.084 +- 0.015

(38) Problem: sigopt/evalset/McCourt06(dim=5, res=12)

Ranking Solver Best (avg +- sd) AUC (avg +- sd) Elapsed (avg +- sd)
1 Optuna (study) 2.847222 +- 0.048512 234.444 +- 4.336 1.185 +- 0.126
1 TPE (study) 2.844444 +- 0.041574 234.839 +- 5.059 0.097 +- 0.014

(39) Problem: sigopt/evalset/Sargan(dim=2, int=[0])

Ranking Solver Best (avg +- sd) AUC (avg +- sd) Elapsed (avg +- sd)
1 Optuna (study) 0.000881 +- 0.002711 73.963 +- 37.908 0.616 +- 0.071
1 TPE (study) 0.056329 +- 0.301516 79.598 +- 53.628 0.048 +- 0.013

(40) Problem: sigopt/evalset/McCourt02(dim=7)

Ranking Solver Best (avg +- sd) AUC (avg +- sd) Elapsed (avg +- sd)
1 Optuna (study) -2.686687 +- 0.026472 -203.463 +- 4.287 1.677 +- 0.165
1 TPE (study) -2.691230 +- 0.027726 -204.052 +- 3.832 0.125 +- 0.018

(41) Problem: sigopt/evalset/Rastrigin(dim=8, res=0.1)

Ranking Solver Best (avg +- sd) AUC (avg +- sd) Elapsed (avg +- sd)
1 TPE (study) 40.000000 +- 9.309493 4412.000 +- 698.185 0.141 +- 0.017
2 Optuna (study) 46.000000 +- 10.198039 4801.333 +- 713.787 1.896 +- 0.163

(42) Problem: sigopt/evalset/Schwefel20(dim=2)

Ranking Solver Best (avg +- sd) AUC (avg +- sd) Elapsed (avg +- sd)
1 Optuna (study) 3.066478 +- 1.381306 952.353 +- 255.962 0.583 +- 0.072
1 TPE (study) 2.518236 +- 1.449889 1103.380 +- 314.851 0.045 +- 0.005

Solvers

ID: 487e04184b5eeb61450126dfd6552b09ffc08402cf3183770d239fb094201cf2

recipe:

{
  "optuna": {}
}

specification:

{
  "name": "Optuna",
  "attrs": {
    "github": "https://github.com/optuna/optuna",
    "paper": "Akiba, Takuya, et al. \"Optuna: A next-generation hyperparameter optimization framework.\" Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. ACM, 2019.",
    "version": "optuna=2.1.0, kurobako-py=0.1.8"
  },
  "capabilities": [
    "UNIFORM_CONTINUOUS",
    "UNIFORM_DISCRETE",
    "LOG_UNIFORM_CONTINUOUS",
    "CATEGORICAL",
    "CONDITIONAL",
    "CONCURRENT"
  ]
}

ID: cd5a17022fd1fec0554d9192671f1901113ace2492e20d59ac0417f3666c4618

recipe:

{
  "command": {
    "path": "cargo",
    "args": [
      "run",
      "--release",
      "--example",
      "tpe-solver"
    ]
  }
}

specification:

{
  "name": "TPE",
  "attrs": {},
  "capabilities": [
    "UNIFORM_CONTINUOUS",
    "UNIFORM_DISCRETE",
    "LOG_UNIFORM_CONTINUOUS",
    "CATEGORICAL",
    "CONCURRENT"
  ]
}

Problems

ID: 762d606fd111289ffacaa9410ef618348513d70d9fe903e5d71a60978c28d213

recipe:

{
  "sigopt": {
    "name": "ACKLEY",
    "dim": 11
  }
}

specification:

{
  "name": "sigopt/evalset/Ackley(dim=11)",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "CONTINUOUS",
        "low": -10.0,
        "high": 30.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": -10.0,
        "high": 30.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p2",
      "range": {
        "type": "CONTINUOUS",
        "low": -10.0,
        "high": 30.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p3",
      "range": {
        "type": "CONTINUOUS",
        "low": -10.0,
        "high": 30.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p4",
      "range": {
        "type": "CONTINUOUS",
        "low": -10.0,
        "high": 30.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p5",
      "range": {
        "type": "CONTINUOUS",
        "low": -10.0,
        "high": 30.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p6",
      "range": {
        "type": "CONTINUOUS",
        "low": -10.0,
        "high": 30.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p7",
      "range": {
        "type": "CONTINUOUS",
        "low": -10.0,
        "high": 30.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p8",
      "range": {
        "type": "CONTINUOUS",
        "low": -10.0,
        "high": 30.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p9",
      "range": {
        "type": "CONTINUOUS",
        "low": -10.0,
        "high": 30.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p10",
      "range": {
        "type": "CONTINUOUS",
        "low": -10.0,
        "high": 30.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}

ID: 5711ea99766d928a6218dbeaadd93bd28e82a9c3b19b0eac5b64e01b7f252fac

recipe:

{
  "sigopt": {
    "name": "ACKLEY",
    "dim": 11,
    "int": [
      0,
      1,
      2
    ]
  }
}

specification:

{
  "name": "sigopt/evalset/Ackley(dim=11, int=[0, 1, 2])",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "DISCRETE",
        "low": -10,
        "high": 30
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "DISCRETE",
        "low": -10,
        "high": 30
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p2",
      "range": {
        "type": "DISCRETE",
        "low": -10,
        "high": 30
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p3",
      "range": {
        "type": "CONTINUOUS",
        "low": -10.0,
        "high": 30.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p4",
      "range": {
        "type": "CONTINUOUS",
        "low": -10.0,
        "high": 30.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p5",
      "range": {
        "type": "CONTINUOUS",
        "low": -10.0,
        "high": 30.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p6",
      "range": {
        "type": "CONTINUOUS",
        "low": -10.0,
        "high": 30.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p7",
      "range": {
        "type": "CONTINUOUS",
        "low": -10.0,
        "high": 30.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p8",
      "range": {
        "type": "CONTINUOUS",
        "low": -10.0,
        "high": 30.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p9",
      "range": {
        "type": "CONTINUOUS",
        "low": -10.0,
        "high": 30.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p10",
      "range": {
        "type": "CONTINUOUS",
        "low": -10.0,
        "high": 30.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}

ID: 48bc49268474110e0c2a181bb78990c6dbd331d61fb6e082d3bd095d4de815e2

recipe:

{
  "sigopt": {
    "name": "ACKLEY",
    "dim": 3
  }
}

specification:

{
  "name": "sigopt/evalset/Ackley(dim=3)",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "CONTINUOUS",
        "low": -10.0,
        "high": 30.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": -10.0,
        "high": 30.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p2",
      "range": {
        "type": "CONTINUOUS",
        "low": -10.0,
        "high": 30.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}

ID: bc80da85f8186f371a2a127d82ff7136379712e311358c60678085830effc226

recipe:

{
  "sigopt": {
    "name": "ACKLEY",
    "dim": 3,
    "res": 1.0
  }
}

specification:

{
  "name": "sigopt/evalset/Ackley(dim=3, res=1)",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "CONTINUOUS",
        "low": -10.0,
        "high": 30.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": -10.0,
        "high": 30.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p2",
      "range": {
        "type": "CONTINUOUS",
        "low": -10.0,
        "high": 30.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}

ID: 66cda488c56779bfd9d3bb1203160dce3d784c9bc293daabacf0509790e43aed

recipe:

{
  "sigopt": {
    "name": "ACKLEY",
    "dim": 5
  }
}

specification:

{
  "name": "sigopt/evalset/Ackley(dim=5)",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "CONTINUOUS",
        "low": -10.0,
        "high": 30.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": -10.0,
        "high": 30.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p2",
      "range": {
        "type": "CONTINUOUS",
        "low": -10.0,
        "high": 30.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p3",
      "range": {
        "type": "CONTINUOUS",
        "low": -10.0,
        "high": 30.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p4",
      "range": {
        "type": "CONTINUOUS",
        "low": -10.0,
        "high": 30.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}

ID: 2318f2882267c2f27d174954c9976ade13be9e978006ce076c6dac22fa7c8463

recipe:

{
  "sigopt": {
    "name": "BRANIN02",
    "dim": 2,
    "int": [
      0
    ]
  }
}

specification:

{
  "name": "sigopt/evalset/Branin02(dim=2, int=[0])",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "DISCRETE",
        "low": -5,
        "high": 15
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": -5.0,
        "high": 15.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}

ID: 6ab293911d0424ebb16aa03a9185f16e62c9023da1c0ccbf4353b0433fa42d10

recipe:

{
  "sigopt": {
    "name": "BUKIN06",
    "dim": 2,
    "int": [
      0
    ]
  }
}

specification:

{
  "name": "sigopt/evalset/Bukin06(dim=2, int=[0])",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "DISCRETE",
        "low": -15,
        "high": -5
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": -3.0,
        "high": 3.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}

ID: 0dcb70363493ed100a24a44a12d3d347aa1a0903a18e1ed8abe003df7a74355b

recipe:

{
  "sigopt": {
    "name": "CARROM_TABLE",
    "dim": 2
  }
}

specification:

{
  "name": "sigopt/evalset/CarromTable(dim=2)",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "CONTINUOUS",
        "low": -10.0,
        "high": 10.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": -10.0,
        "high": 10.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}

ID: 20c7917b2603e9d2e6928ad861cc2b8edf702d96f5759ef27ca85c11335edf16

recipe:

{
  "sigopt": {
    "name": "CARROM_TABLE",
    "dim": 2,
    "int": [
      0
    ]
  }
}

specification:

{
  "name": "sigopt/evalset/CarromTable(dim=2, int=[0])",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "DISCRETE",
        "low": -10,
        "high": 10
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": -10.0,
        "high": 10.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}

ID: 6fd3e8ef60eec3daf630366f99491d35b6f05a4163ebb6d0b4db74926bc42ce2

recipe:

{
  "sigopt": {
    "name": "DEB02",
    "dim": 6
  }
}

specification:

{
  "name": "sigopt/evalset/Deb02(dim=6)",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p2",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p3",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p4",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p5",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}

ID: 1210ccbbf66a139a961e1392f8cc35351922a8f089a440ccd54af4577e5f368b

recipe:

{
  "sigopt": {
    "name": "DEFLECTED_CORRUGATED_SPRING",
    "dim": 4
  }
}

specification:

{
  "name": "sigopt/evalset/DeflectedCorrugatedSpring(dim=4)",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 7.5
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 7.5
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p2",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 7.5
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p3",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 7.5
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}

ID: 30f46f4f59c9a6b9ba33e0dadd611080e606ff70a844b2c68bfeb66a7a07e405

recipe:

{
  "sigopt": {
    "name": "EASOM",
    "dim": 4
  }
}

specification:

{
  "name": "sigopt/evalset/Easom(dim=4)",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "CONTINUOUS",
        "low": -100.0,
        "high": 20.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": -100.0,
        "high": 20.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p2",
      "range": {
        "type": "CONTINUOUS",
        "low": -100.0,
        "high": 20.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p3",
      "range": {
        "type": "CONTINUOUS",
        "low": -100.0,
        "high": 20.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}

ID: a39b57cec0151c892256bc8386f43dc15e83c2a985c9761fbf46f0abc541307e

recipe:

{
  "sigopt": {
    "name": "EASOM",
    "dim": 5
  }
}

specification:

{
  "name": "sigopt/evalset/Easom(dim=5)",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "CONTINUOUS",
        "low": -100.0,
        "high": 20.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": -100.0,
        "high": 20.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p2",
      "range": {
        "type": "CONTINUOUS",
        "low": -100.0,
        "high": 20.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p3",
      "range": {
        "type": "CONTINUOUS",
        "low": -100.0,
        "high": 20.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p4",
      "range": {
        "type": "CONTINUOUS",
        "low": -100.0,
        "high": 20.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}

ID: 29849239fd4b7b5d4543cec4bc2cbb857fa41fa8b4147a765850b43409317dc1

recipe:

{
  "sigopt": {
    "name": "EXPONENTIAL",
    "dim": 6
  }
}

specification:

{
  "name": "sigopt/evalset/Exponential(dim=6)",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "CONTINUOUS",
        "low": -0.7,
        "high": 0.2
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": -0.7,
        "high": 0.2
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p2",
      "range": {
        "type": "CONTINUOUS",
        "low": -0.7,
        "high": 0.2
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p3",
      "range": {
        "type": "CONTINUOUS",
        "low": -0.7,
        "high": 0.2
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p4",
      "range": {
        "type": "CONTINUOUS",
        "low": -0.7,
        "high": 0.2
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p5",
      "range": {
        "type": "CONTINUOUS",
        "low": -0.7,
        "high": 0.2
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}

ID: 1e08466d41a46890e37413c5f989e30c6f2155b3e3e99721997e6d9ed29148d5

recipe:

{
  "sigopt": {
    "name": "HARTMANN3",
    "dim": 3
  }
}

specification:

{
  "name": "sigopt/evalset/Hartmann3(dim=3)",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p2",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}

ID: 4cc33f4e2ff5625738795af29a4d94fa607d5f5949ece8ad44b73b1e63b63339

recipe:

{
  "sigopt": {
    "name": "LENNARD_JONES6",
    "dim": 6
  }
}

specification:

{
  "name": "sigopt/evalset/LennardJones6(dim=6)",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "CONTINUOUS",
        "low": -3.0,
        "high": 3.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": -3.0,
        "high": 3.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p2",
      "range": {
        "type": "CONTINUOUS",
        "low": -3.0,
        "high": 3.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p3",
      "range": {
        "type": "CONTINUOUS",
        "low": -3.0,
        "high": 3.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p4",
      "range": {
        "type": "CONTINUOUS",
        "low": -3.0,
        "high": 3.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p5",
      "range": {
        "type": "CONTINUOUS",
        "low": -3.0,
        "high": 3.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}

ID: ccf73084ef5dbe3746e1d5252d6bfaa643bc1ece3358a4c77decf923d8e0facb

recipe:

{
  "sigopt": {
    "name": "MC_COURT01",
    "dim": 7,
    "res": 10.0
  }
}

specification:

{
  "name": "sigopt/evalset/McCourt01(dim=7, res=10)",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p2",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p3",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p4",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p5",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p6",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}

ID: e7895b7bb5dac94288be96e61ad121d23e185003eaefd87ba5e1023a6132104e

recipe:

{
  "sigopt": {
    "name": "MC_COURT02",
    "dim": 7
  }
}

specification:

{
  "name": "sigopt/evalset/McCourt02(dim=7)",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p2",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p3",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p4",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p5",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p6",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}

ID: cff13c7e9a01ac1b8cce1a957f7caa7cc6d438ced9a8c25b658215794acc6d92

recipe:

{
  "sigopt": {
    "name": "MC_COURT06",
    "dim": 5,
    "res": 12.0
  }
}

specification:

{
  "name": "sigopt/evalset/McCourt06(dim=5, res=12)",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p2",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p3",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p4",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}

ID: 1464bdea4ea2f345666eae6ecdda641b6a196281174f5ba8007a8e8a2bec532e

recipe:

{
  "sigopt": {
    "name": "MC_COURT07",
    "dim": 6,
    "res": 12.0
  }
}

specification:

{
  "name": "sigopt/evalset/McCourt07(dim=6, res=12)",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p2",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p3",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p4",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p5",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}

ID: c402d6ef019d823cfda5127b120f2bd5eb3e021edc112118ae0c9f07e72de789

recipe:

{
  "sigopt": {
    "name": "MC_COURT19",
    "dim": 2
  }
}

specification:

{
  "name": "sigopt/evalset/McCourt19(dim=2)",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}

ID: 541f33a97b45aa4a10f3197b56751eb385ded69f15d40453f8b7dee5ba7882a8

recipe:

{
  "sigopt": {
    "name": "MC_COURT22",
    "dim": 5
  }
}

specification:

{
  "name": "sigopt/evalset/McCourt22(dim=5)",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p2",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p3",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p4",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}

ID: 2fef32d714633c85c412e13e086a1ec8df60117de097a4209b590376b1787729

recipe:

{
  "sigopt": {
    "name": "MC_COURT27",
    "dim": 3
  }
}

specification:

{
  "name": "sigopt/evalset/McCourt27(dim=3)",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p2",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}

ID: 3902a3f083b3cb4c95db2c0166a0a337063e4555d7a2f9d2d31c367d32758f4e

recipe:

{
  "sigopt": {
    "name": "MICHALEWICZ",
    "dim": 4
  }
}

specification:

{
  "name": "sigopt/evalset/Michalewicz(dim=4)",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 3.141592653589793
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 3.141592653589793
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p2",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 3.141592653589793
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p3",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 3.141592653589793
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}

ID: 59c429b4a300b87b138a006f1128ff1d16e2c50a9c909a88ef514a10e3c80148

recipe:

{
  "sigopt": {
    "name": "MISHRA06",
    "dim": 2
  }
}

specification:

{
  "name": "sigopt/evalset/Mishra06(dim=2)",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "CONTINUOUS",
        "low": -10.0,
        "high": 10.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": -10.0,
        "high": 10.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}

ID: 862877d4e1f14c9bea4f924a5c6366938cc8e2aadd70df465af0586d67a99f8b

recipe:

{
  "sigopt": {
    "name": "NED01",
    "dim": 2
  }
}

specification:

{
  "name": "sigopt/evalset/Ned01(dim=2)",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "CONTINUOUS",
        "low": -10.0,
        "high": 10.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": -10.0,
        "high": 10.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}

ID: c6e62d2a584679ad6fcdec6abc3bab8ab394e333a01e662053a28c2837ef57a9

recipe:

{
  "sigopt": {
    "name": "PLATEAU",
    "dim": 2
  }
}

specification:

{
  "name": "sigopt/evalset/Plateau(dim=2)",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "CONTINUOUS",
        "low": -2.34,
        "high": 5.12
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": -2.34,
        "high": 5.12
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}

ID: 1eb7796f1b2d5f244bfd8eefd6b3d97091d460e9ab56c61d71a29ad15e7fa0da

recipe:

{
  "sigopt": {
    "name": "RASTRIGIN",
    "dim": 8
  }
}

specification:

{
  "name": "sigopt/evalset/Rastrigin(dim=8)",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "CONTINUOUS",
        "low": -5.0,
        "high": 2.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": -5.0,
        "high": 2.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p2",
      "range": {
        "type": "CONTINUOUS",
        "low": -2.0,
        "high": 5.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p3",
      "range": {
        "type": "CONTINUOUS",
        "low": -2.0,
        "high": 5.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p4",
      "range": {
        "type": "CONTINUOUS",
        "low": -5.0,
        "high": 2.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p5",
      "range": {
        "type": "CONTINUOUS",
        "low": -5.0,
        "high": 2.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p6",
      "range": {
        "type": "CONTINUOUS",
        "low": -2.0,
        "high": 5.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p7",
      "range": {
        "type": "CONTINUOUS",
        "low": -2.0,
        "high": 5.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}

ID: ec359d6b8722b456204bb2f41274079d5c5c0e08a6b3db98a762d3501a380e96

recipe:

{
  "sigopt": {
    "name": "RASTRIGIN",
    "dim": 8,
    "res": 0.1
  }
}

specification:

{
  "name": "sigopt/evalset/Rastrigin(dim=8, res=0.1)",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "CONTINUOUS",
        "low": -5.0,
        "high": 2.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": -5.0,
        "high": 2.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p2",
      "range": {
        "type": "CONTINUOUS",
        "low": -2.0,
        "high": 5.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p3",
      "range": {
        "type": "CONTINUOUS",
        "low": -2.0,
        "high": 5.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p4",
      "range": {
        "type": "CONTINUOUS",
        "low": -5.0,
        "high": 2.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p5",
      "range": {
        "type": "CONTINUOUS",
        "low": -5.0,
        "high": 2.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p6",
      "range": {
        "type": "CONTINUOUS",
        "low": -2.0,
        "high": 5.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p7",
      "range": {
        "type": "CONTINUOUS",
        "low": -2.0,
        "high": 5.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}

ID: e014fb18e27703930d93dd7e7ee8178ce5ffad97027d013fac90c263049c70d7

recipe:

{
  "sigopt": {
    "name": "SARGAN",
    "dim": 2,
    "int": [
      0
    ]
  }
}

specification:

{
  "name": "sigopt/evalset/Sargan(dim=2, int=[0])",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "DISCRETE",
        "low": -2,
        "high": 4
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": -2.0,
        "high": 4.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}

ID: fcbd60ae002525d084f788a792305d491f32a61ac99ac711a9cfe9d1860f8d34

recipe:

{
  "sigopt": {
    "name": "SCHWEFEL20",
    "dim": 2
  }
}

specification:

{
  "name": "sigopt/evalset/Schwefel20(dim=2)",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "CONTINUOUS",
        "low": -60.0,
        "high": 100.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": -60.0,
        "high": 100.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}

ID: 6de30105fa47b8ecbda30c3676ee059bcfcc221b82a83736bb8b479f4559da6a

recipe:

{
  "sigopt": {
    "name": "SCHWEFEL20",
    "dim": 2,
    "int": [
      0
    ]
  }
}

specification:

{
  "name": "sigopt/evalset/Schwefel20(dim=2, int=[0])",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "DISCRETE",
        "low": -60,
        "high": 100
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": -60.0,
        "high": 100.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}

ID: 47fe6e07816771591c8ea2640dea44bce93ff9171060ac4c3ece8b1c71a32aa2

recipe:

{
  "sigopt": {
    "name": "SHEKEL05",
    "dim": 4
  }
}

specification:

{
  "name": "sigopt/evalset/Shekel05(dim=4)",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "CONTINUOUS",
        "low": -0.0,
        "high": 10.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": -0.0,
        "high": 10.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p2",
      "range": {
        "type": "CONTINUOUS",
        "low": -0.0,
        "high": 10.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p3",
      "range": {
        "type": "CONTINUOUS",
        "low": -0.0,
        "high": 10.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}

ID: cd5ee5f8e7c0328b28837d220987f3ed318887b201e2dccd24559937f37c215a

recipe:

{
  "sigopt": {
    "name": "SHEKEL07",
    "dim": 4
  }
}

specification:

{
  "name": "sigopt/evalset/Shekel07(dim=4)",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "CONTINUOUS",
        "low": -0.0,
        "high": 10.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": -0.0,
        "high": 10.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p2",
      "range": {
        "type": "CONTINUOUS",
        "low": -0.0,
        "high": 10.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p3",
      "range": {
        "type": "CONTINUOUS",
        "low": -0.0,
        "high": 10.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}

ID: 429f7ada6a8f5648aee7930fdf1f3af0807ba052ca921cac32dc102c3d6d32f2

recipe:

{
  "sigopt": {
    "name": "SPHERE",
    "dim": 7
  }
}

specification:

{
  "name": "sigopt/evalset/Sphere(dim=7)",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "CONTINUOUS",
        "low": -5.12,
        "high": 2.12
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": -5.12,
        "high": 2.12
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p2",
      "range": {
        "type": "CONTINUOUS",
        "low": -5.12,
        "high": 2.12
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p3",
      "range": {
        "type": "CONTINUOUS",
        "low": -5.12,
        "high": 2.12
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p4",
      "range": {
        "type": "CONTINUOUS",
        "low": -5.12,
        "high": 2.12
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p5",
      "range": {
        "type": "CONTINUOUS",
        "low": -5.12,
        "high": 2.12
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p6",
      "range": {
        "type": "CONTINUOUS",
        "low": -5.12,
        "high": 2.12
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}

ID: 7a9b62682ddee5ac08c93669255954f154b4bae095dc877c2b478b7d20f9b45d

recipe:

{
  "sigopt": {
    "name": "SPHERE",
    "dim": 7,
    "int": [
      0,
      1,
      2,
      3,
      4
    ]
  }
}

specification:

{
  "name": "sigopt/evalset/Sphere(dim=7, int=[0, 1, 2, 3, 4])",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "DISCRETE",
        "low": -5,
        "high": 2
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "DISCRETE",
        "low": -5,
        "high": 2
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p2",
      "range": {
        "type": "DISCRETE",
        "low": -5,
        "high": 2
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p3",
      "range": {
        "type": "DISCRETE",
        "low": -5,
        "high": 2
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p4",
      "range": {
        "type": "DISCRETE",
        "low": -5,
        "high": 2
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p5",
      "range": {
        "type": "CONTINUOUS",
        "low": -5.12,
        "high": 2.12
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p6",
      "range": {
        "type": "CONTINUOUS",
        "low": -5.12,
        "high": 2.12
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}

ID: 847114b901d551887b3bb6e0686324852e2cc8c4fe1ac75e8ff018da13b1023e

recipe:

{
  "sigopt": {
    "name": "STYBLINSKI_TANG",
    "dim": 5
  }
}

specification:

{
  "name": "sigopt/evalset/StyblinskiTang(dim=5)",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "CONTINUOUS",
        "low": -5.0,
        "high": 5.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": -5.0,
        "high": 5.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p2",
      "range": {
        "type": "CONTINUOUS",
        "low": -5.0,
        "high": 5.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p3",
      "range": {
        "type": "CONTINUOUS",
        "low": -5.0,
        "high": 5.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p4",
      "range": {
        "type": "CONTINUOUS",
        "low": -5.0,
        "high": 5.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}

ID: 15ca748208253e4a2e35adf06a8f7feb49188210b02e1cd08702d54c9267a4f6

recipe:

{
  "sigopt": {
    "name": "TRID",
    "dim": 6
  }
}

specification:

{
  "name": "sigopt/evalset/Trid(dim=6)",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 20.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 20.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p2",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 20.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p3",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 20.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p4",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 20.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p5",
      "range": {
        "type": "CONTINUOUS",
        "low": 0.0,
        "high": 20.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}

ID: 458dd11a95505ddb3a8d96d492dbc35b0de444d84fe0bf3fef739d02baeaf425

recipe:

{
  "sigopt": {
    "name": "TRIPOD",
    "dim": 2
  }
}

specification:

{
  "name": "sigopt/evalset/Tripod(dim=2)",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "CONTINUOUS",
        "low": -100.0,
        "high": 100.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": -100.0,
        "high": 100.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}

ID: 6396ba82fa5de7995ce66e0eed9448e5882508432c8b324d52c5f11a79f16a78

recipe:

{
  "sigopt": {
    "name": "WEIERSTRASS",
    "dim": 3
  }
}

specification:

{
  "name": "sigopt/evalset/Weierstrass(dim=3)",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "CONTINUOUS",
        "low": -0.5,
        "high": 0.2
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": -0.5,
        "high": 0.2
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p2",
      "range": {
        "type": "CONTINUOUS",
        "low": -0.5,
        "high": 0.2
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}

ID: 099c4ef27c777d65a669d88cd599c9e679f956bd04859ed85d2c9403ec82200c

recipe:

{
  "sigopt": {
    "name": "XOR",
    "dim": 9
  }
}

specification:

{
  "name": "sigopt/evalset/Xor(dim=9)",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "CONTINUOUS",
        "low": -1.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": -1.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p2",
      "range": {
        "type": "CONTINUOUS",
        "low": -1.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p3",
      "range": {
        "type": "CONTINUOUS",
        "low": -1.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p4",
      "range": {
        "type": "CONTINUOUS",
        "low": -1.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p5",
      "range": {
        "type": "CONTINUOUS",
        "low": -1.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p6",
      "range": {
        "type": "CONTINUOUS",
        "low": -1.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p7",
      "range": {
        "type": "CONTINUOUS",
        "low": -1.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p8",
      "range": {
        "type": "CONTINUOUS",
        "low": -1.0,
        "high": 1.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}

ID: a0d7aa2cea146419075791ef31a3e4b50dcefbca3e9b2c52d766f0b000607e67

recipe:

{
  "sigopt": {
    "name": "YAO_LIU",
    "dim": 5
  }
}

specification:

{
  "name": "sigopt/evalset/YaoLiu(dim=5)",
  "attrs": {
    "github": "https://github.com/sigopt/evalset",
    "paper": "Dewancker, Ian, et al. \"A strategy for ranking optimization methods using multiple criteria.\" Workshop on Automatic Machine Learning. 2016.",
    "version": "kurobako_problems=0.1.10"
  },
  "params_domain": [
    {
      "name": "p0",
      "range": {
        "type": "CONTINUOUS",
        "low": -5.12,
        "high": 2.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p1",
      "range": {
        "type": "CONTINUOUS",
        "low": -5.12,
        "high": 2.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p2",
      "range": {
        "type": "CONTINUOUS",
        "low": -5.12,
        "high": 2.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p3",
      "range": {
        "type": "CONTINUOUS",
        "low": -5.12,
        "high": 2.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    },
    {
      "name": "p4",
      "range": {
        "type": "CONTINUOUS",
        "low": -5.12,
        "high": 2.0
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "values_domain": [
    {
      "name": "Objective Value",
      "range": {
        "type": "CONTINUOUS"
      },
      "distribution": "UNIFORM",
      "constraint": null
    }
  ],
  "steps": 1
}

Studies

ID: fbc2ddeb5d704422b08bce6880cb7846605a47643fb54981497602f9ac841d8a

ID: ba15b8dac88bcacef7d931cbbc0e78c34cfbbd1dee733ab492eb02164d602544

ID: 266df7c20ab06d059b7148e715d6114b3b999a99a72b20cf50743d6a050f284a

ID: c53a30cf85d11e6f300fd7a542807a272713a199726baacf2dced62d9864c4e7

ID: 62f6dd2b2d5f7230ad8e9414f3cb5c6b9b1e95e6eb0fadc08ce7bf5d597033c2

ID: 718b5e99f75ee31c2e31da79af467600fb753bb14bbac429ac41fab12f3ebaaa

ID: 6bba607f548c2dba45f37c50ee76c66361ccc86e294f8efcaf1098b401dc96b5

ID: ce060de815ae7f252a2ad4fb0b897acc32c89c3ca71b241d869f50851ed8acf1

ID: 2309623e53c7582bf49fcbba8ce8ccf8043730725d8537a8717ccb1ae9b7f061

ID: ca345964f1864c0c20a02638655c9eaf3ff65f5c490c81ed23cdd208ee45d2b8

ID: c3e387959b4b0679916f6468ec6f8ff15692f6240a75d5735ce6c762f0452e96

ID: 864721f31b1c405aec41d8b96455aa16bf6c17f875d7ded242798d3b47a20f66

ID: f95ae1a5b1536be196364428c452ca6deb4eb0f94bb7f9c591df983f922e8c39

ID: 4e772b3309d174340c7086d87533f4d527269b26814d94fdc2d7b494583a226c

ID: d7670eedc4a9ba469ebb8a9e552fe392d331ca2241d11892f48f1d5fce4365d8

ID: 0cfbe87abeab8c28a03a6f1b4f3f04c7d5d885ec919cfa82e98686e38ec357bc

ID: 5edee370b13ca497831aba9541e67c40963dc92568c6b31664320cc37d9f416f

ID: 64558e857bab80a017c8c4c7fd8b4d09f57a8c29c0a46a5a5519e6161ff8070b

ID: 72ab536fb038676aaf065c9e1e4e65a4ab6530aa11134b823dd88bbbfa1deebb

ID: 29db2d0a0b10e6e1f45e270f9ecec8443b58ef7c9557398e1afe0365819b6537

ID: fdc6266cb554357582d89eaabaff61afda2033708b6afe2af41ef7deee03ccfb

ID: 1af22482a0c849a5ab996bc5811347021d5a3352d14a4f50b80b43ae7fa54148

ID: e2fcf7160f1f004ae3bf3d08427bac659d306dd078a7942f26f9b71d45f9476a

ID: 66cafdadd66fa95cb1091980b4508e81ef165e42538acb3db9e560d3d05ce48c

ID: b5087592050024f2d1b5fe2b584a37009e1bb14062943987a8bd892c92e00689

ID: dd65dd8919b645f06649fd4f3b4354a8ed1f98ebcee25c9f24ba1d1a6d1f764f

ID: 026cf006db02205edafefa0373d35c3d21f9bf5f73aaf9b0140ad8b7ea667dac

ID: d375e2076e80276b5f2e1e08767e8bc581c22d9249671af3301934a309dd08f9

ID: 9ed0c9fa5fa1212cf8363517613dead2b412c6255fb4df0c4d577dee87a67dad

ID: 4dcf232ef87b9334534d5e82a45676ac37ff6ad22a7a5e7ec8c955e19aa243e4

ID: f4d9d6052823cc44a52b6309b4dfdcf391b9d2118644e934cdead54394327196

ID: c4920d5bb8f157889bf1fd452f440bf027e50b9a9a2462b243f66f6981f25aa2

ID: 69425670f741004875a1b3cfa90d2b57e0432e009214465d2f8172edacd8f79c

ID: 1eb90774e8680c24522fd46f93f5ae39dbff60ca5759a94a988e97ef89449a6b

ID: ea03ac7ecf4c4cc8ed9913e359db54d805dcee67fbc0b9d8bca723f8c8839856

ID: 306fc33c8ef4f94b223a850316c6f59551bddb5f05fed877d341d712c6d4a683

ID: ad9e6c1993a0232118d33bbb31a712e56766dc308ff424744313fdd729cc7987

ID: f64db0f5f225d9dd15d19bcf8fe0f3c9f5184fd15926e4efc686a7194f85f005

ID: 08b8555de27d6e481943fe427ca3bd548e3febaf37311680a7b272c70e3710c2

ID: bd31ea059a308f9daca40989c25139a56dae4a077c87f63003514c10fff604d5

ID: 43fe49690322ba402c8512d02278478019be3b542a3a8cb76ce7d828c9cc8b74

ID: e8f7b9aca5c3129c5d48e1cb0c9aefd7bb8637f565070276f45ee523e47d48dc

ID: a7be5cdfec8412e2be11fd6d074af5adc0fd65102742e24b02347d49afb5ab69

ID: 4b7f5baef55e5881f15fae9f06ccb9e6b41fb0847ea63ef038d4ad67cd22dad5

ID: 37797eb69370ee72e477422ed1963680255db0d64419937695766041fb0765e2

ID: acce21aa5529dab14f15c43c757d89ec9fb45c2d4dec4eee376a046047e32b8c

ID: 0d891783e7095abbccb7482be7fa62e6895784b943f1c0d14f0be4fe77178bff

ID: 5915ad3448a2a2ab278194a439856b15a49c27ee5b187ef42f9c4200e7782171

ID: 882afc4f976f3805433954c5ab29410fe979ee8a44342e5c62a115ae1af01bf3

ID: 2243e1792beca103bc02555253233e6233b72c418dac2348a37761bf802f994e

ID: 78c00be95fb6dd5e1784950fd52c80cb58e49dfa6b63ffea0a49a9a7ba314461

ID: 0737bbed2e55bfb529e5a2b9a6bbb71946c3f433294ec0a154769426826148ec

ID: 20b0e223a47eb7b1925d3cec82659ee9489d03be3fd99e4ae07dd48d80a056c9

ID: c54cb9323c118f67c087054e7f5dfa2b9a50f796f7cb8d12203fe3664512cf74

ID: e3a02959393250bbfb7dcca21ec305ecc1969b0835e9942fd24deda98d98302a

ID: 85259cf4fb9177475d5a6228f82ced84c6074accaccee41fb1f45c1d1c0e5de8

ID: a6eb383b7fd8da1bd614a548a3666c76600e7bdd8afb3d67d436c5ff6ad6d24d

ID: 60b4a33d8fed5653aac44cec59c1f37310378f30660d7563944b89389ee96957

ID: 5df8a6762384448d08b062d3307ad01f85855509008ea8935a82f2a0f1a9a5c9

ID: 72028594becc17af1f68e37512930c5af2bf5930129450b07b836b32b1adcae8

ID: c643a2192933fcf05692233098dccab54405faa23830adf3310cb2431a10359c

ID: d9b64d593a07ce986390ae55de660fb982e4ad9f879029ee5af34c231a22a9a0

ID: 317c2fafbadb5979c56e237b46b4b11b7fdab4ed72caf7084d0551969c4fc3f3

ID: f23036896292fe1b2573a8ae11720e63e4f619aa9985f92bb11d02295e5f99c9

ID: d7e1ee134d37455ff3eca3d41b4d83dbca364e5026a60c36e73c71e9c9da9621

ID: ab6073ebc902618a8a025fd5e668746b59bb853f533a3cdbd7e417039fe5528c

ID: 69289af9474d8811e36c078810894fdbfe71eec5e64d1e69bd61b59f2c5a4b6d

ID: 62dbb33e0bf3b3cf74c3699625f91b49f46186644d0f09bc7d67bd7b0201a21f

ID: 49338d0a7613afce6f09a487ea78dfb0dc87fd1dcf18aa76adf78433e03cd3dc

ID: 5bd4135f33cbec499f330f86ab5453360b13b85bf6d79913688bedd606b608b8

ID: 93b128d2407b4388e51ebe19e9eadc07443bab174bf60e484ccbe66fe6044316

ID: ccb569defd50efee4eafa191d836ac577a1a0c1a655987faaa00a503abde74ea

ID: 8f1ec9242c699bf1a5e2dbc22c861c16610f08945f7b0acdd8dfc5408ec2fd84

ID: 45e74efb75dc734c4ac0198a47c0155bdbebca05b837c09482f04d2d51905f75

ID: f3b041e8630d5e7f5625f2a1767ec7acd8f5d238fb85218e7d2640dac471f6b3

ID: 11f2f5fbdc24ea60fad3d5c285c93856b35142e4b9454d1d0dae668dd44b879f

ID: c06a343948dfbc04a30e0370894e7d603928d32a73465990a618fd45db78c6bf

ID: 38ad73a9f2a6fbbec8cbd3a2dcbd2ec704c3dc938dd2327cf8bc8ffab84d88b1

ID: 629dcac5341c2fd9fc160fca4a1739f0caac080cd2483223e1edd2fd91387dd4

ID: e49c40b858250805da5a2c737998902974e595b3ed7bec471383b27bd463744b

ID: 1cfe6da4632231dec7f8f3281d600a9453da184c045f04d9ff79aca6b90154ae

ID: 21f47fa1d011900fcd818ec3e529cf71b0e92a491b8fee271db18a52f34d3d3c

ID: 366f5787b13521ee799bc560d165b88ab919ee584084ea6ef722e40c006befa7

ID: 8aa911aade6af58b4d3c21f721f32d5f3139ecac108f63ef6c4194f495c28f9a

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment