Skip to content

Instantly share code, notes, and snippets.

@denis-bz
Last active September 5, 2018 16:41
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save denis-bz/db7cf21517a6829a287249441032d6b0 to your computer and use it in GitHub Desktop.
Save denis-bz/db7cf21517a6829a287249441032d6b0 to your computer and use it in GitHub Desktop.
bump.py: Keane's hard-to-optimize function, constrained-random 2018-09-05 Sep

bump2: an optimization problem from A.J.Keane, https://www.southampton.ac.uk/~ajk/bump.html .

Keywords: optimization, test case, random search, python

Problems in mathematical optimization have several different aspects:

  • contest: whose program reaches minimum __ in cpu time __ and time-to-understand __
  • learning how to explore particular function terrains
  • learning how to constrain.

bump.py wins no prize for best minima nor for speed, but perhaps for time-to-understand.

Lessons learned (if any):

  • random search can do pretty well, if properly constrained
  • the shapes of the curves in 10d and 20d are almost identical. (Shapes, back-of-the-envelope sketches, are important !) Todo: learn a bit about multigrid methods ...

Comments welcome, real problems welcome.

cheers
-- denis

#!/usr/bin/env python2
""" bump2 optimization problem
from Keane, https://www.southampton.ac.uk/~ajk/bump2.html 1994
MAXimize
{ abs ( sum i=1 to n { cos^4 ( x_i ) }
- 2 prod i=1 to n { cos^2 ( x_i ) }
) }
/ { sqrt { sum i=1 to n { i x_i^2 } } }
constraints:
0 < x_i < 10
av x_i < 7.5
prod x_i > .75
n=20: 0.76 ~ 1000 n fevals (derivs ?)
What's here:
bump2(x) = - Keane^2 -- optimizers don't like sharp corners, sqrt
random_bumpx( nrand, dim ) -- random x with product .75 etc.
run this.py dim=4 nrand=1e4: .622 .621 ...
run this.py dim=20 nrand=1e6: .725 .725 ...
nrand=1e7: .745 .740 ... 7 minutes
Why ? Get an overview of rough terrain, *before* zooming in
and fun with constraints:
notice that if xi < xj, swapping them decreases the denominator
while leaving the numerator unchanged; so, constrain x0 >= x1 >= x2 ...
Vanaret et al. "Certified Global Minima for a Benchmark of Difficult Optimization Problems" 2014
code not on the web ?
x = np.r_[ 3.065318, 1.531047, 0.405617, 0.393987 ]
f2 = abs( bump2( x ))
print "bump2( Vanaret %s ): %.3g = sqrt %.3g" % ( # .622 = sqrt .387
x, np.sqrt(f2), f2 )
"""
from __future__ import division
import numpy as np
from numpy import cos, pi, prod, sin, sqrt, sum
#...............................................................................
def bump2( x ):
""" Keane's bump function squared, - to minimize
https://www.southampton.ac.uk/~ajk/bump2.html 1994
"""
x = np.asarray(x)
n = len(x)
cos2 = cos(x) ** 2
num2 = (sum( cos2**2 ) - 2 * prod( cos2 )) ** 2
denom2 = np.arange( 1., n+1 ) .dot( x**2 )
return - num2 / denom2 # - to minimize
def random_bumpx( nrand, n ):
""" -> nrand x n random for bump2: prod .75, trim, sort down """
X = np.random.uniform( high=2, size=[int(nrand), n] ).astype( np.float32 )
prods = X.prod( axis=1 )
scale = (.75 / prods) ** (1/n)
X *= scale[:,np.newaxis] # scale rows
rowavs = X.mean( axis=1 )
rowmax = X.max( axis=1 )
X = X[ (rowavs < 7.5) & (rowmax < 10) ] # trim < 1 %
X = np.sort( X, axis=1 )[:,::-1] # sort down
return X
__version__ = "2018-08-24 Aug denis-bz-py t-online de"
#...............................................................................
if __name__ == "__main__": # standalone test
import sys
np.set_printoptions( threshold=20, edgeitems=10, linewidth=140,
formatter = dict( float = lambda x: "%.3g" % x )) # float arrays %.3g
print "\n", 80 * "="
dim = 4
nrand = 1e4
seed = 3
tag = "" # top few > tag.nptxt
# to change these params, run this.py a=1 b=None c='"str"' ... in sh or ipython
for arg in sys.argv[1:]:
exec( arg ) # name not defined error: forgot '' ?
np.random.seed( seed )
if not tag:
tag = "%dd-%g-bump" % (dim, nrand)
#...........................................................................
Xrand = random_bumpx( nrand, dim )
info = "bump2: dim %d nrand %g - %d %.2g bytes " % (
dim, nrand, nrand - len(Xrand), Xrand.nbytes )
print info
print "average Xrand:", Xrand.mean( axis=0 )
#...........................................................................
f2 = np.fabs( map( bump2, Xrand )) # [bump2(row0), bump2(row1) ...]
f = np.sqrt( f2 )
jsort = f.argsort()[::-1] [:10]
print "# j f x "
for j in jsort:
x = Xrand[j]
print "%8d %-8.3g %s " % (
j, f[j], x )
# save the top few, plot-bump.py --
fx = np.c_[ f[jsort], Xrand[jsort] ]
out = tag + ".nptxt"
print "\nsaving", out
np.savetxt( out, fx, fmt="%-4.3g", delimiter="\t",
header = "f x " + info )
# from: ../bump.py dim=20 nrand=1e7 tag="20d-1e7-bump" nrand=1e7
# 23 Aug 2018 18:32 in ~bz/py/opt/testfuncs/bump Denis-iMac 10.10.5
versions: numpy 1.15.1 scipy 1.1.0 python 2.7.15 mac 10.10.5
================================================================================
bump2: dim 20 nleft 9.99998e+06 nrand 1e+07 8e+08 bytes
average Xrand: [2.55 2.42 2.3 2.19 2.08 1.97 1.84 1.67 1.51 1.35 1.21 1.1 1.01 0.883 0.734 0.602 0.484 0.357 0.234 0.113]
# j f x
7389844 0.745 [3.37 3.18 3.06 3.01 2.94 2.86 2.75 1.89 0.782 0.699 0.685 0.631 0.63 0.597 0.524 0.388 0.358 0.332 0.297 0.273]
8719480 0.74 [3.39 3.27 3.26 3.23 2.97 2.89 2.83 2.34 1.71 0.64 0.636 0.535 0.522 0.473 0.471 0.401 0.343 0.273 0.264 0.264]
7583358 0.74 [3.2 3.2 3.01 2.91 2.88 2.82 2.54 1.66 1.47 0.716 0.608 0.573 0.556 0.506 0.463 0.44 0.404 0.329 0.317 0.275]
6869453 0.737 [3.22 3.2 3.11 3.1 3.04 3.01 2.75 2.58 1.36 1.13 0.663 0.627 0.565 0.519 0.383 0.373 0.357 0.297 0.245 0.168]
7106962 0.735 [3.24 3.18 3.07 3.06 2.99 2.97 2.97 2.91 1.88 1.11 0.865 0.693 0.47 0.463 0.407 0.331 0.305 0.237 0.198 0.191]
6688350 0.732 [3.35 3.15 3.13 3.12 3.08 2.97 2.96 1.97 1.38 0.883 0.683 0.646 0.582 0.58 0.414 0.34 0.33 0.313 0.241 0.214]
6929803 0.73 [3.25 3.25 3.12 3.05 3.02 2.92 2.58 2.28 1.01 0.803 0.766 0.569 0.514 0.508 0.504 0.45 0.427 0.368 0.218 0.2]
5259494 0.729 [3.25 3.21 3.12 3.11 3.01 2.81 2.58 2.12 1.23 0.787 0.724 0.643 0.518 0.51 0.501 0.376 0.364 0.297 0.277 0.24]
6681512 0.726 [3.12 3.02 3 2.94 2.92 2.86 2.85 2.84 2.34 1.44 1.34 0.591 0.364 0.361 0.339 0.321 0.291 0.241 0.238 0.207]
749296 0.725 [3.22 3.19 2.95 2.85 2.78 2.75 2.75 2.74 1.54 1 0.784 0.581 0.578 0.436 0.429 0.391 0.376 0.322 0.205 0.204]
saving 20d-1e7-bump.nptxt
426.30 real 425.10 user 1.49 sys
# f x bump2: dim 20 nleft 9.99998e+06 nrand 1e+07 8e+08 bytes
0.745 3.37 3.18 3.06 3.01 2.94 2.86 2.75 1.89 0.782 0.699 0.685 0.631 0.63 0.597 0.524 0.388 0.358 0.332 0.297 0.273
0.74 3.39 3.27 3.26 3.23 2.97 2.89 2.83 2.34 1.71 0.64 0.636 0.535 0.522 0.473 0.471 0.401 0.343 0.273 0.264 0.264
0.74 3.2 3.2 3.01 2.91 2.88 2.82 2.54 1.66 1.47 0.716 0.608 0.573 0.556 0.506 0.463 0.44 0.404 0.329 0.317 0.275
0.737 3.22 3.2 3.11 3.1 3.04 3.01 2.75 2.58 1.36 1.13 0.663 0.627 0.565 0.519 0.383 0.373 0.357 0.297 0.245 0.168
0.735 3.24 3.18 3.07 3.06 2.99 2.97 2.97 2.91 1.88 1.11 0.865 0.693 0.47 0.463 0.407 0.331 0.305 0.237 0.198 0.191
0.732 3.35 3.15 3.13 3.12 3.08 2.97 2.96 1.97 1.38 0.883 0.683 0.646 0.582 0.58 0.414 0.34 0.33 0.313 0.241 0.214
0.73 3.25 3.25 3.12 3.05 3.02 2.92 2.58 2.28 1.01 0.803 0.766 0.569 0.514 0.508 0.504 0.45 0.427 0.368 0.218 0.2
0.729 3.25 3.21 3.12 3.11 3.01 2.81 2.58 2.12 1.23 0.787 0.724 0.643 0.518 0.51 0.501 0.376 0.364 0.297 0.277 0.24
0.726 3.12 3.02 3 2.94 2.92 2.86 2.85 2.84 2.34 1.44 1.34 0.591 0.364 0.361 0.339 0.321 0.291 0.241 0.238 0.207
0.725 3.22 3.19 2.95 2.85 2.78 2.75 2.75 2.74 1.54 1 0.784 0.581 0.578 0.436 0.429 0.391 0.376 0.322 0.205 0.204
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment