Navigation Menu

Skip to content

Instantly share code, notes, and snippets.

@denis-bz
Created August 21, 2019 09:50
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save denis-bz/53de8a4dcece2b6dcf87fdfafa9f8cc2 to your computer and use it in GitHub Desktop.
Save denis-bz/53de8a4dcece2b6dcf87fdfafa9f8cc2 to your computer and use it in GitHub Desktop.
Runtimes of GLPK on some Netlib test cases 21 Aug 2019 09:50z

Runtimes of GLPK, the gnu Linear Programming kit, on some Netlib test cases

Keywords: linear programming, LP, GLPK, runtime, test case, Netlib

# 16 Aug 2019 08:34z  ~bz/py/opt/lp/netlib  Denis-iMac 10.10.5
Intel(R) Core(TM) i5-3330S CPU @ 2.70GHz
Apple LLVM version 7.0.2 (clang-700.1.81)
GLPSOL: GLPK LP/MIP Solver, v4.65  (uses only 1 core)

--------------------------------------------------------------------------------
# glpsol-simplex
# problem  rows columns non0  obj  seconds

dfl001     5983  10868   34062   1.12664e+07   30.7   
pilot87    1968   4593   70380   301.71         6.7   
stochfor  15336  14382   55088   -39976.8       4.8   
fit2d        25  10500  129018   -68464.3       3.6   
maros-r7   2152   5288   98334   1.49719e+06    2.4   
fit2p      3000  10525   47284   68464.3        1.9   
80bau3b    1992   9166   19823   987224         0.8   
cycle      1404   2254   14469   -5.22639       0.1   

--------------------------------------------------------------------------------
# glpsol-ip
# problem  rows columns non0  obj  seconds

fit2p      3001  13525   60784   68464.3       131.6   
dfl001     6072  12230   41873   1.12687e+07   37.5  unstable 
fit2d        26  10500  138018   -68464.3      24.3   
maros-r7   3137   9408  151120   1.49719e+06    7.4   
pilot87    2031   4883   73804   301.71         6.0   
stochfor  16676  15695   74004   -39976.8       0.4  unstable 
80bau3b    2263   9799   29063   987224         0.3   
cycle      1904   2857   21322   -4.8844        0.2  unstable 

(Simplex has a "Preprocessing" step which makes problems smaller, in some cases a good deal smaller.)

How to rerun GLPK on these test cases

First look at the shell scripts below, see if they make sense.

  1. download and unpack https://www.gnu.org/software/glpk/.../glpk-4.65.tar.gz
    configure; make; make install
  2. download *.mps.gz from http://www.zib.de/koch/perplex/data/netlib/mps, see curls-netlib-zib. (There are other Netlib repos, some with bit rot.)
  3. run-glpsol-netlib my*.mps
  4. summarize the logfiles with glpsol.awk or the like.

So what ?

  • GLPK ain't bad, on these old test cases.

  • Experts generally say that commercial solvers such as Gurobi are waaay faster than any opensource solver. This wouldn't surprise me, but: data on the table please.

  • Moore's laws (of CPU and memory) are making old test cases irrelevant.

Look at LP in the whole flow, input - optimize - output

GLPK may be good enough for your problem. In real-world optimization, the LP optimizer is only part of a flow, a cycle, a process:

  • input: map a problem to a sea of numbers A b c [...]
  • check that there are no mistakes in this process
  • run the LP engine
  • output: make the solution x[...] understandable for yourself and for others, with plots and talks.

See also

Old runtimes from glpk 4.40: .../glpk-4.65/doc/netlib.txt in the GLPK distribution
scipy/scipy#9783: a plot of some scipy linprog runtimes, log scale
lp_np: numpy <--> GLPK, under my gists

Comments welcome, test cases welcome.

It would be nice to have test problems that are small but difficult: small enough to understand, but with difficult constraints for simplex / for IP. (Can one define "ill-conditioned" for LP problems ? SVD can be used to analyze Ax = b without x >= 0, and construct test cases; is there an analog for LP ?)

cheers
-- denis 19 August 2019

#!/bin/bash
[[ $1 ]] || set `egrep -v '^#' zib-mps.list`
for mps
do
case $mps in
'#*' ) continue ;;
*.mps.gz ) ;;
* ) mps=$mps.mps.gz
esac
curl -O --remote-time --silent --show-error \
http://www.zib.de/koch/perplex/data/netlib/mps/$mps
done
GLPSOL: GLPK LP/MIP Solver, v4.65
Parameter(s) specified in the command line:
--interior --freemps zib/mps/dfl001.mps.gz -w dfl001.sol.gz --log dfl001.glpsollog
Reading problem data from 'zib/mps/dfl001.mps.gz'...
Problem: DFL001
Objective: NIL
6072 rows, 12230 columns, 41873 non-zeros
29923 records were read
One free row was removed
Scaling...
A: min|aij| = 8.333e-02 max|aij| = 2.000e+00 ratio = 2.400e+01
GM: min|aij| = 5.946e-01 max|aij| = 1.682e+00 ratio = 2.828e+00
EQ: min|aij| = 3.536e-01 max|aij| = 1.000e+00 ratio = 2.828e+00
Original LP has 6071 row(s), 12230 column(s), and 35632 non-zero(s)
Working LP has 6084 row(s), 12243 column(s), and 35658 non-zero(s)
Matrix A has 35658 non-zeros
Matrix S = A*A' has 44220 non-zeros (upper triangle)
Approximate minimum degree ordering (AMD)...
Computing Cholesky factorization S = L*L'...
Matrix L has 1566516 non-zeros
Guessing initial point...
Optimization begins...
0: obj = 2.849034741e+12; rpi = 1.5e+02; rdi = 1.4e+01; gap = 1.0e+00
1: obj = 1.813653191e+12; rpi = 8.5e+01; rdi = 1.1e+01; gap = 1.3e+00
2: obj = 9.915087681e+11; rpi = 4.8e+01; rdi = 7.8e+00; gap = 1.9e+00
3: obj = 4.562635627e+11; rpi = 1.9e+01; rdi = 2.7e+00; gap = 4.3e+00
4: obj = 2.064021769e+11; rpi = 4.0e+00; rdi = 6.1e-01; gap = 7.5e+00
5: obj = 1.376725239e+11; rpi = 1.1e+00; rdi = 2.0e-01; gap = 5.7e+00
6: obj = 9.756676591e+10; rpi = 3.1e-01; rdi = 5.4e-02; gap = 3.5e+00
7: obj = 6.131853048e+10; rpi = 9.3e-02; rdi = 1.7e-02; gap = 2.9e+00
8: obj = 2.621102076e+10; rpi = 1.3e-02; rdi = 5.5e-03; gap = 2.9e+00
9: obj = 1.120522177e+10; rpi = 3.0e-03; rdi = 1.6e-03; gap = 2.6e+00
10: obj = 3.590439345e+09; rpi = 5.2e-04; rdi = 5.4e-04; gap = 2.8e+00
11: obj = 1.312539069e+09; rpi = 1.0e-04; rdi = 1.9e-04; gap = 2.9e+00
12: obj = 8.921718386e+08; rpi = 4.8e-05; rdi = 8.4e-05; gap = 2.6e+00
13: obj = 4.947491257e+08; rpi = 1.6e-05; rdi = 3.9e-05; gap = 2.7e+00
14: obj = 2.347262054e+08; rpi = 5.0e-06; rdi = 1.1e-05; gap = 2.3e+00
15: obj = 1.479014740e+08; rpi = 1.8e-06; rdi = 3.9e-06; gap = 2.2e+00
16: obj = 1.026107409e+08; rpi = 9.5e-07; rdi = 1.6e-06; gap = 1.9e+00
17: obj = 6.771506444e+07; rpi = 4.9e-07; rdi = 6.0e-07; gap = 1.7e+00
18: obj = 5.355492894e+07; rpi = 2.8e-07; rdi = 3.2e-07; gap = 1.6e+00
19: obj = 4.013647999e+07; rpi = 1.5e-07; rdi = 2.4e-07; gap = 1.7e+00
20: obj = 3.046719114e+07; rpi = 9.1e-08; rdi = 9.8e-08; gap = 1.3e+00
21: obj = 2.638421718e+07; rpi = 5.8e-08; rdi = 6.2e-08; gap = 1.2e+00
22: obj = 2.083344919e+07; rpi = 4.1e-08; rdi = 2.1e-08; gap = 9.5e-01
23: obj = 1.724992866e+07; rpi = 2.4e-08; rdi = 6.6e-09; gap = 5.9e-01
24: obj = 1.579311399e+07; rpi = 1.6e-08; rdi = 6.8e-10; gap = 4.4e-01
25: obj = 1.389913096e+07; rpi = 8.1e-09; rdi = 4.6e-10; gap = 3.3e-01
26: obj = 1.351850009e+07; rpi = 6.4e-09; rdi = 2.4e-10; gap = 2.7e-01
27: obj = 1.281505695e+07; rpi = 4.3e-09; rdi = 3.3e-11; gap = 1.7e-01
28: obj = 1.252967929e+07; rpi = 3.3e-09; rdi = 6.2e-12; gap = 1.3e-01
29: obj = 1.193881291e+07; rpi = 1.6e-09; rdi = 2.4e-12; gap = 7.3e-02
30: obj = 1.173369995e+07; rpi = 1.0e-09; rdi = 1.1e-12; gap = 5.1e-02
31: obj = 1.152234168e+07; rpi = 4.7e-10; rdi = 3.2e-13; gap = 2.9e-02
32: obj = 1.137772953e+07; rpi = 1.6e-10; rdi = 1.6e-13; gap = 1.5e-02
33: obj = 1.132405470e+07; rpi = 7.6e-11; rdi = 7.6e-14; gap = 7.2e-03
34: obj = 1.129984356e+07; rpi = 6.0e-11; rdi = 6.7e-14; gap = 4.3e-03
35: obj = 1.128393177e+07; rpi = 4.3e-10; rdi = 5.8e-14; gap = 2.4e-03
36: obj = 1.127765016e+07; rpi = 2.5e-10; rdi = 5.3e-14; gap = 1.4e-03
37: obj = 1.127309651e+07; rpi = 1.1e-09; rdi = 5.3e-14; gap = 8.0e-04
38: obj = 1.127006056e+07; rpi = 5.7e-10; rdi = 5.7e-14; gap = 4.3e-04
39: obj = 1.126868877e+07; rpi = 1.3e-09; rdi = 5.2e-14; gap = 2.7e-04
NUMERIC INSTABILITY; SEARCH TERMINATED
Best point 1.126868877e+07 was reached on iteration 39
Time used: 37.5 secs
Memory used: 29.5 Mb (30923972 bytes)
Writing interior-point solution to 'dfl001.sol.gz'...
18310 lines were written
37.71 real 37.65 user 0.04 sys
GLPSOL: GLPK LP/MIP Solver, v4.65
Parameter(s) specified in the command line:
--freemps zib/mps/dfl001.mps.gz -w dfl001.sol.gz --log /dev/null
Reading problem data from 'zib/mps/dfl001.mps.gz'...
Problem: DFL001
Objective: NIL
6072 rows, 12230 columns, 41873 non-zeros
29923 records were read
One free row was removed
GLPK Simplex Optimizer, v4.65
6071 rows, 12230 columns, 35632 non-zeros
Preprocessing...
5983 rows, 10868 columns, 34062 non-zeros
Scaling...
A: min|aij| = 8.333e-02 max|aij| = 2.000e+00 ratio = 2.400e+01
GM: min|aij| = 5.946e-01 max|aij| = 1.682e+00 ratio = 2.828e+00
EQ: min|aij| = 3.536e-01 max|aij| = 1.000e+00 ratio = 2.828e+00
Constructing initial basis...
Size of triangular part is 5948
0: obj = 3.447611499e+10 inf = 3.139e+04 (1071)
Perturbing LP to avoid stalling [1992]...
8536: obj = 5.291724041e+09 inf = 2.023e+03 (290) 53
15853: obj = 4.005236187e+09 inf = 1.423e-05 (0) 44
* 22766: obj = 1.474707780e+07 inf = 1.732e-05 (2573) 41
* 29229: obj = 1.138223816e+07 inf = 8.871e-05 (2750) 46
* 35626: obj = 1.130679317e+07 inf = 1.014e-04 (2198) 45
* 42205: obj = 1.126758004e+07 inf = 9.867e-05 (895) 44
Removing LP perturbation [43170]...
* 43170: obj = 1.126639605e+07 inf = 1.477e-12 (0) 7
OPTIMAL LP SOLUTION FOUND
Time used: 30.7 secs
Memory used: 14.8 Mb (15487537 bytes)
Writing basic solution to 'dfl001.sol.gz'...
18310 lines were written
30.84 real 30.83 user 0.01 sys
#!/bin/bash
# glpsol.awk: summarize *.glpsollog
# --simplex != --interior
# cf. run-glpsol-netlib
# glpks-4.65/doc/netlib.txt -- glpsol 4.40
trap exit INT QUIT TERM
case $1 in
-h | --help )
exec less $0 ;;
esac
[[ $1 ]] ||
set glpsol-simplex glpsol-ip # dir/*.glpsollog
From # date pwd etc.
sysctl -n machdep.cpu.brand_string
clang --version | sed 1q
echo "`glpsol --version | sed 1q` (uses only one core)"
for dir
do
cat <<!
--------------------------------------------------------------------------------
# $dir
# problem rows columns non0 obj seconds
!
#...............................................................................
for log in $dir/*.glpsollog
do
gawk '
/^Problem: / {
name = tolower( $2 )
}
/^[0-9]+ rows,/ { # simplex: Preprocessing e.g. dfl001 6071, 12230 -> 5983, 10868
nr = $1 + 0
nc = $3 + 0
nnz = $5 + 0
}
/^ *[0-9]+: obj / { # ip
obj = $4 + 0
}
/^\* *[0-9]+: obj / { # simplex
obj = $5 + 0
}
/^NUMERIC INSTABILITY/ {
unstable = " unstable"
}
/^Time/ {
sec = $3 + 0
printf "%-8s %6d %6d %7d %-12g %4.1f%s\n", \
name, nr, nc, nnz, obj, sec, unstable
exit
}
' $log
done |
sort -k 6rg
done
#!/bin/bash
help() {
cat <<!
$0 [-opt "" or --interior] [-dir zip/mps] *.mps
runs the GLPK solver glpsol on *.mps
!
exit
}
opt="" # "": simplex
dir=zib/mps # file or $dir/file
trap exit INT QUIT TERM
while [[ $1 == -* ]]
do
case "$1" in
-h | --help )
help ;;
-dir )
dir="$2"
shift 2 ;;
-opt )
opt="$2"
shift 2 ;;
* )
break
esac
done
# default files from http://www.zib.de/koch/perplex/data/netlib/mps/
# see ../netlib/glpsol-netlib.sum glpsol-{ip,simplex}/*.glpsollog
[[ $1 ]] || set -- \
`awk '/^[a-zA-Z]/ { print $1 }' <<\!
# glpsol-simplex
# problem rows columns non0 obj seconds
dfl001 5983 10868 34062 1.12664e+07 30.7
pilot87 1968 4593 70380 301.71 6.7
stochfor 15336 14382 55088 -39976.8 4.8
fit2d 25 10500 129018 -68464.3 3.6
maros-r7 2152 5288 98334 1.49719e+06 2.4
fit2p 3000 10525 47284 68464.3 1.9
80bau3b 1992 9166 19823 987224 0.8
cycle 1404 2254 14469 -5.22639 0.1
!
`
#...............................................................................
for file
do
[[ -f $file ]] || file=$dir/$file.mps.gz # try $dir
[[ -f $file ]] || {
bold "? $file not found"
continue
}
prob=${file##*/}
prob=${prob%%.*} # basename
log=$prob.glpsollog # tty too
tmp=$prob.tmp
cat <<!
--------------------------------------------------------------------------------
glpsol $opt --freemps $file -w $prob.sol.gz --log $log
!
/usr/bin/time \
glpsol $opt --freemps $file -w $prob.sol.gz --log $log 2> $tmp
# --model file.mod --lp file.lp
cat $tmp >> $log
/bin/rm $tmp
done
# mv *.sol.gz *.glpsollog glpsol-simplex/
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment