Visualizations of Mittelmann benchmarks

Interactive plots showing pairwise time differences for every instance and every solver

Interactive charts comparing the results of Hans Mittelmann’s benchmarks. Each solver can be selected to show pairwise running time factors for every other solver in the respective benchmark. These plots should make browsing the results easier. Please let me know if you have a question or if there is an error.

Benchmark of Simplex LP solvers (18 Jan 2021)

Choose base solver for comparison:

solver score solved
COPT-1.4.7 1.00 100%
MindOpt-0.11.0 1.28 100%
Gurobi-9.1.0 1.60 100%
CLP-1.17.6 6.43 100%
MOSEK-9.2.20 13.70 95%
HiGHS-1.0.0 24.30 82%
Google-GLOP 26.80 85%
MATLAB-R2020a 29.70 80%
SOPLEX-5.0.2 31.70 90%
GLPK-5.0 104.00 65%

older benchmarks:


Benchmark of Barrier LP solvers (28 Dec 2020)

Choose base solver for comparison:

solver score solved
Gurobi-9.1.0 1.00 100%
COPT-1.4.7 1.58 100%
MindOpt-0.11.0 2.00 98%
MOSEK-9.2.21 2.34 98%
KNITRO-12.3.0 12.40 85%
Tulip-0.7.0 29.70 80%
MATLAB-R2020a 31.10 76%
CLP-1.17.6 46.70 83%

older benchmarks:


Large Network-LP Benchmark (commercial vs free) (29 Dec 2020)

Choose base solver for comparison:

solver score solved
COPT-1.4.7 1.00 100%
Gurobi-9.1.0 1.02 100%
Clp-1.17.6 2.60 100%
MATLAB-R2020b 10.20 90%
QSopt-1.01 38.50 60%
SOPLEX-5.0.0 52.10 60%
MOSEK-9.2.21 60.60 80%

older benchmarks:


The MIPLIB2017 Benchmark Instances (15 Nov 2020)

Choose base solver for comparison:

solver score solved
Gurobi-9.1.0 1.00 84%
SCIP/spx-7.0.0 4.48 52%
CBC-2.10.5 8.59 37%
MATLAB-2020a 13.50 26%
GLPK-4.65 20.50 10%
LP_SOLVE-5.5.2 21.70 8%

older benchmarks:


The MIPLIB2017 Benchmark Instances - 8 threads (15 Nov 2020)

Choose base solver for comparison:

solver score solved
Gurobi-9.1.0 1.00 92%
FSCIP/spx-7.0.0 7.30 57%
CBC-2.10.5 11.80 41%

older benchmarks:


Mixed-integer SOCP Benchmark (10 Nov 2020)

Choose base solver for comparison:

solver score solved
Gurobi-9.1.0 1.00 100%
MOSEK-9.2.10 12.30 68%
SCIP-7.0.0 46.90 62%

older benchmarks:


Binary Non-Convex QPLIB Benchmark (10 Nov 2020)

Choose base solver for comparison:

solver score solved
Gurobi-9.1.0 1.00 98%
Baron-20.10.16 15.00 49%
OCTERACT-1.07.29 20.00 43%
FSCIP-7.0.0 42.00 40%
SCIP-7.0.0 50.00 31%
ANTIGONE-1.1 60.00 27%
MINOTAUR-0.2.1 80.00 8%

older benchmarks:


Discrete Non-Convex QPLIB Benchmark (non-binary) (10 Nov 2020)

Choose base solver for comparison:

solver score solved
Gurobi-9.1.0 1.00 87%
FSCIP-7.0.0 10.50 42%
SCIP-7.0.0 17.40 39%
Baron-20.10.16 24.30 38%
ANTIGONE-1.1 25.50 38%
OCTERACT-1.07.29 48.30 11%
COUENNE-0.5 57.40 11%
MINOTAUR-0.2.1 66.10 5%

older benchmarks:


Continuous Non-Convex QPLIB Benchmark (11 Nov 2020)

Choose base solver for comparison:

solver score solved
GUROBI-9.1.0 1.00 61%
ANTIGONE-1.1 4.68 51%
Baron-20.10.16 5.49 46%
OCTERACT-1.07.29 7.27 28%
SCIP-7.0.0 10.50 21%
COUENNE-0.5 13.00 14%
MINOTAUR-0.2.1 13.60 11%

older benchmarks:


Convex Discrete QPLIB Benchmark (13 Dec 2020)

Choose base solver for comparison:

solver score solved
GUROBI-9.1.0 1.00 74%
Baron-20.10.16 5.54 55%
MOSEK-9.2.29 10.20 42%
SCIP-7.0.0 12.50 35%
Bonmin-1.8.7 15.60 32%
KNITRO-12.3.0 16.00 29%
OCTERACT-1.07.29 22.20 32%
MINOTAUR-0.2.1 25.80 35%
ANTIGONE-1.1 41.00 6%
Shot-1.0 48.00 3%

older benchmarks:


Convex Continuous QPLIB Benchmark (13 Dec 2020)

Choose base solver for comparison:

solver score solved
MOSEK-9.2.29 1.00 94%
Gurobi-9.1.0 1.27 91%
KNITRO-12.3.0 1.74 88%
IPOPT-3.13.0 3.61 88%

older benchmarks: