Visualizations of Mittelmann benchmarks

Interactive plots showing pairwise time differences for every instance and every solver

Interactive charts comparing the results of Hans Mittelmann’s benchmarks. Each solver can be selected to show pairwise running time factors for every other solver in the respective benchmark. These plots should make browsing the results easier. Please let me know if you have a question or if there is an error.

Benchmark of Simplex LP solvers (3 Mar 2021)

Choose base solver for comparison:

solver score solved
COPT-1.6.0 1.00 100%
MindOpt-0.11.0 1.28 100%
Gurobi-9.1.0 1.60 100%
CLP-1.17.6 6.43 100%
MOSEK-9.2.20 13.70 95%
HiGHS-1.0.0 24.30 82%
Google-GLOP 26.80 85%
MATLAB-R2020a 29.70 80%
SOPLEX-5.0.2 31.70 90%
GLPK-5.0 104.00 65%

older benchmarks:


Benchmark of Barrier LP solvers (3 Mar 2021)

Choose base solver for comparison:

solver score solved
Gurobi-9.1.0 1.00 100%
COPT-1.6.0 1.09 100%
MindOpt-0.11.0 2.00 98%
MOSEK-9.2.21 2.34 98%
KNITRO-12.3.0 12.40 85%
Tulip-0.7.0 29.70 80%
MATLAB-R2020a 31.10 76%
CLP-1.17.6 46.70 83%

older benchmarks:


Large Network-LP Benchmark (commercial vs free) (3 Mar 2021)

Choose base solver for comparison:

solver score solved
COPT-1.6.0 1.00 100%
Gurobi-9.1.0 1.39 100%
Clp-1.17.6 3.62 100%
MATLAB-R2020b 14.20 90%
QSopt-1.01 53.70 60%
SOPLEX-5.0.2 72.90 60%
MOSEK-9.2.21 84.50 80%

older benchmarks:


The MIPLIB2017 Benchmark Instances (5 Feb 2021)

Choose base solver for comparison:

solver score solved
Gurobi-9.1.0 1.00 84%
SCIP/spx-7.0.0 4.48 52%
CBC-2.10.5 8.59 37%
MATLAB-2020a 13.50 26%
GLPK-5.0 20.50 10%
LP_SOLVE-5.5.2 21.70 8%

older benchmarks:


The MIPLIB2017 Benchmark Instances - 8 threads (5 Feb 2021)

Choose base solver for comparison:

solver score solved
Gurobi-9.1.0 1.00 92%
FSCIP/spx-7.0.0 7.30 57%
CBC-2.10.5 11.80 41%

older benchmarks:


Mixed-integer SOCP Benchmark (16 Feb 2021)

Choose base solver for comparison:

solver score solved
Gurobi-9.1.1 1.00 100%
MOSEK-9.2.37 12.80 68%
SCIP-7.0.0 46.80 62%

older benchmarks:


Binary Non-Convex QPLIB Benchmark (25 Jan 2021)

Choose base solver for comparison:

solver score solved
Gurobi-9.1.0 1.00 98%
Baron-21.1.13 14.50 50%
OCTERACT-1.07.29 19.80 43%
FSCIP-7.0.0 41.90 40%
SCIP-7.0.0 49.50 31%
ANTIGONE-1.1 59.80 27%
MINOTAUR-0.2.1 80.30 8%

older benchmarks:


Discrete Non-Convex QPLIB Benchmark (non-binary) (27 Jan 2021)

Choose base solver for comparison:

solver score solved
Gurobi-9.1.0 1.00 87%
FSCIP-7.0.0 10.50 42%
SCIP-7.0.0 17.40 39%
Baron-21.1.13 22.60 36%
ANTIGONE-1.1 25.50 38%
OCTERACT-1.07.29 48.30 11%
COUENNE-0.5 57.40 11%
MINOTAUR-0.2.1 66.10 5%

older benchmarks:


Continuous Non-Convex QPLIB Benchmark (2 Dec 2020)

Choose base solver for comparison:

solver score solved
GUROBI-9.1.0 1.00 60%
ANTIGONE-1.1 4.70 50%
Baron-21.1.13 5.56 43%
OCTERACT-1.07.29 7.27 28%
RAPOSA-2.0.0 10.60 14%
SCIP-7.0.0 10.90 21%
COUENNE-0.5 13.00 14%
MINOTAUR-0.2.1 13.60 10%

older benchmarks:


Convex Discrete QPLIB Benchmark (23 Jan 2021)

Choose base solver for comparison:

solver score solved
GUROBI-9.1.0 1.00 74%
Baron-21.1.13 5.42 55%
MOSEK-9.2.29 10.20 42%
SCIP-7.0.0 12.50 35%
Bonmin-1.8.7 15.60 32%
KNITRO-12.3.0 16.00 29%
OCTERACT-1.07.29 22.20 32%
MINOTAUR-0.2.1 25.80 35%
ANTIGONE-1.1 41.00 6%
Shot-1.0 48.00 3%

older benchmarks:


Convex Continuous QPLIB Benchmark (13 Dec 2020)

Choose base solver for comparison:

solver score solved
MOSEK-9.2.29 1.00 94%
Gurobi-9.1.0 1.27 91%
KNITRO-12.3.0 1.74 88%
IPOPT-3.13.0 3.61 88%

older benchmarks: