Visualizations of Mittelmann benchmarks

Interactive plots showing pairwise time differences for every instance and every solver

Interactive charts comparing the results of Hans Mittelmann’s benchmarks. Each solver can be selected to show pairwise running time factors for every other solver in the respective benchmark. These plots should make browsing the results easier. Please let me know if you have a question or if there is an error.

Benchmark of Simplex LP solvers (21 Jul 2021)

Choose base solver for comparison:

solver score solved
COPT-2.0.0 1.00 100%
Gurobi-9.1.2 1.50 100%
MindOpt-0.14.0 1.77 100%
CLP-1.17.6 6.09 100%
MOSEK-9.2.47 12.60 95%
Google-GLOP 19.60 80%
SOPLEX-5.0.2 31.60 90%
MATLAB-R2020b 37.30 80%
GLPK-5.0 103.00 72%

older benchmarks:


Benchmark of Barrier LP solvers (21 Jul 2021)

Choose base solver for comparison:

solver score solved
COPT-2.0.0 1.00 100%
Gurobi-9.1.2 1.56 100%
MindOpt-0.14.0 1.96 100%
MOSEK-9.2.47 3.94 98%
KNITRO-12.4.0 13.70 87%
Tulip-0.8.0 45.20 78%
MATLAB-R2020b 47.00 78%
CLP-1.17.6 68.00 83%

older benchmarks:


Large Network-LP Benchmark (commercial vs free) (21 Jul 2021)

Choose base solver for comparison:

solver score solved
COPT-2.0.0 1.00 100%
Gurobi-9.1.2 1.44 100%
Clp-1.17.6 4.25 100%
MATLAB-R2020b 14.70 100%
QSopt-1.01 61.90 70%
SOPLEX-5.0.2 94.30 60%
MOSEK-9.2.47 108.00 80%

older benchmarks:


The MIPLIB2017 Benchmark Instances (4 Jun 2021)

Choose base solver for comparison:

solver score solved
Gurobi-9.1.0 1.00 84%
COPT-2.0.1 4.19 55%
SCIP/spx-7.0.0 4.48 52%
CBC-2.10.5 8.59 37%
MATLAB-2020a 13.50 26%
GLPK-5.0 20.50 10%
LP_SOLVE-5.5.2 21.70 8%

older benchmarks:


The MIPLIB2017 Benchmark Instances - 8 threads (4 Jun 2021)

Choose base solver for comparison:

solver score solved
Gurobi-9.1.0 1.00 92%
COPT-2.0.1 5.17 65%
FSCIP/spx-7.0.0 7.30 57%
CBC-2.10.5 11.80 41%

older benchmarks:


Mixed-integer SOCP Benchmark (16 Feb 2021)

Choose base solver for comparison:

solver score solved
Gurobi-9.1.1 1.00 100%
MOSEK-9.2.37 12.80 68%
SCIP-7.0.0 46.80 62%

older benchmarks:


Binary Non-Convex QPLIB Benchmark (27 May 2021)

Choose base solver for comparison:

solver score solved
Gurobi-9.1.2 1.00 99%
Baron-21.1.13 13.40 49%
OCTERACT-2.1.0 19.90 45%
FSCIP-7.0.0 37.60 40%
SCIP-7.0.0 44.30 30%
ANTIGONE-1.1 53.30 27%
MINOTAUR-0.2.1 71.10 8%

older benchmarks:


Discrete Non-Convex QPLIB Benchmark (non-binary) (27 Jan 2021)

Choose base solver for comparison:

solver score solved
Gurobi-9.1.0 1.00 87%
FSCIP-7.0.0 10.50 42%
SCIP-7.0.0 17.40 39%
Baron-21.1.13 22.60 36%
ANTIGONE-1.1 25.50 38%
OCTERACT-1.07.29 48.30 11%
COUENNE-0.5 57.40 11%
MINOTAUR-0.2.1 66.10 5%

older benchmarks:


Continuous Non-Convex QPLIB Benchmark (2 Dec 2020)

Choose base solver for comparison:

solver score solved
GUROBI-9.1.0 1.00 60%
ANTIGONE-1.1 4.70 50%
Baron-21.1.13 5.56 43%
OCTERACT-1.07.29 7.27 28%
RAPOSA-2.0.0 10.60 14%
SCIP-7.0.0 10.90 21%
COUENNE-0.5 13.00 14%
MINOTAUR-0.2.1 13.60 10%

older benchmarks:


Convex Discrete QPLIB Benchmark (23 Jan 2021)

Choose base solver for comparison:

solver score solved
GUROBI-9.1.0 1.00 74%
Baron-21.1.13 5.42 55%
MOSEK-9.2.29 10.20 42%
SCIP-7.0.0 12.50 35%
Bonmin-1.8.7 15.60 32%
KNITRO-12.3.0 16.00 29%
OCTERACT-1.07.29 22.20 32%
MINOTAUR-0.2.1 25.80 35%
ANTIGONE-1.1 41.00 6%
Shot-1.0 48.00 3%

older benchmarks:


Convex Continuous QPLIB Benchmark (13 Dec 2020)

Choose base solver for comparison:

solver score solved
MOSEK-9.2.29 1.00 94%
Gurobi-9.1.0 1.27 91%
KNITRO-12.3.0 1.74 88%
IPOPT-3.13.0 3.61 88%

older benchmarks: