Visualizations of Mittelmann benchmarks

Interactive plots showing pairwise time differences for every instance and every solver

Interactive charts comparing the results of Hans Mittelmann’s benchmarks. Each solver can be selected to show pairwise running time factors for every other solver in the respective benchmark. These plots should make browsing the results easier. Please let me know if you have a question or if there is an error.

Benchmark of Simplex LP solvers (3 Oct 2021)

Choose base solver for comparison:

solver score solved
MindOpt-0.15.0 1.00 100%
COPT-3.0.0 1.25 100%
Optverse-0.2.1 1.38 98%
Gurobi-9.1.2 2.26 100%
CLP-1.17.6 8.73 92%
HiGHS-1.1.0 20.10 78%
MOSEK-9.2/3 21.10 88%
Google-GLOP 32.60 68%
MATLAB-R2020b 45.70 72%
SOPLEX-5.0.2 47.30 78%
GLPK-5.0 121.00 62%

older benchmarks:


Benchmark of Barrier LP solvers (3 Oct 2021)

Choose base solver for comparison:

solver score solved
COPT-3.0.0 1.00 100%
Gurobi-9.1.2 1.56 100%
MindOpt-0.15.0 2.33 100%
MOSEK-9.2/3 4.71 89%
KNITRO-12.4.0 14.30 79%
HiGHS-1.1.0 15.20 87%
MATLAB-R2020b 36.70 74%
Tulip-0.8.0 55.90 66%
CLP-1.17.6 68.60 74%

older benchmarks:


Large Network-LP Benchmark (commercial vs free) (22 Oct 2021)

Choose base solver for comparison:

solver score solved
COPT-3.0.2 1.00 100%
MindOpt-0.15.1 1.35 100%
Clp-1.17.6 3.04 100%
Gurobi-9.1.2 3.64 100%
HGHS 6.39 100%
MATLAB-R2020b 12.60 100%
QSopt-1.01 22.90 85%
MOSEK-9.3.1 38.60 90%
SOPLEX-5.0.2 41.40 80%

older benchmarks:


The MIPLIB2017 Benchmark Instances (5 Oct 2021)

Choose base solver for comparison:

solver score solved
Gurobi-9.1.0 1.00 84%
COPT-3.0.0 4.19 55%
SCIP/spx-7.0.3.5 4.48 52%
CBC-2.10.5 8.59 37%
MATLAB-2020a 13.50 26%
GLPK-5.0 20.50 10%
LP_SOLVE-5.5.2 21.70 8%

older benchmarks:


The MIPLIB2017 Benchmark Instances - 8 threads (5 Oct 2021)

Choose base solver for comparison:

solver score solved
Gurobi-9.1.0 1.00 94%
COPT-3.0.0 4.52 73%
FSCIP/spx-7.0.0 7.98 61%
SCIP/spx-7.0.3.5 9.60 56%
HiGHS-1.1.1 12.00 56%
CBC-2.10.5 13.30 45%

older benchmarks:


Mixed-integer SOCP Benchmark (3 Sep 2021)

Choose base solver for comparison:

solver score solved
Gurobi-9.1.2 1.00 100%
MOSEK-9.3.1 21.20 68%
SCIP-7.0.3.5 26.20 66%

older benchmarks:


Binary Non-Convex QPLIB Benchmark (23 Aug 2021)

Choose base solver for comparison:

solver score solved
Gurobi-9.1.2 1.00 99%
Baron-21.1.13 13.40 49%
OCTERACT-3.4.1 21.00 42%
FSCIP-7.0.0 37.60 40%
SCIP-7.0.0 44.30 30%
ANTIGONE-1.1 53.30 27%
MINOTAUR-0.2.1 71.10 8%

older benchmarks:


Discrete Non-Convex QPLIB Benchmark (non-binary) (7 Sep 2021)

Choose base solver for comparison:

solver score solved
Gurobi-9.1.0 1.00 85%
SCIP-7.0.3.5 10.80 47%
Baron-21.1.13 20.70 35%
ANTIGONE-1.1 23.30 37%
OCTERACT-3.4.1 26.00 26%
COUENNE-0.5 51.50 10%
MINOTAUR-0.2.1 59.10 5%

older benchmarks:


Continuous Non-Convex QPLIB Benchmark (16 Sep 2021)

Choose base solver for comparison:

solver score solved
GUROBI-9.1.0 1.00 62%
ANTIGONE-1.1 4.82 52%
OCTERACT-3.4.1 5.74 36%
Baron-21.1.13 6.97 43%
RAPOSA-2.0.0 11.00 14%
SCIP-7.0.3.5 11.20 23%
COUENNE-0.5 13.60 14%
MINOTAUR-0.2.1 14.30 11%

older benchmarks:


Convex Discrete QPLIB Benchmark (21 Aug 2021)

Choose base solver for comparison:

solver score solved
GUROBI-9.1.2 1.00 74%
Baron-21.1.13 5.46 55%
MOSEK-9.2.29 10.30 42%
SCIP-7.0.0 12.60 35%
Bonmin-1.8.7 15.70 32%
KNITRO-12.3.0 16.00 29%
MINOTAUR-0.2.1 26.00 35%
OCTERACT-3.4.1 34.40 10%
ANTIGONE-1.1 41.30 6%
Shot-1.0 48.30 3%

older benchmarks:


Convex Continuous QPLIB Benchmark (5 May 2021)

Choose base solver for comparison:

solver score solved
MOSEK-9.2.43 1.00 97%
Gurobi-9.1.2 1.27 97%
KNITRO-12.3.0 1.74 97%
IPOPT-3.12.13 3.61 88%

older benchmarks: