Visualizations of Mittelmann benchmarks

Interactive plots showing pairwise time differences for every instance and every solver

Benchmark of Simplex LP solvers (26 Sep 2020)

Choose base solver for comparison:

  score solved
COPT-1.4.2 1.00 100%
MindOpt-0.10.0 1.41 92%
Gurobi-9.0.3 1.45 100%
CLP-1.17.6 3.37 100%
MOSEK-9.2.20 7.18 95%
SAS-OR-15.1 7.25 90%
HiGHS-1.0.0 13.60 82%
Google-GLOP 14.00 85%
MATLAB-R2020a 15.60 80%
SOPLEX-5.0.0 18.50 90%
GLPK-4.65 51.90 68%

older versions:


Benchmark of Barrier LP solvers (8 Sep 2020)

Choose base solver for comparison:

  score solved
Gurobi 1.00 100%
MOSEK-9.2.21 1.66 98%
SAS-OR-15.1 3.64 96%
MATLAB-R2020a 18.30 76%
Tulip-0.5.1 21.90 78%
CLP-1.17.6 29.80 82%

Large Network-LP Benchmark (commercial vs free) (26 Sep 2020)

Choose base solver for comparison:

  score solved
SAS-OR-15.1 1.00 100%
Gurobi-9.0.3 1.91 100%
COPT-1.4.2 2.42 100%
Clp-1.17.6 4.60 100%
MATLAB-R2020a 17.40 90%
MindOpt-0.10.0 49.80 90%
QSopt-1.01 68.10 60%
SOPLEX-5.0.0 92.10 60%
MOSEK-9.2.21 107.00 80%

older versions:


Mixed-integer SOCP Benchmark (14 Jun 2020)

Choose base solver for comparison:

  score solved
GUROBI 1.00 100%
MOSEK 10.80 68%
SCIP 40.90 62%

Binary Non-Convex QPLIB Benchmark (12 Aug 2020)

Choose base solver for comparison:

  score solved
Gurobi-9.0.1 1.00 98%
Baron-20.14.4 15.00 51%
OCTERACT-1.07.29 19.00 43%
FSCIP 41.00 41%
SCIP 49.00 31%
ANTIGONE-1.1 59.00 28%
MINOTAUR-0.2.1 80.00 8%

Discrete Non-Convex QPLIB Benchmark (non-binary) (20 Aug 2020)

Choose base solver for comparison:

  score solved
Gurobi-9.0 1.00 86%
FSCIP 7.68 42%
SCIP 12.70 39%
Baron-20.14.4 15.30 40%
ANTIGONE-1.1 18.40 38%
OCTERACT-1.07.29 34.70 10%
COUENNE-0.5 41.10 10%
MINOTAUR-0.2.1 47.30 5%

Continuous Non-Convex QPLIB Benchmark (20 Aug 2020)

Choose base solver for comparison:

  score solved
GUROBI-9.0 1.00 48%
ANTIGONE-1.1 1.46 52%
Baron-20.4.14 1.69 45%
OCTERACT-1.07.29 2.29 29%
SCIP-7.0.0 3.32 21%
COUENNE-0.5 4.13 14%
MINOTAUR-0.2.1 4.35 11%

Convex Discrete QPLIB Benchmark (15 Aug 2020)

Choose base solver for comparison:

  score solved
GUROBI-9.0.1 1.00 74%
Baron-20.14.4 5.42 52%
MOSEK-9.2.18 8.98 35%
SCIP-7.0.0 10.50 35%
Bonmin-1.8.7 13.10 32%
KNITRO-12.0 13.90 29%
OCTERACT-1.07.29 20.30 23%
MINOTAUR-0.2.1 21.70 35%
ANTIGONE-1.1 34.50 6%
Shot-1.0 40.40 3%

Convex Continuous QPLIB Benchmark (3 Apr 2020)

Choose base solver for comparison:

  score solved
MOSEK-9.2.4 1.00 91%
KNITRO-12.0 1.56 94%
IPOPT-3.13.0 3.64 88%