Visualizations of Mittelmann benchmarks

Interactive plots showing pairwise time differences for every instance and every solver

Interactive charts comparing the results of Hans Mittelmann’s benchmarks. Each solver can be selected to show pairwise running time factors for every other solver in the respective benchmark. These plots should make browsing the results easier. The score (scaled shifted geometric mean) is recomputed using the reported solving times. We also calculate a “virtual best” or “ensemble” solver that picks the best performance over all solvers for every single problem instance. This might reveal how much potential the individual solvers still have. Please let me know if you have a question or if there is an error.

LPfeas Benchmark (find PD feasible point) + ADDENDUM (31 Jul 2025)

Choose base solver for comparison:

solver score (as reported) solved of 65
⭐ virtual best 0.30 100%
🥇 COPT 7.2.0 1.00 (1.00) 100%
🥈 Optverse 1.0.0 1.54 (1.54) 100%
🥉 cuOpt-H100 1.86 (1.54) 89%
📊 cuPDLP-C-H100 2.22 (1.80) 88%
📊 cuPDLP-C 3.05 (2.43) 86%
📊 MOSEK 11.0.5 3.29 (3.29) 91%
📊 cuOpt 25.05 3.31 (2.36) 86%
📊 XOPT 0.0.8 5.77 (5.77) 91%
📊 HPR-LP 0.1.0 13.24 (9.00) 74%
📊 PDLP 16.67 (16.70) 77%
📊 KNITRO 15.0.0 17.05 (17.10) 75%
📊 HiGHS 1.11.0 22.01 (22.00) 75%
previous benchmarks 🔽

LPopt Benchmark (find optimal basic solution) (16 Jun 2025)

Choose base solver for comparison:

solver score (as reported) solved of 65
⭐ virtual best 0.90 100%
🥇 COPT 7.2.0 1.00 (1.00) 100%
🥈 Optverse 1.0.0 1.68 (1.68) 97%
🥉 XOPT 0.0.8 6.60 (6.60) 80%
📊 MOSEK 11.0.13 7.45 (7.45) 80%
📊 HiGHS 1.11.0 17.04 (17.00) 78%
📊 CLP 1.17.7 27.23 (27.20) 62%
📊 Google-GLOP 59.38 (59.40) 51%
📊 SOPLEX 7.1.2 93.53 (93.50) 49%
previous benchmarks 🔽

Large Network-LP Benchmark (commercial vs free) (26 Jun 2025)

Choose base solver for comparison:

solver score (as reported) solved of 25
⭐ virtual best 0.99 100%
🥇 OptVerse 1.0.0 1.00 (1.00) 100%
🥈 COPT 7.2.0 2.15 (2.15) 100%
🥉 Clp 1.17.7 5.97 (5.97) 100%
📊 HiGHS 1.11.0 14.07 (14.10) 80%
📊 MOSEK 11.0.16 27.20 (27.20) 84%
📊 QSopt 1.01 34.89 (34.90) 68%
📊 SOPLEX 7.1.2 65.43 (65.40) 64%
previous benchmarks 🔽

The MIPLIB2017 Benchmark Instances (preprocessed data) - 8 threads (20 Jun 2025)

Choose base solver for comparison:

solver score (as reported) solved of 240
⭐ virtual best 0.75 93%
🥇 COPT 1.00 (1.00) 90%
🥈 optverse 1.89 (1.89) 85%
🥉 XOPT 5.05 (5.05) 67%
📊 HiGHS 6.56 (6.56) 65%
📊 LEOPT 6.69 (6.69) 62%
📊 SCIPC 6.97 (6.97) 60%
📊 SCIP 8.76 (8.76) 53%
previous benchmarks 🔽

MILP cases that are slightly pathological (preprocessed data) (23 Jun 2025)

Choose base solver for comparison:

solver score (as reported) solved of 45
⭐ virtual best 0.70 93%
🥇 COPT 7.2.0 1.00 (1.00) 531%
🥈 OptVerse 1.0.0 2.59 (2.59) 1376%
🥉 TAYLORMIP 0.8 3.57 (3.57) 1893%
📊 HiGHS 1.11.0 9.03 (9.03) 4791%
📊 XOPT 0.0.8 9.90 (9.90) 5253%
📊 LEOPT 0.5.1 11.21 (11.20) 5951%
📊 SCIPC 12.13 (12.10) 6436%
📊 SCIP 9.2.1 17.54 (17.60) 9311%
previous benchmarks 🔽

Infeasibility Detection for MILP Problems (25 Jun 2025)

Choose base solver for comparison:

solver score (as reported) solved of 32
⭐ virtual best 0.95 94%
🥇 COPT 7.2.0 1.00 (1.00) 94%
🥈 OptVerse 1.0.0 1.21 (1.21) 94%
🥉 XOPT 0.0.8 5.08 (6.06) 78%
📊 SCIPC 5.17 (5.17) 81%
📊 HiGHS 1.11.0 6.60 (6.60) 78%
📊 SCIP 9.2.1 7.35 (7.35) 69%
📊 CBC 2.10.5 16.19 (16.20) 62%
previous benchmarks 🔽

Several SDP-codes on sparse and other SDP problems (also on GPUs) (9 Aug 2025)

Choose base solver for comparison:

solver score (as reported) solved of 75
⭐ virtual best 0.35 100%
🥇 COPT 7.2.0 1.00 (1.00) 100%
🥈 MOSEK 11.0.16 2.69 (2.69) 97%
🥉 cuLoRADS 1.0.0 3.97 (3.97) 95%
📊 SDPT3 4.0 5.34 (5.34) 92%
📊 CSDP 6.2.0 5.41 (5.41) 93%
📊 SDPA 7.4.4 8.01 (8.01) 81%
📊 SeDuMi 1.3.5 29.99 (30.00) 83%
previous benchmarks 🔽

Large Second Order Cone Benchmark (5 Apr 2025)

Choose base solver for comparison:

solver score (as reported) solved of 18
⭐ virtual best 0.88 100%
🥇 COPT 7.2.0 COPT 1.00 (1.00) 100%
🥈 Optverse 1.0.0 OPTVERSE 1.10 (1.10) 100%
🥉 MOSEK 11.0.13 MOSEK 1.24 (1.24) 100%
📊 KNITRO 14.0.0 Knitro 11.11 (11.10) 94%
📊 ECOS 2.0.4 ECOS 116.88 (117.00) 61%
previous benchmarks 🔽

Mixed-integer SOCP Benchmark (9 Apr 2025)

Choose base solver for comparison:

solver score (as reported) solved of 47
⭐ virtual best 0.98 98%
🥇 COPT 7.2.0 1.00 (1.00) 98%
🥈 MOSEK 11.0.13 6.36 (5.47) 77%
🥉 SCIP 9.2.1 12.26 (9.81) 66%
previous benchmarks 🔽

Binary Non-Convex QPLIB Benchmark (26 Jun 2025)

Choose base solver for comparison:

solver score (as reported) solved of 97
⭐ virtual best 0.67 98%
🥇 SHOT 1.1 1.00 (1.00) 94%
🥈 Baron 25.3.19 7.11 (7.11) 67%
🥉 RAPOSa 4.4.1 10.26 (10.30) 71%
📊 SCIP 9.2.1 31.72 (31.70) 36%
📊 ANTIGONE 1.1 63.08 (63.10) 16%
📊 COUENNE 0.5 74.93 (74.90) 6%
previous benchmarks 🔽

Nonconvex QUBO-QPLIB Benchmark (12 Jul 2025)

Choose base solver for comparison:

solver score (as reported) solved of 23
⭐ virtual best 0.96 70%
🥇 QuBowl 1.00 (1.00) 65%
🥈 Baron 25.3.19 1.80 (1.80) 57%
🥉 SHOT 1.1 1.84 (1.84) 52%
📊 McSparse 2.0 2.89 (2.89) 52%
📊 SCIP 9.2.1 6.82 (6.82) 30%
📊 Biqbin 9.33 (6.60) 39%
previous benchmarks 🔽

Discrete Non-Convex QPLIB Benchmark (non-binary) (13 Jul 2025)

Choose base solver for comparison:

solver score (as reported) solved of 104
⭐ virtual best 0.75 96%
🥇 SHOT 1.1 1.00 (1.00) 91%
🥈 Baron 25.3.19 7.95 (7.95) 66%
🥉 SCIP 9.2.1 36.96 (37.00) 38%
📊 ANTIGONE 1.1 84.87 (84.90) 27%
previous benchmarks 🔽

Continuous Non-Convex QPLIB Benchmark (11 Jul 2025)

Choose base solver for comparison:

solver score (as reported) solved of 52
⭐ virtual best 0.23 100%
🥇 Baron 25.3.19 1.00 (1.00) 67%
🥈 MINOTAUR 0.4.1 4.49 (4.49) 48%
🥉 ANTIGONE 1.1 5.23 (5.23) 52%
📊 SCIP 9.2.1 11.39 (11.40) 27%
previous benchmarks 🔽

Convex Continuous QPLIB Benchmark (also on GPUs) (17 Jul 2025)

Choose base solver for comparison:

solver score (as reported) solved of 42
⭐ virtual best 0.82 100%
🥇 COPT 7.2.0 1.00 (1.00) 100%
🥈 OptVerse 1.0.0 1.08 (1.08) 100%
🥉 KNITRO 15.0.0 1.90 (1.90) 98%
📊 MOSEK 11.0.16 2.32 (2.32) 98%
📊 IPOPT 3.14.5 11.92 (11.90) 83%
📊 Mnotaur 60.35 (60.30) 60%
previous benchmarks 🔽

Convex Discrete QPLIB Benchmark (6 Jul 2025)

Choose base solver for comparison:

solver score (as reported) solved of 32
⭐ virtual best 0.53 91%
🥇 COPT 7.2.0 1.00 (1.00) 75%
🥈 Shot 1.1 1.03 (1.03) 78%
🥉 Baron 25.3.19 2.15 (2.15) 72%
📊 MOSEK 11.0.16 6.79 (6.79) 62%
📊 KNITRO 15.0.0 11.16 (11.20) 47%
📊 SCIP 9.2.1 18.92 (18.90) 44%
📊 Bonmin 1.8.7 45.69 (45.70) 22%
📊 MNTAUR 53.24 (53.20) 25%
previous benchmarks 🔽

Mixed Integer Nonlinear Programming Benchmark (MINLPLIB) (24 Jun 2025)

Choose base solver for comparison:

solver score (as reported) solved of 200
⭐ virtual best 0.28 92%
🥇 BARON 1.00 (1.00) 80%
🥈 SCIP 1.59 (1.50) 76%
🥉 LINDO 4.95 (4.80) 58%
📊 SHOT 5.23 (5.10) 48%
previous benchmarks 🔽

MPEC Benchmark (Math. Progr. w. Equilibrium Constraints) (14 Apr 2025)

Choose base solver for comparison:

solver score (as reported) solved of 29
⭐ virtual best 1.00 97%
🥇 KNITRO 14.2 1.00 (1.00) 97%
🥈 filter-MPEC 18.15 (18.10) 62%
🥉 LOQO 7.03 39.84 (39.80) 21%
previous benchmarks 🔽