Interactive plots showing pairwise time differences for every instance and every solver
Interactive charts comparing the results of Hans Mittelmann’s benchmarks. Each solver can be selected to show pairwise running time factors for every other solver in the respective benchmark. These plots should make browsing the results easier. The score (scaled shifted geometric mean) is recomputed using the reported solving times. We also calculate a “virtual best” or “ensemble” solver that picks the best performance over all solvers for every single problem instance. This might reveal how much potential the individual solvers still have. Please let me know if you have a question or if there is an error.
Choose base solver for comparison:
| solver | score (as reported) | solved of 65 |
|---|---|---|
| ⭐ virtual best | 0.76 | 100% |
| 🥇 COPT 7.1.0 | 1.00 (1.00) | 100% |
| 🥈 Optverse 1.0.0 | 1.45 (1.45) | 100% |
| 🥉 MindOpt 1.0.0 | 1.81 (1.81) | 97% |
| 📊 MOSEK 10.1.9 | 2.62 (2.62) | 98% |
| 📊 XOPT 0.0.6 | 5.41 (5.41) | 91% |
| 📊 ORTOOLS 9.10 | 15.66 (15.70) | 77% |
| 📊 HiGHS 1.6.0 | 19.35 (19.30) | 80% |
| 📊 KNITRO 14.1.0 | 21.52 (21.50) | 74% |
| 📊 MATLAB R2023a | 27.07 (27.10) | 77% |
| 📊 Tulip 0.9.4 | 66.89 (66.90) | 55% |
Choose base solver for comparison:
| solver | score (as reported) | solved of 65 |
|---|---|---|
| ⭐ virtual best | 0.85 | 100% |
| 🥇 COPT 7.1.0 | 1.00 (1.00) | 100% |
| 🥈 Optverse 1.0.0 | 1.61 (1.61) | 97% |
| 🥉 MindOpt 1.0.0 | 1.89 (1.89) | 97% |
| 📊 MOSEK 10.1.9 | 5.61 (5.61) | 80% |
| 📊 XOPT 0.0.6 | 6.33 (6.33) | 80% |
| 📊 HiGHS 1.6.0 | 17.44 (17.40) | 78% |
| 📊 CLP 1.17.7 | 26.13 (26.10) | 62% |
| 📊 Google-GLOP | 56.97 (57.00) | 51% |
| 📊 SOPLEX 6.0.0 | 87.20 (87.20) | 49% |
Choose base solver for comparison:
| solver | score (as reported) | solved of 25 |
|---|---|---|
| ⭐ virtual best | 0.86 | 100% |
| 🥇 OptVerse 1.0.0 | 1.00 (1.00) | 100% |
| 🥈 MDPT | 1.16 (1.16) | 100% |
| 🥉 COPT 7.1.0 | 2.12 (2.12) | 100% |
| 📊 Clp 1.17.7 | 5.97 (5.97) | 100% |
| 📊 HiGHS 1.7.0 | 12.71 (12.70) | 80% |
| 📊 MOSEK 10.1.9 | 26.41 (26.40) | 88% |
| 📊 QSopt 1.01 | 34.89 (34.90) | 68% |
| 📊 SOPLEX 6.0.0 | 66.34 (66.30) | 64% |
Choose base solver for comparison:
| solver | score (as reported) | solved of 240 |
|---|---|---|
| ⭐ virtual best | 0.73 | 95% |
| 🥇 COPT | 1.00 (1.00) | 92% |
| 🥈 LEOPT | 2.90 (2.90) | 75% |
| 🥉 MindOpt | 2.90 (2.90) | 82% |
| 📊 XOPT | 4.41 (4.41) | 78% |
| 📊 HiGHS | 6.94 (6.94) | 66% |
| 📊 SCIPC | 7.36 (7.36) | 63% |
| 📊 SCIP | 8.61 (8.61) | 58% |
| 📊 CBC | 12.80 (12.80) | 45% |
Choose base solver for comparison:
| solver | score (as reported) | solved of 45 |
|---|---|---|
| ⭐ virtual best | 0.58 | 100% |
| 🥇 OptVerse 1.0.0 | 1.00 (1.00) | 91% |
| 🥈 COPT 7.1.0 | 1.11 (1.11) | 91% |
| 🥉 XOPT 0.0.6 | 7.88 (7.90) | 69% |
| 📊 MindOpt 1.0.0 | 10.44 (10.40) | 53% |
| 📊 HiGHS 1.7.0 | 11.79 (11.80) | 64% |
| 📊 LEOPT 0.5.1 | 12.45 (12.40) | 60% |
| 📊 SCIPC | 12.99 (13.00) | 62% |
| 📊 SCIP 9.0.0 | 22.40 (22.40) | 44% |
| 📊 CBC 2.10.7 | 39.28 (39.30) | 22% |
| 📊 GLPK 5.0 | 41.63 (41.60) | 13% |
Choose base solver for comparison:
| solver | score (as reported) | solved of 32 |
|---|---|---|
| ⭐ virtual best | 0.91 | 94% |
| 🥇 COPT 7.1.0 | 1.00 (1.00) | 94% |
| 🥈 OptVerse 1.0.0 | 1.23 (1.23) | 94% |
| 🥉 MindOpt 1.0.0 | 5.71 (5.71) | 84% |
| 📊 XOPT 0.0.6 | 5.72 (5.72) | 78% |
| 📊 SCIPC | 5.82 (5.82) | 78% |
| 📊 HiGHS 1.7.0 | 6.04 (6.04) | 78% |
| 📊 SCIP 9.0.0 | 7.80 (7.80) | 81% |
| 📊 CBC 2.10.5 | 16.46 (16.50) | 62% |
Choose base solver for comparison:
| solver | score (as reported) | solved of 75 |
|---|---|---|
| ⭐ virtual best | 0.58 | 100% |
| 🥇 COPT 7.1.0 | 1.00 (1.00) | 100% |
| 🥈 MindOpt 1.0.0 | 1.45 (1.45) | 100% |
| 🥉 MOSEK 10.2.3 | 2.85 (2.85) | 97% |
| 📊 SDPT3 4.0 | 5.18 (5.18) | 92% |
| 📊 CSDP 6.2.0 | 5.25 (5.25) | 93% |
| 📊 HDSDP 1.0.0 | 7.91 (7.91) | 93% |
| 📊 SDPA 7.4.2 | 10.52 (10.50) | 81% |
| 📊 SeDuMi 1.3.5 | 29.06 (29.10) | 83% |
Choose base solver for comparison:
| solver | score (as reported) | solved of 18 |
|---|---|---|
| ⭐ virtual best | 0.85 | 100% |
| 🥇 Optverse 1.0.0 OPTVERSE | 1.00 (1.00) | 100% |
| 🥈 COPT 7.1.0 COPT | 1.09 (1.09) | 100% |
| 🥉 MOSEK 10.1.28 MOSEK | 1.14 (1.14) | 100% |
| 📊 KNITRO 14.0.0 Knitro | 10.12 (10.10) | 94% |
| 📊 ECOS 2.0.4 ECOS | 106.44 (106.00) | 61% |
Choose base solver for comparison:
| solver | score (as reported) | solved of 47 |
|---|---|---|
| ⭐ virtual best | 0.92 | 98% |
| 🥇 COPT 7.1.0 | 1.00 (1.00) | 98% |
| 🥈 MOSEK 10.1.28 | 4.94 (4.17) | 77% |
| 🥉 SCIP 9.0.0 | 11.63 (9.31) | 66% |
Choose base solver for comparison:
| solver | score (as reported) | solved of 97 |
|---|---|---|
| ⭐ virtual best | 0.65 | 98% |
| 🥇 SHOT 1.1 | 1.00 (1.00) | 94% |
| 🥈 Baron 24.5.8 | 5.91 (5.91) | 66% |
| 🥉 RAPOSa 4.4.1 | 10.26 (10.30) | 71% |
| 📊 SCIP 9.0.0 | 31.66 (31.70) | 35% |
| 📊 ANTIGONE 1.1 | 63.08 (63.10) | 16% |
| 📊 COUENNE 0.5 | 74.93 (74.90) | 6% |
Choose base solver for comparison:
| solver | score (as reported) | solved of 23 |
|---|---|---|
| ⭐ virtual best | 0.94 | 70% |
| 🥇 QuBowl | 1.00 (1.00) | 65% |
| 🥈 Baron 24.5.8 | 1.82 (1.82) | 52% |
| 🥉 SHOT 1.1 | 1.92 (1.76) | 52% |
| 📊 McSparse 2.0 | 2.36 (2.36) | 52% |
| 📊 SCIP 9.0 | 5.73 (5.73) | 30% |
| 📊 Biqbin | 7.63 (5.40) | 39% |
Choose base solver for comparison:
| solver | score (as reported) | solved of 101 |
|---|---|---|
| ⭐ virtual best | 0.60 | 100% |
| 🥇 SHOT 1.1 | 1.00 (1.00) | 94% |
| 🥈 Baron 24.5.8 | 5.19 (5.19) | 71% |
| 🥉 SCIP 9.0.0 | 30.29 (30.30) | 41% |
| 📊 ANTIGONE 1.1 | 74.47 (74.50) | 28% |
Choose base solver for comparison:
| solver | score (as reported) | solved of 50 |
|---|---|---|
| ⭐ virtual best | 0.23 | 100% |
| 🥇 Baron 24.5.8 | 1.00 (1.00) | 70% |
| 🥈 MINOTAUR 0.4.0 | 3.58 (3.58) | 50% |
| 🥉 ANTIGONE 1.1 | 3.92 (3.92) | 52% |
| 📊 SCIP 9.0.0 | 6.97 (6.97) | 30% |
Choose base solver for comparison:
| solver | score (as reported) | solved of 42 |
|---|---|---|
| ⭐ virtual best | 0.84 | 100% |
| 🥇 OptVerse 1.0.0 | 1.00 (1.00) | 100% |
| 🥈 COPT 7.1.0 | 1.23 (1.23) | 100% |
| 🥉 MOSEK 10.1.21 | 1.97 (1.97) | 98% |
| 📊 KNITRO 14.1.0 | 2.37 (2.37) | 95% |
| 📊 MindOpt 1.0.0 | 7.09 (6.70) | 81% |
| 📊 IPOPT 3.14.5 | 11.05 (11.10) | 83% |
| 📊 Mnotaur | 55.95 (55.90) | 60% |
Choose base solver for comparison:
| solver | score (as reported) | solved of 31 |
|---|---|---|
| ⭐ virtual best | 0.72 | 87% |
| 🥇 COPT 7.1.0 | 1.00 (1.00) | 77% |
| 🥈 Shot 1.1 | 1.07 (1.07) | 81% |
| 🥉 Baron 24.5.8 | 4.28 (4.28) | 65% |
| 📊 MOSEK 10.2.2 | 10.03 (10.00) | 58% |
| 📊 KNITRO 14.1.0 | 12.75 (13.60) | 52% |
| 📊 SCIP 9.0.0 | 21.82 (21.80) | 45% |
| 📊 Bonmin 1.8.7 | 54.63 (54.60) | 23% |
| 📊 MNTAUR | 63.98 (64.00) | 26% |
Choose base solver for comparison:
| solver | score (as reported) | solved of 87 |
|---|---|---|
| ⭐ virtual best | 0.28 | 94% |
| 🥇 BARON | 1.00 (1.00) | 86% |
| 🥈 SHOT | 1.90 (1.90) | 61% |
| 🥉 SCIP | 3.89 (3.90) | 75% |
| 📊 ANTIGONE | 15.96 (16.00) | 60% |
| 📊 LINDO | 18.77 (18.80) | 32% |
Choose base solver for comparison:
| solver | score (as reported) | solved of 29 |
|---|---|---|
| ⭐ virtual best | 1.00 | 93% |
| 🥇 KNITRO 14.0 | 1.00 (1.00) | 93% |
| 🥈 filter MPEC | 16.90 (16.90) | 62% |
| 🥉 LOQO 7.03 | 37.10 (37.10) | 21% |