I have good news and bad news. To start with the bad news, it appears that I’ve once again cursed another pick for winning a supermajor. But the good news is that despite apparently being King Midas in reverse, I learned a good bit about predicting Melee tournaments. After Tipped Off 15 happened, I decided to compare my simulations for each Top 64 seed with how they actually performed. Based on the number of times a player ended up with their real placement, I categorized their results as follows:
- Predicted Outcomes – where the model’s most common simulated placement matched a given player’s actual placement.
- Common Outcomes – where anywhere from a given player’s 2nd to 4th most simulated placements matched their actual placement.
- Unaccounted Outcomes – where the model outright lacked simulated instances of a given player’s actual placement.
- Uncommon Outcomes – where the model’s 5th to 8th most simulated placements matched a given player’s actual placement.
- Rare Outcomes – where the model’s 9th to 12th most simulated placements matched a given player’s actual placement.
What did the model get exactly right? (Predicted Outcomes)
| Player | Average Simulated Placement | Most Common Simulated Placement | Actual Placement | How Many Simulations/1000 Captured Their Actual Placement? |
| SDeems | 41.97 | 49th | 49th | 638 |
| mayb | 39.3 | 49th | 49th | 494 |
| Panko | 40 | 49th | 49th | 483 |
| Egg$ | 38.67 | 49th | 49th | 467 |
| Zasa | 35.17 | 49th | 49th | 403 |
| Gahtzu | 34.95 | 49th | 49th | 402 |
| S2J | 24.19 | 33rd | 33rd | 268 |
| Krudo | 22.58 | 17th | 17th | 245 |
| Lucky | 24 | 33rd | 33rd | 242 |
| Colbol | 23.28 | 33rd | 33rd | 240 |
| Akir | 24.61 | 33rd | 33rd | 238 |
| Panda | 22.71 | 25th | 25th | 235 |
| Joshman | 20.08 | 17th | 13th | 212 |
| Spark | 18.66 | 17th | 17th | 206 |
| SDJ | 21.71 | 17th | 17th | 206 |
| Ossify | 21.49 | 17th | 17th | 200 |
| Aklo | 21.24 | 17th | 17th | 198 |
| Wizzrobe | 13.45 | 9th | 9th | 192 |
| Jmook | 11.79 | 5th | 5th | 129 |
It turns out that predicting exact placements of players – particularly ones in a heavily volatile field – is quite difficult, but it’s more doable than you might think. Although 19 out of 64 may not actually seem like a very big number, in reality, if you compared this with strictly how seeding would have “predicted” outcomes, the model tied seeding for exact placements. Not too shabby for a first attempt at a supermajor Top 64.
What did the model appropriately consider as likely to happen? (Common Outcomes)
| Player | Average Simulated Placement | Most Common Simulated Placement | Actual Placement | How Many Simulations / 1000 Captured Their Actual Placement? |
| salami | 39.06 | 49th | 33rd | 387 |
| Preeminent | 35.66 | 49th | 33rd | 359 |
| Unsure | 34.34 | 49th | 33rd | 341 |
| max | 35.39 | 49th | 33rd | 333 |
| essy | 35.17 | 49th | 33rd | 323 |
| Fro116 | 34.99 | 49th | 33rd | 310 |
| KJH | 34.94 | 49th | 33rd | 308 |
| Lowercase hero | 35.62 | 49th | 33rd | 306 |
| Inky | 35.29 | 49th | 33rd | 305 |
| CPU0 | 35.94 | 49th | 33rd | 289 |
| Balloon Day | 36.19 | 49th | 33rd | 288 |
| bobby big ballz | 23.43 | 33rd | 17th | 213 |
| n0ne | 35.17 | 49th | 25th | 205 |
| Dawson | 31.73 | 33rd | 49th | 202 |
| Wally | 24.12 | 33rd | 25th | 190 |
| Zamu | 22.3 | 33rd | 25th | 188 |
| Junebug | 21.12 | 25th | 33rd | 170 |
| Cody Schwab | 6.33 | 1st | 2nd | 161 |
| Beezy | 33.98 | 33rd | 25th | 138 |
| Soonsay | 12.23 | 9th | 13th | 130 |
| Khryke | 35.32 | 49th | 25th | 130 |
| ckyulmiqnudaetr | 21.76 | 33rd | 25th | 129 |
| Zanya | 35.34 | 49th | 25th | 117 |
| moky | 12.14 | 9th | 7th | 116 |
| Drephen | 34.61 | 49th | 17th | 109 |
| Hungrybox | 11.92 | 5th | 9th | 96 |
| Wevans | 35.65 | 49th | 17th | 87 |
Let’s address the elephant in the room here: before it happened, it was easy to look at the near-decade-long streak Hungrybox had as insurmountable. In fact, when I noticed within my simulation that Hungrybox’s average placement was around the 9th to 13th range, and that he had a roughly 10 percent chance of finishing in ninth place, I was initially hostile to this idea.
At the same time, not every major top eight comes from the same path. Tipped Off 15 was an extraordinarily stacked event, one which we might see a couple to a few times a year at best. Clearly, this was not your average major. Looking at it in hindsight, the end of Hungrybox’s major top eight streak was actually quite possible. He simply lost a tossup vs. Mango and then lost another coin flip vs. moky. Hungrybox is an amazing player who could win a major this year, but even he can’t escape probability.
What did the model not account for? (Unaccounted Outcomes)
If you remember from my write-up last week, you’ll note that a limitation to my current methodology for simulating events comes from the fact that I’m only simulating matches played between the Top 64 seeds, assuming that all of them make it to their respective winners matches. Furthermore, when someone loses the first round of my simulated events, I automatically ‘float’ them over into 49th place matches.
Obviously, this does not happen in real life. Instead, the loser of such a match in the winners bracket, would have to play a 65th place match vs. someone else in the losers bracket. As a result, you’ll notice a common recurring theme in what my model did not account for at all. It will also give some perspective, hopefully, on the relatively low number of consequences that come from its current limitations.
| Player | Average Simulated Placement | Most Common Simulated Placement | Actual Placement |
| Komodo | 35.33 | 49th | 65th |
| Paladin | 38.79 | 49th | 65th |
| Chango | 39.1 | 49th | 65th |
| billybopeep | 39.05 | 49th | 65th |
| Kalvar | 39.63 | 49th | 65th |
| Maelstrom | 34.98 | 49th | DQ |
| mvlvchi | 23.59 | 33rd | 97th |
| Louis | 42.8 | 49th | 97th |
| 2saint | 21.97 | 33rd | 65th |
In the case of Komodo, Paladin, Chango, billybopeep, and Kalvar, it ended up not mattering much at all. All of them still finished within a placement of their most commonly simulated placement. However, in the case of Maelstrom, he DQ’d from the event, so his results were significantly off. And more fundamentally in the case of mvlvchi, Louis, and 2saint, each of these three lost in significant upsets, with mvlvchi falling to each of bambi and lynxwynx, Louis losing to Equilateral and Zimberfizz, and 2saint losing immediately to Unsure upon losing to Wevans (with Unsure also losing an unaccounted-for set to POG Epic Gamer).
Generally, I’m not too concerned about some of these outcomes not being accounted for. However, as a ‘fix’ to the floating issue that leads to 65th place match erasure, as well as the lack of accounting for DQs, what I’ll likely implement is some portion of the following steps:
- Running a random number generator function that gives every Top 64 seed a 1 in 64 chance of DQ’ing before a match in Top 64.
- For players who are DQ’d, replace them with “Top 96 Seed” which takes the presumptive set records and expected win-rates for every player within the 65 to 95 seed range and combines them into one generic player.
- When players lose in Top 96, the winners side, instead of being floated to 49th place matches, they play a “Top 96 Seed” for 65th place.
What did the model consider unlikely? (Uncommon Outcomes)
| Player | Average Simulated Placement | Most Common Simulated Placement | Actual Placement | How Many Simulations / 1000 Captured Their Actual Placement? |
| Salt | 22.09 | 17th | 9th | 119 |
| KoDoRiN | 21.83 | 17th | 9th | 110 |
| Chem | 21.82 | 33rd | 13th | 101 |
| aMSa | 11.96 | 5th | 4th | 84 |
| Zain | 6.16 | 1st | 5th | 65 |
For each of Salt, Chem, and KoDoRiN, the model was roughly low on them, but it was within the range of possibility, with the average placements not being too far away. Although their respective placements did not fall within the “Common” threshold I had set above, they each fell at the top of the “Uncommon” one, with their actual placements being their fifth most simulated ones. On a slightly related note, Zain and aMSa each fell within this category, with Zain’s fifth place coming the 6th most number of times, and aMSa’s fourth place coming the 7th most number of times, but all in all, their average performance was not too far off from reality.
What did the model consider the biggest surprises? (Rare Outcomes)
| Player | Average Simulated Placement | Most Common Simulated Placement | Actual Placement | How Many Simulations / 1000 Captured Their Actual Placement? |
| Plup | 12.17 | 13th | 3rd | 61 |
| Mang0 | 11.07 | 5th | 1st | 46 |
| MOF | 36.13 | 49th | 13th | 31 |
| Axe | 21.97 | 17th | 7th | 24 |
There’s two players that I think my model totally underrated. The first one is MOF. On one hand, her results in 2024, frankly speaking, were quite bad and nothing like what we’ve come to expect from a Top 50 player. I had no way of accounting for this in the model I had set up. Meanwhile, the second one is Plup. This one was admittedly my bad; I should know better than to ever doubt Plup’s ability to pop up and “surprise” top players given his penchant for doing exactly that. For next time, I’ll take a look at 12 months of data prior to an event; not only six.
In my opinion, there were two real shockers of the tournament: Mang0’s victory and Axe’s huge run to seventh place. For Mang0, it took surviving game five scares from Hungrybox, Aklo, and Egg$ right before top eight, where he somehow 9-1’d the top two players in the world. One of them, mind you, had beaten Mang0 11 times in a row. Zain, quite literally, is the toughest opponent Mang0 has ever had, and Mang0 destroyed him. As a reminder: the model gave the field just about a third’s chance of overcoming Zain and Cody, with Mang0 as its respective champion, with a small, but noticeable 4.6 percent chance.
In the case of Axe, his top eight run here is genuinely one of his best for the current phase of his career. With wins against the odds over Krudo, SDJ, Soonsay, and Wizzrobe, it really is way more astounding than just the already strong placement. If you’re a fan of Axe, this is the exact type of run you want to see from him: proof that he can get back into being a major dark horse.
Was it worth it?
To see how my model broadly performed, I took a look at all Top 64 seeds, their average simulated placements, their actual placements, and checked whether the distribution of their said simulated placements fell within the “Common” threshold or not. If any of their simulated placements fell within the “Common” threshold, or if their average placement fell within a placement (ahead or behind) of their actual placement, I considered the model appropriately within range.
If you’re curious, the full table is here – the long story short is that even including placements I didn’t account for, I ended up with 54 out of 64 results within range: an 84.3 percent rate. Using a similar methodology, I compared how I simulated these player’s placements to how within range predicting placements off seeding would have been and found out that it was slightly lower, at 46 out of 64 results: a 71.9 percent rate. Although there’s fixes and clear takeaways from my model’s current limitations, I’m satisfied with my current progress. Maybe one day, I will correctly predict the winner of a Melee tournament.