Wednesday, June 19, 2013

Matchmaking in DOTA2

A quick guide to how matchmaking works in DOTA2.

Statements from the DOTA2 dev team

The following were culled from the developer's forums:

1. There are no brackets or skill-tiers. You are rated on a continuous scale, so you can think of it as a percentile rank from 1-100.

2. The tiers you know from the Replay-Filters are not used in MM.

3. Battle Points and Battle Level are not used in matchmaking.

4. Performance Bars are not used in matchmaking.

5. We don't enforce a 50-50 win rate.

(And you are certainly not getting matched with bad players on purpose when you have a win-streak. There's no conspiracy just because your win streak ended)

What we know from match analysis

That's not very much to go on. But fortunately, we can do our own analysis on the match data on the servers.

Dotametrics provided me the data to put together this infographic showing the size of the match skill brackets in DOTA2. Yes the data is a few months old, but this distribution has been stable for more than a year now -  it's stood up to repeated sampling - the last time Valve changed their distribution was early 2012. All matches are tagged with a "skill level". Roughly speaking, Normal is anywhere up to 1 standard deviation above the mean, High is between 1 and 2 standard deviations above the mean, and Very High is 3 standard deviations above the mean.

You can find out what skill level your matches are by going to Watch -> Recent Games -> Filter by exact name and skill level.

It's true that players aren't officially tagged by skill level, but rest assured if all your games are tagged as "Very High Skill" it's very likely that you're in the top 4% percentile of players, according to the data that was collected. You will likely get matched into different quality matches in consecutive games, not because your hidden MMR is fluctuating, but because a higher level game might need a player and pull you in, or next time a lower level game might need a player and pull you in. If your MMR is straddling a bracket this is more likely to happen.

What affects MMR, and why it's hidden

From anecdotal reports from players (and my own experiments) the hidden MMR is influenced by more than just win rate. An account at the bottom of the skill curve can rise very rapidly into the Very High Skill matches if you manage to stomp a few games. It is likely Valve is using some secret game statistics to judge player skill - no one knows what it could be. Maybe it's denies, maybe the game detects if you're able to stack and pull creeps, it could also be KDR, or GPM. Maybe it's buying "advanced" items that higher skill players generally buy and which low skill players don't. Whatever the case, their smurf detection speed is way faster than what HON and LOL can do.

This is the most plausible reason why MMR remains hidden in DOTA2 - if it was visible, and rating change could be observed from game to game, it would be possible to reverse engineer the MMR calculation and thus manipulate it - if players figured out buying 4 wards triggered a bonus 5 MMR for the match, for example, or that if you reached 20 denies you got a bonus 2 MMR for the match, it could potentially be abused.