Bluetracker

Tracks Blizzard employees across various accounts.


Dear Blizzard, I would really like a Q&A session on upcoming MMR system changes

I am sure people want more details and I am as sure people have ten times more questions, but here's what I came up with or gathered already:

  • Any changes also planned for uncertainty?
  • Looks like new system promotes maining a few heroes rather than flexibility, true or not, and how big is the impact if true?
  • Is draft/banning taken into account? Choosing the right hero in draft may be more important than performing better than peers on a certain hero.
  • All in all, do you have some additional insight into what parameters are used? How zoning, peeling, bodyblocking, skillshot blocking is taken into account? How will Force Wall-Tassadar be able to compete with Archon-Tassadar?
  • Also do you plan to add more parameters in the mix?
  • There's some concern about how reliable is data on low played heroes — maybe like Chen, Rexxar, TLV, anything on that? (Though I can't imagine any answer here but "we think our data is reliable enough" :D)
  • Are we right to assume that new MMR calculation would be applied to all game modes? For QM, it looks like a step in the direction of per-hero MMR, do you think we may end up there, or at least do you think it may be viable to maintain?

  • BlizzTravis

    Posted 6 years, 10 months ago (Source)

    I'm looking at pulling a blog post together at least. To answer some of these quickly...

    • Uncertainty doesn't exist in the same form in the new system. The core concept of something that limits how much MMR can adjust on a per-game basis is there, but the factors that go into it are different.

    • The most important thing is still winning the game. That hasn't changed. Sometimes, that means sticking to heroes you know really well. Other times, that means having flexibility in your hero roster and playing the heroes that best fit the situation.

    • Drafting matters in the same way as the above statement. Drafting is a skill and how well you draft affects your win rate.

    • We have great insight into what parameters are used...but aren't going into that level of detail externally.

    • We'll add more parameters as it makes sense.

    • We have more than enough data on all existing heroes. For new heroes or major reworks, the performance-based adjustments will basically be disabled until enough data exists.

    • It will apply to all game modes that have MMR.

  • BlizzTravis

    Posted 6 years, 10 months ago (Source)

    Woah, thanks a lot, that was prompt!

    As I expected, despite some scepticism in comments, the only really sensible question was about the specific parameters :)

    Alas, I only now remembered a topic that was intended from the very start, hope you can add something on that in an upcoming blog post at least.

    It is: revealing MMR and the general clarity. Assuming "no" on "if you're going to reveal MMR" (:D) I'd like to ask whether we would be able to see when the adjustment takes place on the post-match screen, like we see PRA now.

    And while we're at it, PRA is still really unclear system, community certainly would appreciate an insight into it; at the very least I suggest displaying rank points gained/lost not just in hovered tooltip but explicitly, like:

    • + 200
    • - 2
    • + 4

    ..and save toolptips for the source of the points (base, relative team strength, PRA).

    We won't have visible MMR for this update, but I do still want to do it in the future.

    For now, you'll mainly see the effects through rank points. We're planning to swap out personal rank adjustment for a more direct performance adjustment which will give better insight while filling a similar role of keeping rank close to MMR. I'm not sure right now whether that will be for this season or the next yet.

  • BlizzTravis

    Posted 6 years, 10 months ago (Source)

    I think people's main concerns are along the lines of "who determines which stats are important" and "why these stats and not these other ones"

    I assume you guys are running some kind of adaptive machine learning, but most people are not going to understand that so maybe some kind of brief summary into how that works can help to alleviate the main concerns. Just a suggestion.

    Quite right. For the "who determines which stats are important", its the players. We've chosen the stats to monitor, but which ones are important for that situation is something we are measuring, not determining ourselves.

  • BlizzTravis

    Posted 6 years, 10 months ago (Source)

    Right. But the system, unless specifically programmed to do otherwise, will think a death at the last moment is extremely meaningful since it is normally a 60s death timer.

    Time is factored into the measurements. If you're dead for the last few seconds of a match, there wouldn't be any measurable impact relative to everything else that happened that game.

  • BlizzTravis

    Posted 6 years, 10 months ago (Source)

    Here's a, hopefully, easy one:

    Can you win a game yet still lose MMR points? IE: a player goes full tilt, intentionally tries to throw. But the other 4 players are just dominating and still win the game.

    No. You'll only gain points on a win and lose them on a loss.

    If you get full carried for a game, you won't lose points, but you also aren't going to gain very many either.

  • BlizzTravis

    Posted 6 years, 10 months ago (Source)

    Let's just hope this personal performance measure is a little more accurate than the MVP award.

    Very different system with very different goals.

  • BlizzTravis

    Posted 6 years, 10 months ago (Source)

    " * The most important thing is still winning the game. That hasn't changed."

    Might be a stupid question, but if it's only about winning, why do we have stats-based MMR ? why the win isn't the only factor ?

    Isn't the concept of stats-based MMR less accurate than the winrate itself to calculate your odd of winning ?

    It's about speed more than accuracy.

    The current system, based entirely on wins vs loses, works, but because your skill is diluted in a team of 5, unless you are dramatically better (or worse) than the rest of the team, it can take a large number of games before the difference in skill translates to more wins (or loses) and moves your MMR to where it should be.

    By factoring in individual skill to the equation, it can arrive at the proper MMR much faster which means the matchmaker can make better matches overall.

  • BlizzTravis

    Posted 6 years, 10 months ago (Source)

    We won't have visible MMR for this update, but I do still want to do it in the future.

    If you show a player a number, he/she wants to maximise it or minimise it. While self-improvement (which would be visible by rising MMR) is something humans crave naturally (or so I've heard beep boop), visible MMR can lead to self-inflicted frustration and anxiety, like worrying over every MMR point lost or gained during a match.

    By definition, MMR is relative and obeys an ordering, i.e. I can immediately tell whether my MMR is higher than that of another player or vice versa. This can lead to acts of condescension towards players with lower MMR in one's games.

    Considering these points, I have always understood the leagues and divisions in ranked to be an abstraction of MMR to avoid these (and other) problems. What then makes you want to add visible MMR and how would you try to mitigate some of the more negative consequences of such a change?

    Spot on and part of why visible MMR hasn't been a big priority for us.

    But, there are some advantages to it.

    • Currently, rank is the only visible indicator of skill. This forces us to link rank to MMR so they don't diverge too much because players get upset when they see people of different rank in their games even if they are actually the same skill. That's not ideal and leads to oddities with matchmaking when rank and MMR do diverge.

    • Even though we don't show MMR, players have created other sources to approximate their MMR. The information isn't very accurate, but being the only source of knowledge, it gets cited frequently when players feel there are issues with matchmaking. There's two issues here: first, it gives a false impression that matchmaking is poor for some players and second, it makes it harder to understand when there are actually issues with matchmaking since the real issues can get buried. Effectively, for players that really care about MMR, we already have most of the downsides associated with visible MMR without having the upsides of it being accurate information.

  • BlizzTravis

    Posted 6 years, 10 months ago (Source)

    But what order of magnitude are you all thinking? 10 points? 20, 50? Just trying to get a rough understanding of the overall impact it might have on gaining.

    That's a tuning point we haven't settled on yet.




Tweet