Bluetracker

Tracks Blizzard employees across various accounts.


Performance Matchmaking explained - Talk with Heroes Lead Designer Travis McGeathy


  • BlizzTravis

    Posted 6 years, 10 months ago (Source)

    Haven't watched the video yet, but I thought Blizz said they have been tracking these stats for months now behind the scenes to see what the effect of the performance based match making was going to be? It seems like they should be able to apply those already collected stats immediately if that is the case.

    Just to clarify, the system needs data, but its not individual data. It needs data for the situation you're in (ie. hero, battleground, game mode, region).

    There's plenty of data for existing heroes. It's primarily when new heroes are released or major reworks happen that the performance system will be effectively disabled until sufficient data exists.

  • BlizzTravis

    Posted 6 years, 10 months ago (Source)

    Hey Travis, I want to say that the system seems like it should help but I'm gonna hold off on my judgement for now.

    I do have a question though, I should start by saying I am a masters/grand masters player. My kharazim has a high win rate in HL I believe 60%+ but I often end the game with the lowest hero damage in the game. I will still get several kills and contribute in every other factor of the game.

    Might this effect my mmr despite the fact that I consistently win and perform well on Khara, or could it effect other players in masters or below?

    There's a few factors to it:

    First, winning is still the most important factor.

    Second, your stats aren't being compared to other players in the same game. They're being compared to stats from other master/GM tier players on Kharazim on the battleground in that game mode in your region.

    Finally, generally if you're performing well and lower in some stats, you're doing better in other stats which make up for it.

  • BlizzTravis

    Posted 6 years, 10 months ago (Source)

    Okay, I know that people generally think that Khara should be doing a lot of hero damage in 1 game. If I were to be considered by the system to be one of the best Kharazim players in NA and I have a significantly lower overall hero damage, could this impact how important the system considers hero damage numbers for all players and specifically masters/grand masters players?

    There's a subtlety here that's worth calling out:

    The weighting factor for the different stats isn't based on whether you should have a lot of it or not, but how much the difference in the stat differs between low skill and high skill players. Basically, the system is looking for the differences that correlate to high skill. If a hero has consistently high damage across all levels of play, its a baseline expectation rather than something that correlates to skill and weighted accordingly for performance.

    For Kharazim, damage isn't a primary stat that significantly differentiates skill. It's more middle-of-the-road and weighted accordingly.

    For comparison purposes, its going to look at the type of damage done by other GM tier players on kharazim in that situation, but damage itself doesn't factor all that heavily into the overall calculation.

  • BlizzTravis

    Posted 6 years, 10 months ago (Source)

    In the interview he said that draft is not taken into account, as it had a small impact on a players individual performance (or something like that).

    It's not taken into account directly as the system is looking for outputs rather than inputs. The output of your draft decisions comes through in how well you do during the match, which is what is being monitored.

  • Loesby

    Posted 6 years, 10 months ago (Source)

    Talking about outputs, what did you set as the "desired output" parameter for the machine learning? Is it wins and losses?

    We look at what statistical differences exist between players with high ratings, and players with low ratings.

    The assumption implied there is something like:

    Players who have won more in the past are likely, on average, to have played better than players who have won less.

  • Loesby

    Posted 6 years, 10 months ago (Source)

    New things we learned from this interview:

    • Talent choices are not taken into consideration by the PBMM system.
    • Unique play styles are not taken into consideration by the PBMM
    • CC time is one of the 20 stats
    • Death time is a important stat for all heroes in the new PBMM system.
    • The system will still include PRA and PRA combined with PBMM adjustment will be a 0 to 50 +/- scale.
    • If there is insufficient data for a comparison between low and high MMR players then no PBMM will take place. This can occur or is more likely to occur for new heroes, reworked heroes, less played heroes like chogall.
    • There was no mention of balance changes turning off PBMM only new heroes, reworks and lesser played heroes mentioned.
    • Team comps on either team not taken into consideration by PBMM system
    • Vehicles used not taken into consideration by PBMM
    • If you had a AFK or Feeding teammate on your team this is not taken into consideration by PBMM
    • Rank limitations are included when matchmaking a game (I assume until the time limit is reached on searching)
    • Who and How your Hero Damage/CC/Siege Damage is achieved is not taken into consideration.
    • Placements inflated uncertainty is going away. No mention of when but it seems with the PBMM patch it will be gone for placements.
    • Taking camps is 1 of the 20 stats and you only get credit for "taking a camp" if you are in proximity when the camp is taken, which I assume is globe range possibly same range as exp.

    One clarification:

    Personal Rank Adjustment and Performance Adjustment (PBMM) will be separate items on the Ranked Post-game screen. Each can be up to +- 50 right now, but numbers may change.

  • BlizzTravis

    Posted 6 years, 10 months ago (Source)

    Likewise a healer in a double support comp should probably be expected to heal less on average.

    And they do, but the overall impact on their performance metric is negligible. They heal a bit less, but make up for that in other metrics since they're doing other things instead of healing during that time.

    This is something we've looked into. To give you an idea of the statistical impact of solo vs. double support, on a scale of 0 to 1 where 0.5 is average performance, the difference in average performance for a support in a single support comp and double support comp is less than 0.01.




Tweet