Bluetracker

Tracks Blizzard employees across various accounts.

A look at Arena offering rates from before Hallow's End, during Hallow's End, and right now. The Pre/Post Hallow's End rates still appear wrong.

Note: Due to a variety of behind the scenes alterations from Blizzard, as well as in general recording errors + statistical variance, its impossible to determine a 100% accurate offering rate. My point is not to show a 100% accurate offering rate, but rather to show the change from Pre-DCA (I'm going to abbreviate Hallow's End to Dual Class Arena/DCA) to DCA to post-DCA , as well as show that the stated Blizzard offering rates are likely incorrect. I'm also going to add my own theory as to what the actual rates are, not as a statement of this being 100% but as a possibility of what could be correct. As a request: I'm more than willing to answer any questions about the data and refine my process if you disagree with what I did, but before being excessively critical, please consider what I'm trying to do. Just because I'm wrong on 1 small thing will not invalidate the larger picture.

With this said, while I've got very little background in stats, I'm still going to take a look at the observed data and make my own conclusions from it. Understand this is my personal opinion/theory on whats going on and not a statement of 100% fact and very well could be wrong. I'm going to make definitive statements, because its what I believe, but it very well could be incorrect. If you have any other theories, please feel free to post. I'm happy being wrong, I just want to figure out what's going on, which is why I went through this effort to gather data.

Also, I've included my own chi-squared calculations for each section. /u/Nerdtron93 has been busy and I didn't get much of a response from /r/askstatistics so I may be (read: almost certainly am) interpreting this incorrectly somehow, so any advice on interpreting this correctly is appreciated. I'm also aware its not proper to use P-values the way I am, but again, I'm not trying to show something is 100% correct. I'm just trying to show that something is more likely than another scenario in general. My Null hypothesis in general is "Is there a 1/1000 chance my data matches Blizzards? Is there a 1/1000 chance my data matches my own theory. If no, why does my data not match?" end note

Intro

Previously, I've made posts about how Blizzard's stated Arena offering rates were incorrect and how in Hallow's End the rates changed. Many people took issue with how I collected data, my sources of data (even if just supplemental), and the sample size, so, with the help of /u/Nerdtron93 (the same statistician who took issue with my Hallow's End data), I've refined how I collected my data and calculated the offering rates. The pre-DCA data is newly gathered, and the DCA and post-DCA is gathered under the same methods. FWIW: The numbers are more accurate, but they point to the same conclusions I made before about the KFT bonus not being properly implemented and the Class Bonus likely being non-existent.

As these posts are long because this is reddit and if I leave one tiny thing out I'll get jumped on for it, I'm going to post the data first and go through my methodology second. I'll make a note for when the boring stuff comes up.

#1: The pre-DCA Arena offering rates, focused on Mage.

Line 60 give the observed offering bonus. Line 66 is the bonus taken from Blizzard's own rules, assuming a multiplicative bonus.

First thing's first: Over 48 runs, Mage class minions are offered at 97% of Neutrals, when its stated it should be at 200%. This is not sample size, this is not minor recording error, this is not micro-adjustments or anything behind the scenes, this is there is no offering bonus being applied to class minions. Looking at spells, its similar, 190% observed bonus instead of the expected 275%. The KFT bonus also appears much lower than the stated 200%, although Mage KFT spells showed up 2.37x compared to normal spells. I'm inclined to believe this is a random outlier because of the data, but I can't say for certain.

Chi Squared calculations, first tab. The top chart is if it matches the Blizzard offering rates, which turns back a 93.5. With a DF of 6, to get p-value < .001, you'd expect a 22.458. Obviously, that far away, its incredibly unlikely that Blizzard's rules would match what I observed.

Bottom chart, I calculated it under the assumptions that the class bonus = 0 and the KFT bonus is 1.6. While it returns a 26.83, still higher than the p < .001 of 22.458, most of the number comes from the Mage KFT spells being so off. In any case, the p-value is much smaller than compared against Blizzard's, so I think my scenario/theory would be much more likelier.

TLDR: Class Bonus appears non existent, KFT bonus appears to be less than 2 and around 1.6-1.7, but inconclusive due to the outlier that is the Mage KFT spell bonus.

#2: The DCA Offering Rates, focused on the Druid/Mage combo.

Line 62 for the observed offering rates, 65 for the expected rates.

At /u/Nerdtron93's recommendation, it was best to focus on one class combo rather than split up the classes into their own separate rates (see the other tabs). I don't have Heartharena profile data for this because of the limited time for DCA, so instead I used Dr. Stein's DCA streamer spreadsheet to find streamers who did Druid/Mage and took the data from their stream. I have the streamer data written down if people really want to see who drafted what, but Twitch generally doesn't archive streamer videos for long. Also, I did not differentiate between if Druid or Mage was the first pick, which may have some influence on the data.

Looking at the class cards, there's a huge jump from the pre-DCA cards offered to DCA cards offered. This is why I speculated that the offering bonus was fixed. To put this in perspective: in 48 pre-DCA runs, Mage got 353 spells and 116 minions. In 48 runs split with Druid, Mage got 367 spells, and 157 minions. Even though Mage was splitting its picks with Druid, it got offered more Spells/Minions across the same 48 runs that it did without splitting. That, to me, is real strong evidence that something changed in the offering rates, that something happened during this two week period where class cards were offered more.

The other really weird thing is looking at KFT neutrals. KFT neutrals at 134% is absurdly low. What makes this even more absurd is that, if you compare KFT Class Minions or Spells vs. non-KFT Class Minions/Spells, its returning a KFT bonus of 1.8-1.9 (and higher because of how few Druid minions were offered). I have no explanation for this. I went back and double-checked my data just because I saw how low this was, but the data was the same. On the right I included snapshots of the data 16, 32, and 48 runs, and in each snapshot it was real low. While some other data outliers regressed toward what you'd expect, KFT neutrals remained real low. Also, if you compared my earlier more flawed data (individual class tabs), none of them reach anywhere near 134 over a decent sample size. I genuinely have no idea what to make of it.

Chi Squared data, second tab. Top is from the Blizzard expected rates, bottom is compared to my theory values from pre-DCA. Top returns 76.12, Bottom returns 307.13. A p < .0001 for this would be 29.588. Obviously, neither one seems likely at all to happen. Again, the outliers are Neutral KFT cards, as well as Druid Spells and KFT spells in the top. I have no reasonable explanation for the Neutral KFT cards, and the Druid stuff might just be an outlier.

TLDR: There is a large difference between class cards pre-DCA and DCA, possibly being from the class offering bonus being increased but there may be other influences. KFT neutrals are extremely weird and don't match the class data for the KFT bonus, so I really have no idea what happened here other than something happened.

#3: Class data post-DCA, most classes updated.

Offering bonus calculations start from K so that I can continually add to the list and it'll automatically update the values. Starting from K11 is the observed offering bonus. I'm going to continually update this until KnC comes out, so if anyone wants to observe the changes over time feel free to bookmark this.

You can click class by class, and there is variance class by class, but its back to the pre-DCA data where just by looking at it, its clear there is no class card bonus implemented. Outside Paladin KFT spells and Rogue KFT Spells (where, due to only 1 KFT card in the pool, there's a ton of room for variance that will likely drop as more data is gathered), there are no instances where any observed offering rate comes close to the stated offering rates of Blizzard. Additionally, the highest KFT Neutral bonus is in Hunter, and even then its only 177% instead of 200%.

Of note: Rogue KFT minions are offered absurdly low. They're practically offered the same rate as normal Rogue minions from the data I've observed. I observed this too in my first flawed sets of data. While still a smallish sample size, this stands out to be as real weird.

Chi-squared values, third tab. Because Mage and Rogue were at 16 runs, I did it for both classes. Compared to the Blizzard rates, it returned chi-squared values of 32.29 and 58 for Mage and Rogue respectively, above the 22.458 for p-value < .001. Comparing with my theorized 0% class bonus and 160% KFT bonus, it returned values of 5.12 and 13.11, under the 22.458 value, meaning that for once, I can reject the null hypothesis, namely because there are no observable outliers here. Oddly enough, this only comes from my smallest set of data as well. But again, asssuming my calculations are correct, this should be solid evidence the offering rates are flawed.

TLDR: Its extremely unlikely Blizzard's information about class cards/KFT cards is correct, and its plausible/likely that my theories are, if not correct, within a reasonable boundary of being correct.

Conclusion

Really, I said it all in the points I made. I've gathered a large sample size of data, more than sufficient to show my point (that the rates are wrong), and it is backed up by other sources of data as well, that even if not 100% accurate, point to the same conclusion that things are not what they should be in Arena. In DCA, things changed, much more than just adding a second class into the pool, and it looked to me like things may have been fixed, but it reverted back to its flawed form after DCA ended. At this point, I think its clear that the stated rates are wrong. I have my theories about what the correct rates are, but those are just theories and guess work to try to figure out what the actual problem is. All I really want is for the offering rates to be fixed to what they should be. DCA was a glimpse into what that kind of Arena could be, with answers and board states where you have access to responses.

Boring stuff, methodology and answering questions I expect people to have

1: Why not contact Blizzard? I have, more than a month ago, and a few times over the last month, the last interaction was them asking me for cards to compare, and I never got a response one way or another on that. When I asked about the change in DCA or return to "normal" after DCA I didn't get a response.

2: Why not ask Heartharena for data? HA's response to someone asking this same thing. FWIW: I did, he's busy, is likely busy preparing for KnC, and its no skin off my back to take ~20-30 minutes/day to chart some numbers while watching TV.

3: How do you collect the data? I take a look at the drafts, chart each common card offered into its proper offering rate category, and tabulate that data into the spreadsheets. This is much easier to do using HA profiles which list the draft out from card 1-30 than streamer data where I have to slow down or wait on streamers to make picks, and I've got it down to the point I can track a draft in 1-2 minutes tops now. I chose commons because it has the largest sample size and because Rare+ doesn't impact commons. Pre-DCA and DCA was focusing on a specific class to get a sufficient sample size because of people who were concerned with this.

4: Why do this? I like looking at numbers and if I see weird things I'll tell people about it. With the pre-DCA data, people did not trust my other data, so I regathered it because people won't believe anything I say if part of what I say is wrong. I still expect that in this post honestly. The time isn't that large of a concern because, as mentioned, I do this while watching programs and can multi-task. Honestly, I just want the community to be informed.

5: How did you do the offering rate calculations? I previously just compared ratios of categories to neutrals. This time, I got a weighted average for each card (out of 100%, what % each different offering rate makes up), set n=1 since that's how Blizzard does their comparisons, and compared it to the base number of cards in the set assuming no offering bonus. The difference is the observed offering bonus. For the expected chi squared values, I took the Blizzard stated offering rates, multiplied the base cards by the offering rates, found the weighted average, and multiplied the weighted average by the total number of cards I observed.

6: What are the Blizzard stated rates? Here, check Arena rules from 8/31. Note that at the time, the Ungoro offering rate was +50% instead of +100%, but its been stated by Blizzard repeatedly the KFT bonus is +100% and there's been no announcement of a change anywhere from Blizzard. There are 7 separate rates (KFT Spell/Weapon, Class Spell/Weapon, KFT Class minion, Class minion, KFT neutral, Neutral, and Basic/Classic). I'm assuming that when Blizzard says +100%, that, if N = 1, that's N2, and for spells, N2.75 to represent the increase, and that the KFT bonus is an additional 2x multiplicative.

7: How does Mage have 15.5 spells? Flamestrike has a 50% reduction in offering rate, so at /u/Nerdtron93 recommendation, I changed Mage from 16 to 15.5 spells. I also removed cards that are not available to draft due to being C'thun cards or purged for being too bad for a class. While I don't think this is correct, and that from my interpretation that other cards in the offering bonus will get the boost from that card not being there, I'm going with Nerd's recommendation here.

8: Why do the spreadsheets look different? I have never used a spreadsheet until the last month, never had to. So, I was unaware of the formulas or features, which slowly I have became more aware of and are using more.

9: Why are there ? or skipped cards in the post-DCA data? After DCA ended some runs were still counted as Dual Class runs, so they don't properly appear in the Heartharena last 10 run archives, of which those ? represent. The skipped cards refers to if in a draft someone made a quick pick, and the other two picks did not register. I skipped those while recording data because it might introduce bias. If I skipped a large number I made a note of that as why some data might be weird. Ideally those runs should be thrown out, but considering I'm dealing with low sample sizes, I'm using what I can get.

10: Why is this so long? Have you been on reddit? On the internet? If one small thing is wrong, people will jump on you and cast everything you did as incorrect or invalid. I'd rather document my entire thought process so people can point out if something I did was wrong, or to answer questions people have in advance, than leave it out and have to answer people asking about this.

Any more questions feel free to ask, if its something common sense I'll update it here.

Tweet