This is huge. It's incredibly huge. Going to quote my grinninggoat post on this:

You should look up iksarhs by his reddit user name and read every single comment he's written in the past 60 days. He's on a binge right now and you'll catch most of that, but you might miss some, and he's commenting on non /r/arenahs threads and explaining the role of "micro-changes" - which really might not be that micro at all given what he's saying.

He explicitly states that they're moving to a new means of balancing the arena that doesn't involve the power level of cards in expansions, but does involve making "hopefully" "small" changes to availability, but (suggested) on a very, very widespread basis.

TL/DR you're not going to be able to accurately assess class power level anymore. You're just not. His explicit goal is to balance class power level by making availability changes that are non-transparent, 'small' enough to be 'not something you should worry about as you draft for any one specific card', but yet widespread enough to balance class winrates.

One thing he hasn't considered is that the perception of class balance has more impact than the actual class balance. If the balance is opaque and hidden, confirmation bias and uncertainty and many other cognitive forces will lead the "agenda setters", streamers and etc, to stick with initial class balance perceptions based on bad data.

To say that perception of class balance is not considered isn't exactly accurate. Perception of balance is important in constructed as well, but has a particularly interesting effect on data in arena because of our matchmaking system. Actual win rate data from constructed tends to inform player behavior more than it does in arena. If Evolve Shaman is actually strong, data will reflect that and the behavior of people monitoring that data will adjust. In arena, if Mage is perceived to be strong by players monitoring data, that perception actually leads to Mage having a higher win rate because players that monitor data are usually on the higher end of play skill. I often discuss this topic because I think it is interesting, but it is more a reality we have to work around rather than a problem to solve. This is all just to say that we do consider balance perception when making decisions.

In terms of assessing class power level I would argue that the data from places like Heartharena has always been there. Determining what classes have a higher win rate than others has been less a fun puzzle to solve and more a web page with a correct answer. When a new set comes out with a 2X bonus to those cards we are unlikely to make adjustments straight away to address class balance and I think that will be a great time for invested players to form opinions on the cards of the latest set and determine what classes stand to benefit most.

In arena, if Mage is perceived to be strong by players monitoring data, that perception actually leads to Mage having a higher win rate because players that monitor data are usually on the higher end of play skill. I often discuss this topic because I think it is interesting, but it is more a reality we have to work around rather than a problem to solve.

There are slightly more advanced approaches you could take to analyzing the data to resolve this issue. For example, instead of simply looking at the overall Mage win rate, divide each player's average Mage win rate by that player's average win rate across all classes, then take a weighted average of those results (weighted by number of Mage runs by each player) to give an overall "power index" for the class.

Presumably you have thought of this already, and the operation would simply be too difficult to perform, or is too calculation intensive to be feasible, across a dataset as huge as the full Hearthstone arena results.

I don't think any external source, such as HearthArena, would be able to perform this analysis accurately because, as you suggest, the players who contribute data to external websites tend to be self-selected as significantly above average in overall win rate, and this might skew the result.