Curse that number. Curse it all to heck. We're sorry!

Launching software at scale is hard. It's a hard problem to solve, it's a hard problem to predict, and there are hundreds of things that can go wrong no matter how many things you try to catch.

I can say we put hundreds of hours into load testing to help us understand how, why and when we fail. We put tons of time into studying our data and debating how many players would show up, what's going to be peak load etc. We spent every minute of every hour today pouring over our launch data and comparing that to our test data to see where we got the math right... and wrong.

We're pleased with how things went today, we're pleased that "most" players didn't see the things that did go wrong that we're working on and sad for the ones that did (for those that had issues, we hear you, we saw you and we care). We're ready for the weekend, we're ready for all hell to break lose and we're excited to see if we did enough testing, planning and readiness.

We don't have "better" servers than anyone else (Riot has access to the same technology everyone does), we know how hard this is to get right. I for one, am not hoping for a meltdown tomorrow. I'm hoping that everyone who wants to play, can play and we'll be working to make sure that happens.

What I do know is that we take your experience seriously and we prioritize getting that right.

*sniffle*

Fun fact. Nothing in our stack is just one "server". LoR is a highly distributed, complex set of components following a "micro-service" architecture. You'd be surprised what it takes to run. It gives us some interesting ability to scale but also increases complexity and surface area.

That said, you are correct, we are not deployed like League and thus our "clusters" do have to handle a much larger potential player base given their regional coverage than "most" League shards (some League shards are HUGE, you guys like to play a lot of League).

There were multiple problems yesterday, I'm not privy to *every* fix associated with "Error 170000". That said, the specific issue with Eastern European countries being unfortunately unable to access the game (esp. Poland) I answered in another thread. Without getting to in the weeds, there was actually a very large routing/peering problem on the "interwebs" so a large amount of traffic was not making it to our servers/services. It required engaging our global networking teams to work with the ISP/Peering partners to resolve (again this is a very generic high level explanation).

We had no deliberate blocks in our software or Geo Location systems on those countries and made no changes to them (which is a bit my team does own), but to your point we did check that almost immediately to see if we some how messed up, it would've been real easy to fix if we had.

We hear this a lot. I'm not in a position to make any commitments to you but I thought I'd let you know that we hear you! It's on the list of "things players are asking for".

No promises, but I'm hoping when the dust settles and we have a handle on things we might be able to do a Technical Blog series on some of it. Personally I want to make sure we've had enough time running it in production with real world player traffic so we can understand every bit that works well and especially what doesn't.

If you've been following Riot's technical blogs about service architectures and container based development over the past few years you can get a good idea however of what the building blocks are.

I've worked on many game launches in my career (and many failures). I was pleasantly surprised last night when the team was able to go home and get some sleep. It was harder to get people to go home however, we're all pretty excited :)

Here's hoping we can do the same this weekend. I was even able to get a few hours in last night just playing the game and relaxing. Won my first competitive match, then proceeded to lose my first expedition and cry into my Freljord/Demacia expedition deck that just didn't work.

Oh man, the data from Previews was EXTREMELY helpful. Prior to the preview events we had simulated load tests but there's a big difference between how we predict load will go (using just internal Riot playtests of a few hundred players) and how having tens of thousands of players playing the game and doing things will go.

It enabled us to dial in things like the ratio of games starting (and ending), what types of games players will play, how frequently they'll be making new decks and how often players would be making card purchases. One of my favorites was we were way off on how long it would take players to make expedition card picks (we were way too aggressive in our load tests).

At the core it let us understand/have a good idea of how many players in actual games we could support with a given amount of hardware. Ironically we were too pessimistic (i.e. cautious) in our early estimates.

It's safe to say the the "relative" smoothness of yesterday was directly due to us being able to run those early preview events and then being able to to turn that into better load testing.