The day Steam stopped
On the 15th of December, the unthinkable happened when a storm hit the Seattle area, knocking the power out for one million people and putting Valve’s Steam servers out of commission. With the authentication system for Steam unavailable, helped by the fact that Steam’s Offline Mode only works under certain very specific circumstances, Steam users weren’t able to login and run Steam-based games or applications anymore. The authentication blackout lasted for about 20 hours, according to Valve; how did this translate in the number of online players for Steam-based games?
The graphic above shows the average daily presence of each game under range of top 100,000 players. Differently from most graphics featured on this website, there’s no weekly average, so it’s possible to see the weekly variations – stronger on the weekends, weaker during weekdays – and how the drop actually happened.
While the absence of Steam was surely a great hit, it happened during the day, so the impact was distributed between December 15th and 16th (a Friday and a Saturday).
One of the interesting aspects to notice is that the fact that Steam wasn’t working didn’t immediately reflect on the number of players online on other games. That is, Steam users didn’t switch to another online game – either because they don’t have them installed or didn’t want to – and simply opted to wait for the system to be back.
While it’s curious to think that a system of this magnitude didn’t have authentication servers deployed somewhere else, the situation was solved fast enough for the incident to be no more than a small bump on the road for Steam players. Many of them didn’t even notice that the system was offline; users that were already logged when the servers went down were unaffected by the blackout.
Valve has addressed the incident briefly on a news post, promising to “[take] steps to make sure that when the next storm like this hits our area 15-20 years from now […] people will still be playing games on Steam“.
December 27th, 2006 at 1:48 pm
Just to add a more personal note to this Steam debacle, I must say that, the way it’s implemented right now, Steam’s “Offline Mode” is a joke. It doesn’t work unless the system is already on, or you’ve taken precautions before trying to go online (like doing a backup of a random file nobody told you about before) simply because failing to login once (like, when the servers are down) disables your ability to go Offline. The option also never shows up by itself unless you disable your network.
This whole incident was pretty rare – an exception. It’ll take a while for something similar to happen – 15-20 years, if ever, like Valve says on their site. But to me it’s a painful wake-up call to what can happen when an application is so dependent on some online authentication, no matter what you want to do with it… like run a game locally or run some tool.
You know one of the reasons I say this? Well, just one week before this happened was the end of my semester at my college (4th semester, “Digital Interface Design” bachelor class). Well, we had this class where we had to complete some work that was based on the Source engine… a “virtual tour” – basically, we had to create an HL2 map. Several different groups had several different projects, or maps, to create. The last day for the assignment (a Friday) was the day people worked like mad to finish all the content of their maps – textures, models, all that – and put it together using Hammer. It was due to be delivered at night – on our last day of classes. So on that particular Friday, everybody from my class was pumping mad blocks at Hammer and compiling BSP’s like there’s no tomorrow. Again, this was a week before the blackout that hit Steam’s servers.
Now, can someone picture what would have happened if this disaster struck one week before? We wouldn’t be able to use the last day of school to finish the work, we wouldn’t be able to deliver it, and the teachers wouldn’t be able to judge it – simply because the Steam authentication servers were down (and the “Offline Mode” was busted) and we wouldn’t be able to run the game or the tools locally. It was the last day of college and grades were due out on the following Monday. It would be hell for the students, teachers and the school.
Comparatively, this issue is just a small problem in the greater order of things. I mean, people were killed or had their houses destroyed because of the storm that hit Seattle. It’s not Valve’s fault either… they’re not God, they couldn’t have predicted it.
Regardless, I still believe it’s still Valve’s responsibility to make sure the games and tools people purchased work – either by having a real Offline Mode, or by having a second authentication server somewhere else. Ask yourself what would happen if authentication servers for a major Credit Card firm went offline on Christmas days – say, Visa credit cards. You’d be at EBGames or Gamestop or something buying your Wii after spending 3 days on a line, and suddenly your Visa wouldn’t work anymore. Would you be happy? Who would you blame, the random storm that hit their servers, or the fact that they didn’t plan ahead?
I’ve never seen that happening, so that’s why believe Credit Card companies have took this into consideration – probably using secondary backup servers, I don’t know. And how Valve never thought of something similar, specially because Offline Mode doesn’t work properly, I don’t know. But I really hope they take this into consideration in the future. If they want online purchases to be as attractive as brick and mortar purchases, I don’t see any sense in crippling software this way.
December 30th, 2006 at 10:21 am
I wasn’t aware that offline mode didn’t work (my laptop having died a few days previously). Talk about adding insult to injury: it was also a problem during Steam’s last major downtime, nearly two years ago! That’s the second weakness that ought to be long-resolved by now.
December 30th, 2006 at 2:23 pm
Indeed. Offline mode is a (badly) shoehorned in fix for a problem that was never really thought about properly in Steam’s development. It’s scandalous that it’s still unfixed, and it makes me kinda mad that they’re off developing very peripheral new features (like guest passes) while a lot of the core functionality is pretty flakey.
The solution is annoyingly obvious. Steam should be designed to work offline, but to take advantage of the Internet when it’s availiable. In the development of reliable software, you should always assume resources are going to be unavailiable. The Steam client assumes the Steam service will always be perfectly reliable. It’s a very bad assumption.
January 6th, 2007 at 1:52 am
I remember this, the SERVERs window of steam no longer worked, so you had to go in game to see if they were still up. They were -of course, but the VALVe Anti-Cheat system (VAC) was offline. Naturally, more than a few hackers decided to take this opportunity to rape most of the now unprotected servers. Furthermore, nothing was stopping banned accounts from hacking while they could.
Note: this was an outbreak in CSS.