Why Do Online Only Games Have Such Rocky Starts? September 12, 2016

Anyone who downloaded Pokémon Go when it first launched probably found themselves unable play it most of the time. The application was fundamentally tied into its online component, which was bogged down for weeks by the sheer number of simultaneous user requests. It would take a while for Niantic to iron out all the issues with the game on both the client and server end, but they made the adjustments needed and now the game is (barring the occasional odd crash) pretty playable.

So why did the game have such a bad launch period?  Well, Pokémon Go is hardly alone when it comes to issues with games requiring online services during their launch.  Heck they’re hardly tied to the smartphone platform, because triple-A PC titles like Sim City and Diablo III were downright unplayable at launch, rightfully angering fans who bought them at release.

So why is this happening?  It can’t be for lack of dependable testers, I know for a fact that millions of people all over the world were willing to test Pokémon Go before it launched, and those players could have been used to test the game before it was released. While I’m not totally certain of the exact number of testers the game had during development I can certainly confirm they didn’t accept everyone who applied because I applied and I know for sure I wasn’t picked!

After all these bad game launches, a lot of frustrated players have asked why weren’t these games properly tested before they launched? The truth is they had been tested, but after their poor launches it’s clear they weren’t tested adequately.  How could this be?

I spoke to a friend of mine who had a passing understanding of Google’s online testing methods who shared his thoughts with me. He doesn’t work for Google or Niantic, but he does work in the tech industry and he is familiar with a lot of their testing methods.  While I can’t confirm Niantic (or any other online game developer) uses this method to test their games, his information did make these day one problems gain some sense.

Games are tested in controlled environments before they’re released to the public. We call that QA Testing for Quality Assurance.  You don’t have to read further than The Trenches webcomic to see just how bad QA Testing can get, but what about games that require an online component to function?  Those are tested in what are called “proportional” circumstances. Just like the Mythbusters will test theories in smaller scale conditions before replicating a myth in full size, online game developers traditionally test their games in limited environments with fewer devices. The idea behind it is that if a server with limited bandwidth can remain stable under a proportionally limited test case of players, their servers can handle the expected amount of end users at full bandwidth when the game is finally released.  It’s believed that testing online games proportionally during development is the best possible testing method.

After he finished giving me this information I told him, “Wow after Pokémon Go‘s launch was such a disaster, they must be really rethinking that flawed test method, aren’t they?”

You would have thought I insulted the man’s mother if you could have seen the face he gave me after I made that statement.  After telling me in no short order that there was nothing wrong with that testing method (ignoring the fact it failed miserably when the final games were brought online in many different cases), he told me that I had no idea what I was talking about and I just looked at him like he was completely out of touch with reality.  If proportional testing was the indeed the method Niantic used to test Pokémon Go or EA used to test Sim City, and that testing method had worked, Pokémon Go would not have had the plague of crashes, login failures and random quits for three weeks after it launched, and Sim City would have been playable.

When I grew up testing games on the PC, developers would traditionally hold a “stress test” period where they would get as many simultaneous users as they could to see if their game would break or buckle under the strain of the number of users testing it. Sometimes they would start with fewer testers and add more as time went on, but by the end of the testing period they would usually offer everyone they could the game’s online beta test client as a free download. This testing method is still being used for games like Gears of War 4, Titanfall 2, and Halo Wars 2, whose developers have all offered open online stress testing this year.  From a practical standpoint, this seems like a far more fruitful method of testing a game toward the end of its development cycle.  By offering your game’s test client for free to everyone with even a passing interest in the game, developers can better predict player numbers as high as or higher than a game could expect to get at launch.  It can also help investors shape sales expectations and ensure a smoother launch period.

I sought advice from other peers of mine familiar with the tech industry as I was writing this article and they had plenty they felt needed to be added to this discussion. They argued that hosting an open beta test for a game like Pokémon Go would have been a bad idea, since the normal spectrum of bugs and glitches that players could experience during testing might have had the side effect of giving testers a poor initial impression of the game, and make them lose interest in playing it when the full version was released. While I understand some players could accept this arguement, I do not.  It is reasonably accepted amongst gamers that test clients could have their fair share of bugs and glitches. In fact, every EULA I’ve ever read for a beta game references this, so players are prepared for it. However, nothing turns potential players off a game more than a glitchy launch, and I would argue that it would make more sense to have bugs show up during the game’s test phase then to hold back testing and discover your game has problems only when the game is in the hands of paying customers.

I have not talked to anyone from Niantic and I’m no more familiar with insider information about recent Pokémon Go developments than anyone with access to the company’s Twitter feed. Pokémon Go earned millions of dollars of income in the first few weeks since it was released. There’s no telling how much more money Niantic could have made if they provided a stable platform on day one.  Perhaps if they had done a stress test they would have been better prepared for what they were in for but I guess we’ll never know for sure.

Gaming, Star Trek Style: Starfleet Academy September 8, 2016

Today is the fiftieth anniversary of the broadcast of the very first episode of the groundbreaking Star Trek television series.  Not too many classic staples of science fiction get to say that they’re still going strong fifty years later, but today Star Trek gets to say that.

On this website we’ve talked about our past with the franchise, particularly when it comes to gaming. This is after all a gaming website.  Earlier this week, we discussed a Star Trek game that never came out, Secret of Vulcan Fury. Today, we want to talk about the game I played that made me aware of Secret of Vulcan Fury in the first place, the PC flight sim Star Trek: Starfleet Academy.

I first became aware of Starfleet Academy after completing Star Trek: Borg for the first time.  In 1997, I was becoming an enormous Star Trek fan. After being one of the first tourists to check out Star Trek: The Experience in Vegas, I found myself enjoying reruns of Star Trek: The Next Generation and post-season 3 episodes of Star Trek Voyager.  However, even though I had a decent PC at the time, I didn’t have much money to spend for new games, due to the fact I was still in grade school. One day, after celebrating my birthday, I saw a copy of Starfleet Academy for sale at Circuit City and I happily put down some birthday money for a copy of the game and its official strategy guide.

I installed the game as soon as I got home and was greeted by this trailer once the installation concluded, and I couldn’t imagine a better way to fire myself up.  I watched the game’s live-action introduction cinematic and got through the first simulator mission.  With the first mission complete, I saved my game and selected what I thought would be the next simulation mission only to find myself watching another live-action cinematic. The game was putting a heavy focus on the personalities of the crew I would be directing through these simulations and it turned out I would be making major decisions for these trainees as the school year went on!

Starfleet Academy‘s live-action component was no small undertaking. Heck, these FMVs were so detailed they deserved to be their own game and to this day I’m shocked they haven’t been rereleased on DVD.  They included a great cast of original actors for the training bridge crew, and William Shatner, George Takei and Walter Koenig all returned to their iconic roles of Captain Kirk, Pavel Checkov, and Mr. Sulu.  These three served as the experienced mentors they needed to be, a perfect fit for a game focused on the next generation of Starfleet.

The game’s story was so fleshed out it was later adapted as a novel by Diane Carey.  I actually used the book as the basis for an oral presentation in my sophomore English class. If you want to know how well I did on the presentation, most of my peers put the class to sleep droning on about their books, but the class enjoyed my presentation, as I weaved them a complex story about a group of students dealing with a difficult year while tragedy strikes.  My peers liked it so much they wouldn’t let me step off my podium until I told them how the book ended.  I’m pretty certain I got an A.

However, unlike Star Trek: Borg, which was an interactive FMV game with a fairly linear narrative, decisions you made in Starfleet Academy could have an enormous effect later in the game. Heck, halfway into the game you could make the wrong decision in a cinematic and lose the whole game.  Cinematic decisions could also affect your simulator scores, which I later learned would have an effect on the game’s ending.  My strategy guide had no information about these cinematic sequences, only the simulations, so I had to trust I was making the right decisions as I went on.

I was never much of a flight-sim player growing up and this game’s simulation had a really steep learning curve. In fact it took me months to figure out how to complete the final mission in the Alshoff campaign. After what must have been the hundredth attempt, I was finally able to beat the mission.  Once I had that level beat, I was able to complete the rest of the game in less than two days.

If you want to know my favorite missions I fondly remember a mission based on the plot of The Wrath of Khan, a mission taking place in a nebula (where sensors were limited), the final mission and of course the mission where you actually got to play the Kobiashi Maru no-win scenario. I totally won that.

In fact I remember the night I beat the twentieth mission and saw the game’s credits roll. I was puzzled as hell that the game was over because the strategy guide listed strategies for one more mission. That’s when I realized that the final mission would only be playable depending on if you got the game’s best ending.

What can I say about getting this game’s last mission?  To this day I don’t think anyone has come up with a strategy on how to unlock it, which is a real shame because it puts the player in control of the real USS Enterprise NCC-1701-A.  The only advice I could offer is to make decisions in both the simulator and the cinematics that keep your team’s scores as high as possible, and devote as much time as you can to further studying the McClanty during that story arc in the last few missions.

In my case, I may have lucked into the best ending because of some kind of bug. After the game’s credits finished rolling, the game immediatly started rolling one of the earlier cinematics. Apparently I had been sent back to replay the last three or four missions. I figured further study of the McClanty was important to unlocking the last mission so I decided to choose to investigate the McClanty as much as possible. Of course this is not a guarantee to earn the ending because your crew still needs to have high simulator scores, and tons of different variables can affect them.

Regardless of if you can unlock the last mission or not, this game still has the best presentation of the Kobiashi Maru no-win scenario, hands down. If you want to hear more about my thoughts on it, you can read them here. I actually played this game before I watched most of the Star Trek feature films. Imagine my surprise when I saw Star Trek II: The Wrath of Khan on VHS for the first time.  I felt like I was experiencing the fandom the game’s developers had for the franchise in reverse.

I know Interplay later released an expansion pack for Starfleet Academy called Checkov’s Lost Missions but by the time I finished the original game I could not find a single software store selling it, and to this day I’ve never played it.

I’m afraid I’ll have to end this article on a down note. After talking all this time writing about this triumph Interplay published I’m afraid to say that earlier today I’ve just read that Interplay is planning to sell off their assets. My question is what happens to Run Like Hell?  I think Bawls should get it.