Science Check: Heavy Rain October 19, 2011

Sometimes, you’re forced to make some severe leaps of logic as to just how plausible a video game’s grounded reality can be. Some things we’re willing to take for granted, like enemies will simply just carry health and ammunition supplies with them at all times, and you will be immediately able to make use of them.

But then sometimes there will be moments in gaming which skirt the bounds of reality and you are forced to ask yourself…COULD THAT REALLY HAPPEN? Fortunately for me, I happen to have a bunch of friends on speed dial with science backgrounds and when I ask them questions, they have no problem filling me in on just what reality would do in these situations.

So this is Science Check, where I take a look at the leaps and bounds of scientific logic that games have made over the years and check if it would indeed work, or if you tried doing it in the real world, you’d be totally screwed.

I was going through my shelves eyeballing various games I have in my collection this time around, looking for something that grounded itself in reality enough for me to talk about it for this article, and low and behold I came across Heavy Rain by Quantic Dream. I picked up this game on day one based upon my enormous love of the previous game Quantic Dream had made, Indigo Prophecy (Fahrenheit for you Europeans and American Director’s Cut owners). A Playstation 3 exclusive, the game has you control four different characters as you try to determine the identity of a serial killer to stop him before he kills his next victim. The game had a gripping story and intuitive controls, either on the Dual Shock 3 or Playstation Move.

One of the characters you play as in the game is Detective Norman Jayden, with the FBI. He’s assigned to the NYPD as a Profiler in order to help them out with this case. Jayden’s unique ability is his use of the ARI system, which was given to him by the FBI to use for the case. What is ARI? Take a look at this and fast forward to about three and a half minutes in.

The ARI system that Norman uses in the game comprises of a pair of VR Goggles (made up to look like stylish sunglasses) and an interface glove. They provide the player with Night Vision, as well as a full uplink to an identification and analysis database, probably the FBI crime lab. In short, they’re a forensic scientist’s dream, able to immediately identify clues in a crime scene with a few waves of Norman’s gloved hand. He can examine trace evidence from a multitude of clues including blood, tire tracks, and identify objects. It stores all this information so it can be investigated further at the user’s convenience. Finally, ARI even comes with a few minigames the user can select from when killing time off duty. That sure is a great device! Can we get a pair of our own someday? Well, I’m going to tell you. Since ARI has multiple functions in one device, I’m going to have to break down the device into its basic functions and determine if it is indeed a plausible device as shown in the game, and if it is, tell you how they could be made in reality.

First off I want to talk about the Night Vision, or if you prefer, Light Amplification, feature. Night Vision as we know it can amplify light with usually a green and black or black and white color spectrum. There are other options for seeing in the dark as well, such as infrared which reads only heat, usually responding well to body heat or starlight vision, which can also be used to film at night. ARI’s Night Vision appears to be light amplification, which is very common and is also used by the military for night missions. Heck, you could probably buy yourself your own pair of goggles that can do light amplification for around $100. However, I would be negligent if I didn’t mention night vision goggles are usually big and bulky and trade light enhancement at the cost of depth perception, at least for civilian models, but I must remind anyone reading this that ARI’s tech is not civilian, it is government, which is usually a few years ahead of whatever civilians can get a hold on, and remind you that we never actually see what ARI’s interface looks like from behind Jayden’s eyes, only in third person, so it is possible that Norman could have depth perception problems while wearing them that the player isn’t aware of. This gets a pass.

While investigating, Jayden can just take his gloved hand and do a down motion with it, which ARI recognises as a command to sweep the immediate area for clues. Once the area has been swept, any clues will be highlighted, allowing for Jayden to investigate them further. This interface is almost exactly like what the Microsoft Kinect can do. The way the Kinect works is by shining an infrared light over a whole area. The light is invisible to the naked eye but the effect would read VERY similar to how the player sees it when investigating. The Kinect then reads the infrared spectrum with a 3D camera which can only read in infrared. The 360’s software compares the motions read by the camera and compares them to the motions it is programmed to recognize, and if it recognizes the move from the user, it registers it on the TV. If there was some kind of IR transmitter inside of the ARI goggles, and a sensor that picked up the infrared light, it would be able to take real-time 3D images of a crime scene almost instantly. It could also be programmed to only recognise the interface glove, and specifically take commands only while the user is giving them with it on.

It looked like the ARI had a direct uplink to some sort of FBI crime database since Jayden could instantly pull information, rap sheets, and other data in real-time that only the FBI or other local Police would have access to. There must be some sort of wireless antenna inside the ARI that allows for the fast transfer of data. Since ARI operates in areas that do not have WiFi hotspots the immediate assumption is that ARI must have some sort of cellular modem which can transmit data directly to the FBI and receive responses back. Of course, the downside would be what would happen if ARI operated in a cellular dead zone (which are quite common where I’m from, even though tons of people live here)?

The thing that the game fudges with is that ARI would not be able to take samples of what it sees like it does in the game. It may be able to identify two-dimensional images and identify what they were if the crime database already had it on file, such as fingerprints of ex-convicts or even tire-tracks of previously cataloged tires. It would NOT be able to identify DNA evidence and attribute who it belonged to just by closely examining it. Norman would need to collect a sample of the evidence, by putting it into some kind of sample container for processing. It is not enough for Norman to simply touch a DNA sample, even with the ARI’s interface glove, because in order for DNA to be processed, it must be done in a sterile environment free of ALL contaminants. Doing a DNA test in a live outdoor crime scene would be a bad idea.

ARI also has the ability to identify trace objects, even if they are invisible to the naked eye, like plant pollen. In reality, ARI would need an EXTREMELY high-resolution camera in order to identify objects that small, since even the smallest blur around the edge of a microscopic object could alter its identification. Since there are a nearly infinite amount of variations of objects, ARI or the crime database would need to have an extremely lengthy three-dimensional database of all kinds of objects for ARI to have to compare to. They would all need to be pre-scanned into the ARI database in 3D or else the ARI system will not be able to recognise what the object was. I would hate to be the FBI intern who would have to spend every day digitally scanning new coffee mugs with the hopes that some FBI field agent would run into one while on assignment.

But once ARI has all that information recorded, what do you do with it? In one of my favorite scenes from the game, Norman appropriates a decrepit abandoned office he’s given by the NYPD to make it compatible with ARI’s specifications. He moves his desk up against the wall and pushes all the clutter on top of it to the floor so he can use the desk’s surface to help him interface with ARI. When he puts his glasses back on, he’s almost magically transported to a virtual world of his choosing, rendered by the ARI glasses. He could be underwater, on top of a cliff side, or on the surface of Mars, all to go through his clues. Imagining that the lenses of his glasses would be able to display a 3D VR environment to him, as well as use the same interface system in the crime scene to respond to his commands, ARI would need a high-end central processing unit and graphics card to render that kind of environment in real time, and they would need to be small enough to fit in a pair of sunglasses, making them have to be smaller than even they would be inside the tiniest of cell phones. It would also need to have all these environments in memory as well, or else ARI would not be able to work at all in areas with no wireless coverage, and I honestly don’t think that’s a limitation the FBI would be willing to have in such an important device they would be giving to field agents.

But the ARI is more than just for work, just like your iPhone or Android, ARI includes minigames that Norman likes to play around with. While we don’t actually get to play with more than one minigame, the interface showed it had a few games in its memory. Devices like the Playstation Vita and iPhone already have the capability to augment reality. With the use of the device’s cameras, an image can be displayed over them in 3D which the user can manipulate using the device’s controls. Gyroscopes and GPS systems allow the rendered image to stay in place even if the camera is moved or the user’s perspective changes. The Kinect also was designed to play games, and using the assumption it would have similar hardware to the Kinect, that interface could be used for gaming. ARI’s games were pretty simple, like throwing a virtual baseball against a virtual brick wall, and that should be doable even with low processing power.

The problem is that basically ARI would be a device with high-definition cameras, night vision, a high end CPU and GPU, a cellular modem, an infrared transmitter, and all have to function on a battery no larger than one that could fit in a pair of sunglasses. I don’t think batteries that small with that much power (even rechargeable ones) exist quite yet. The other thing is that these things were implied to be cheap, almost throwaway technology. In one possible ending of the game, Jayden quits the FBI. When he offers to turn in his ARI along with his badge and gun, his superior tells him not to worry, that they were expecting the shipment of an updated model and were planning on throwing away the current models anyway. Even if you could build one of these devices yourself, it would not be as cheap as the game implies, at least not with technology made today. Maybe someday some of the advances in technology we would need to make ARI a possibility will come, like the smaller high-capacity battery, but for now the goggles, it seems, do nothing.