A new vision for gaming: Tobii eye tracking technology

Cameras which are controlled by thumbsticks have always been my nemesis. For whatever reason, I’ve never been able to master this seemingly basic function, and it has left me out in the cold where a number of games are concerned. I did a laughably poor job of controlling the camera at my Mario Odyssey hands on, struggled with the camera the whole way through The Legend of Zelda: Breath of the Wild, and Overwatch? That game is out of the question. Over the years, I’ve simply come to accept that there are an ever expanding set of games which I will never be able to fully enjoy, appreciate, or play with any real level of skill. Shockingly, a half hour demo at E3 changed my world.

The Swedish company Tobii has been developing eye tracking software for years, though they found themselves restrained due to the bulky, cumbersome size of the sensors and processors their technology required. Fortunately for both Tobii and gamers such as myself, technology has recently caught up and scaled dramatically down in size, making Tobii eye tracking technology standard in most Alienware laptops and monitors, and allowed them to expand into other brands, including Acer. So what exactly does this technology do, and how well does it work?

I sat down with a laptop and a copy of Agents of Mayhem, warning that I am categorically terrible at these kinds of games and threatening bodily harm to anyone who teased me about it. Controller in hand, I spawned and stepped onto a busy street. Someone moved on the far left of the screen, drawing my eye…and the screen panned smoothly to the left along with my gaze. It wasn’t a sudden, jerky, or even a dramatic motion; if I were playing a shooter and needed to quickly turn my field of vision 90 degrees to identify someone shooting at me, the thumbstick would still be the way to go, but this… this was easy, natural, and for someone who has played a lot of virtual reality games, very familiar.

I sent my character jogging down the street, looking left and right as I went, my course firmly decided by my directional thumbstick, never wavering because of my wandering eye. Upon being prompted, I stopped and stared at a random NPC. At first he was oblivious to my scrutinizing gaze, but after a few moments he took notice, sneered, and threw up his hands in a universal “What’s your problem?” gesture. Tobii has partnered with a number of games to fully integrate its software, including alerting in game characters to the fact that you are staring at them. Assassin’s Creed was one of the first games to fully embrace eye tracking technology, and has this partnership with Assassin’s Creed Origins. Today, Tobii boasts a rather impressive roster of games, and some of the most recent announcements include Agents of Mayhem and For Honor.

Stepping away from the PC, I put on my skeptic hat as I donned a VR visor outfitted with eye tracking lenses. How much could this really do, I wondered, since VR already used the headset and sensors to track what I was looking at? At first, there was no difference. I saw nothing out of the ordinary and failed, rather miserably, at grabbing virtual rocks and throwing them at virtual bottles. After a few pathetic attempts, I was told to find the ‘magic’ purple rock, grab it, and throw it at the bottles. I couldn’t, I explained, because the purple rock was located far away, out of the bounds of the VR cube. “Just look at it and grab it,” my handler instructed.

I looked, I grabbed, and the rock came to me. Then, as the name ‘magic purple rock’ implied, it magically hit a bottle when I threw it. Then it hit the next. Then the next. I looked, I threw, and the rock went where I wanted it to, even if my throw was pathetically weak. I switched to an outdoor pool scene where I used an oversized mounted tablet to order a beach ball and one giant inflatable rubber duckie, which were conveniently and immediately dropped off by a pair of droids. Tossing the ball away, I followed Obi Wan’s advice and used the Force, looking at the ball, reaching towards it and pulling the Vive wand slowly with my finger, causing it to miraculously return to my virtual hand.

Perhaps the highlight of the demo was when I swapped places with my friend, allowing him to go for a spin in the visor as I furiously took notes. Making his own sound effects, he casually tossed a ball away, then pulled it back. At one point, our handler pressed the button, and as my friend threw the ball, it drifted slowly away, refusing to return to my friend’s hand as he called it back. His face fell, as if a puppy had just refused to let him pet it. “It didn’t come back,” he said forlornly, pawing at the air in front of him.

“I turned off the eye tracking.” The simple confession resulted in an eruption of laughter while also proving a point: This technology is not psychosomatic. It’s not something you notice only if you’re at a certain level, it’s not something gimmicky or awkward. It feels right, it feels natural, and you notice the instant it has been turned off.

This impressive technology feels deceptively simple. This is exactly what looking around inside a game should feel like. That brief hands-on demo left me clamoring to get an eye tracking bar of my own, and has given me the courage to explore a whole subset of games which I had previously thought of as closed off and beyond my skill level. Tobii provides more information about their eye tracking technology and their products at their website. Have you gotten to try this technology yourself? Please share your experiences in the comments, I’d love to hear your thoughts!

See below for our list of partners and affiliates: