Eye tracking hits a level of maturation – Tobii EyeX eye tracker review

There hasn’t been a lot of innovation in the peripheral input device market beyond higher DPI mice. Sure, we had trackballs, gun-shaped mice, the Novint Falcon, and the SpaceOrb 360, just to name a few, but none of these added much to the equation. One idea that has had limited, but not well-integrated, success up to this point is eye tracking. Marred by cumbersome head-mounted eye pieces and jittery movement, they never found wide adoption. A company named Tobii is looking to change all that with their Tobii EyeX eye tracking hardware.

Tobii has been tackling eye-based input for special needs, as well as eye tracking for use-case analysis, but could this same technology bring something new to the table for the gaming world? Before we get into specific gaming uses, let’s talk about the technology and concepts powering the Tobii EyeX.

Setting up the EyeX
The Tobii EyeX is far smaller than most camera technologies on the market. Compared to the Kinect products on the Xbox, the EyeX is positively svelte, measuring 0.8” x 0.6” x 12.5” in total. Connecting via an included USB 3.0 Mini B type cable, the device is meant to mount at the bottom of your monitor. The included metal strip with an adhesive backing is placed on the bezel of the screen, and the EyeX’s magnetized back adheres the device at just the right angle to see your eyes.

With the device mounted, I plugged the Tobii EyeX into my machine and ran into a roadblock. Doing a bit of research, I found that I had a specific issue with my motherboard. The MSI Gaming M7 (which is less than six months old) uses the ASMedia 3.0 USB controller — something that causes the Tobii EyeX to connect and disconnect repeatedly. I updated my drivers to no avail, but the solution was quite simple — plug the cable into a USB 2.0 port. The short cable gave me fits to reach my case, but thankfully I wasn’t twice-bitten by another common issue — incompatibility with some USB extenders. It’s unlikely to be an issue for most.

With the device behaving as intended, the 125mb software installed, and a little while after a quick hands-off firmware update, the Tobii profile program came to life. Showing my eyes as two little dots on the screen, the software asked me to pop some bubbles by looking at them. With the calibration complete, it dropped me into a short minigame where I could make eye contact with aliens on ships, target asteroids with my gaze, and scroll around as I looked to the edges of my screen.

I was surprised to see that there are different profiles for normal eyes, eyes with contact lenses, and another for players with glasses. Similarly, there are settings for tracking both eyes, or your left or right eye independently. This is likely to help people with amblyopia (colloquially called “lazy eye”) or similar ocular disorders. I found no difference in my experience when I used my Gunnar shades versus the naked eye, but prescription glasses might make a difference.

Most movement trackers I’ve used required cumbersome glasses or a clunky headset. The EyeX doesn’t require any of those things, providing a great deal of movement freedom. I was able to move quite a bit to the left and right (beyond what would be natural movement), as well as towards and further away from the screen without the EyeX losing track of my eyeballs.

It was time to take the $129 peripheral for a spin.

Windows Hello
One of the functions of Windows 10 is Windows Hello. Windows Hello allows users to log into their machine using facial recognition biometrics. Sitting in front of the Tobii, my machine recognized me and signed me in without so much as a keystroke. Beyond the basic Windows login, you can also use this method to launch applications as well. It’s a cool novelty, and I’ll admit it felt a little like Mission Impossible.

An ever-expanding catalog
The Tobii application has a fairly extensive catalog of applications, but only recently has mainstream game support caught fire. Recently, AAA titles like Deus Ex: Mankind Divided, Masters of Orion, Tom Clancy’s The Division, Elite Dangerous, Assassin’s Creed Rogue, Assassin’s Creed Syndicate, Arma 2 and 3 have all been added to the roster. To test out the Tobii, I installed a handful of these titles and detailed their enhanced functionality below.

Assassin’s Creed Rogue
Assassin’s Creed Rogue is the oldest AAA title on the list, but it’s also the moment where Ubisoft dipped their toe into the eye tracking waters. Rogue has two integrations for Tobii EyeX: Infinite Screen, and Pause/Play. Infinite Screen means that the screen centers where your eyes are looking. If you keep your eyes forward, the camera will stay straight. Looking at the edges causes the camera to turn in that direction in a smooth motion. The default for this is somewhat slower than you’d need for an action-oriented title, but thankfully there is a great degree of calibration available. It’s not a replacement for the thumbstick, but after a short while, it’s a pretty good analog.

Pause/Play is exactly what you might expect — when you look away from the screen, the game pauses. When you look back, it immediately resumes. Simple.

Assassin’s Creed Syndicate
With Syndicate, Ubisoft embraced the Tobii EyeX a bit more completely. In what looks to have become a staple for all Tobii EyeX-supported games, Assassin’s Creed: Syndicate gets a “Clean UI” option. When enabled, the opacity of your HUD objects such as ammunition and minimap drops to around 15%. When you need them, you’ll obviously look at them, and the game immediately brings them back up to the screen at full opacity. It’s an immersion enhancing thing that might go unnoticed, but for me it removes elements that reminds me that I’m playing a game and lets me enjoy the experience and worldbuilding.

Aim at gaze lets you snap your targeting for ranged weapons for weapons like pistols and throwing knives to where your eyes are resting. Similarly, you can utilize your grappling hook in a similar way. Looking across a building edge, you can see all the would-be anchor points for Evie or Jacob Frye to zip across.

A very subtle feature I didn’t find featured in any other games I tested is a dynamic lighting option. Looking directly at objects bumps their light up or down according to where they lie in sunlight. It’s slight, but as the video below shows, it’s noticeable.

Similar to Assassin’s Creed Rogue, when you look around the screen the camera will follow your eyes and let you look at distant objects without moving the camera. Even though The Division is a newer title, I found the implementation in Syndicate made me feel less disoriented and added a bit of immersion instead of detracting from it.

In Syndicate, you can set map coordinates that show in the distance like a shaft of light. These sit alongside other markers like minigames and story missions. With Tobii enabled, they show almost completely opaque when they are extremely far away, coming sharper into focus as you close distance.

Deux Ex: Mankind Divided
Mankind Divided’s cyberpunk setting is perfect for the Tobii. Protagonist Adam Jensen already has augmented eyes that would suggest some of this technology exists in his skull, so I was eager to see how to Tobii was able to capture this setting with their technology.

The first, and probably the most noticeable change between Mankind Divided and other games is that the EyeX isn’t trying to emulate the right thumbstick. Players will still navigate as they normally would, but the EyeX allows you to move your eyes independent of where the camera is controlled. It’s not a massive amount of movement, but that’s a bit how it works in real life — where the head goes, the eyes generally follow. It’s how it ties to the “Aim at gaze” option that really wowed me.

Aim at gaze is pure genius. Sometimes when you are prepping for a shot, you are facing one direction, but glancing another direction to watch for reinforcements. Aim at gaze is built exactly for this, allowing me to snap the scope of my rifle where I’m looking without having to move the camera to do so. As you’ll see below, this is incredibly quick, allowing me to snap to a new target quickly. It’s less useful for melee combat, but for ranged shots, it’s indispensable. I didn’t ask for this, but I’m incredibly glad I have it!

Interact at gaze and Icarus Dash at gaze both let you use the Aim at gaze functionality, combined with other augmentations. Interact at gaze does exactly what it describes, letting you snap your attention to an object, enemy, or action where your eyes have focused. Icarus Dash at gaze lets you do the same thing, but combines it with Jensen’s high-powered dash move.

The coolest upgrade the Tobii EyeX brings to Deus Ex: Mankind Divided is once again the clean UI option. Just like Assassin’s Creed: Syndicate, the opacity of your HUD objects (minimap, quickselect for augments, quick use items, and ammunition) is suppressed, coming back only when your eyes land on them. I can’t stress enough how much I appreciated this feature.

Master of Orion
I had limited success with Master of Orion. While the game itself is great, the eye tracking was mostly relegated to being able to move around more quickly when paired with a gamepad. For a 4X game, I use a keyboard and mouse exclusively, so much of what Tobii EyeX was bringing to the table was lost on me. That said, there was one feature that was somewhat useful — Bungee Zoom at Gaze. Being able to zoom out with a keypress and then zoom back in all the way across the galaxy where my eyes were aiming was rather useful. I chalk this one up to Wargaming partner NGD Studio’s lack of imagination on how this could have worked. Given how meticulously detailed the eye tracking is, this could have made navigating menus a breeze.

Tom Clancy’s The Division
Taking a page out of Deus Ex’s book, The Division uses a lot of similar elements. Aim at gaze lets you focus where your eyes land, and Grenade at gaze lets you toss explosives in that direction.

Cover at gaze operates a little differently than the aforementioned Eidos title, instead highlighting objects that might serve as cover. It’s helpful when you aren’t sure if that piece of debris ahead is going to save your life or serve as window dressing.

Collaborative targeting leans on the Aim at gaze function, allowing you to look at a target and then hit the Z key (or whatever button you’ve mapped) and mark them. It’s not much better than using the mouse, but it at least works as well.

Extended view, like most titles with EyeX support, allows you to independently move your eyes while keeping the camera trained forward. I found it somewhat helpful for spotting targets hiding above me, but I also found it somewhat disorienting when I was running. I imagine I could get used to it, but in about 15 minutes of play, it was irritating enough to disable.

Just like Assassin’s reed Syndicate and Deus Ex: Mankind Divided, the Clean UI support from Tobii is excellent. The Division allows a great deal of customization of the HUD, but none of that bothers the EyeX — you can put whatever you want wherever you want and it’ll only pop into view when looking directly at it.

There is one aspect of the EyeX that might give some gamers pause — multiple monitors. Obviously the EyeX has a specific range that it’ll detect the extremely small movements of your eyes, and that does not extend to a second monitor. To that end, The Division does support moving your HUD elements across multiple monitors, but the EyeX will not. There is no simple way to resolve this for The Division, as moving your HUD onto the one monitor with the EyeX will fix the Clean UI implementation, but you’ll lose everything else, or visa versa.

I could never get the map functions to work with Tobii. Supposedly, you can use your eyes to scan around the map, marking spots with ease, but no matter how much I fiddled with it, that support seems to be missing.

Elite Dangerous and Horizons
If there is a game that demands eye-tracking support, it’s space sims. Elite Dangerous allows players a great deal of immersion by allowing them to look around the cockpit. The game has a great deal of information coming at you from all of the various computers in the cockpit, so I was excited to see what eye tracking could bring to the table. Before I could embark on that journey, I had to download a second application called Tobii Infinite Screen Extension.

With Infinite Screen Extension, the center of the screen always follows the movement of your eyes. This means you can look around the cockpit as much as you’d like, bringing up your HUD, tracking targets as you whiz by, and having a lot more player agency over your visual experience. Truth be told, it’s a bit disorienting to begin with, as our eyes are ‘untrained’ and tend to flick about the screen. As this controls where the center of your ship orients, it takes a little bit of time to acclimate, and a lot of discipline and focus — more akin to what I expect a real fighter pilot would need to do in the cockpit. Given how hardcore most Elite Dangerous players are, I suspect this feature will fit in nicely, though I would have liked something like independent gun reticles locked to my eye movements, not unlike the modern day Apache gun system.

Elite Dangerous Horizons is currently in Beta, so there wasn’t a setting in the Infinite Screen Extension profile to enable the functionality. I suspect that driving vehicles on the surface of planets shouldn’t be too tricky for Tobii to enable, so keep an eye peeled for support to pop up once Horizons leaves Beta.

Professional gamers – tracking your hotspots
I’m not a professional gamer, and if my recent battle against some of them at the recent Call of Duty XP is any indication, I’d better keep my day job. That said, there are a lot of competitive people who might find a lot of utility in knowing what their eyes are doing. The EyeX has a mode called Gaze Trace that puts a small mouse-tails-esque water mark on your screen, allowing you to see where your eyes are at all times. Players could, in theory, record their game session and then (whether they use Tobii EyeX game features or not) play back their session, noting where their eyes were at moments before they were killed. Perhaps they spend a great deal of time looking at the minimap when they should be watching other things. Developing a heatmap of where your eyes come to rest during a professional match could be helpful, I would imagine.

Foveated rendering
As both VR and eye-tracking hardware slowly begins to reach a maturation point, the topic of foveated rendering frequently comes into play. Foveated rendering is essentially recreating how the eye functions, allowing the pinpoint where the retina (or more accurately, the center dark part of the front of the eye, or fovea centralis) focuses to be extremely clear, but the area outside of that being less detailed. Commonly referred to as ‘peripheral’, you might notice that the ‘edges’ of your vision are blurrier than the center. Foveated rendering uses eye tracking to replicate this to create a more realistic image, rather than rendering the image at maximum clarity and resolution from edge to edge. While none of the games I tested currently support this rendering system, Assassin’s Creed Syndicate came closest. Having color fade away at the peripheral is a step towards foveated rendering, and something I believe could be expanded upon as the next possible evolution in eye tracking gameplay enhancement. The EyeX could enable it, now we just need game developers embrace it.

Specific tool for a specific job
If there was one aspect I was surprised was missing was a generic profile for the EyeX. While the game and application list for the product has recently hit 40 strong, I can’t help but feel like a generic application would be helpful to grow the community. For example, it would be nice to map the castle functions in a helicopter or flight sim to my eye movements, regardless of whether it has official support or not. Aim at gaze could be equally as useful in any shooter. Without it, the best gamers can hope for is the frequent pop-up from the Tobii website asking what game they should tackle next.

See below for our list of partners and affiliates: