Eye Tracking Coming To Video Games 102
An anonymous reader writes "Over the past several years, we've had a spate of new input methods for playing video games. Instead of just pushing buttons, now we can wave body parts around, yell at the screen, or even (weakly) control things with our thoughts. Now, we're adding an eye-tracking sensor bar. It's being created by SteelSeries, but it's based on tech from a Swedish company called Tobii, who built similar tech for cars. 'Inside the device there are two cameras and an infrared light source. The infrared light reflects off your pupil and cornea, which is then captured by the two camera sensors. Throw in a healthy serving of Tobii's proprietary image processing algorithms, and a physiological 3D model of the eye, and you can work out the position of the eye and the direction of the gaze with high accuracy. Tobii doesn't seem to put an exact figure on the resolution/accuracy, merely saying that "within less than a centimeter" is possible.' Of course, the biggest question will be how well it works, but it seems like it could be a useful supplement to normal control schemes. I can see how it would be nice to simply flick your eyes to an icon to do something, or to make it easier to dig through your in-game inventory."
yuck (Score:2, Insightful)
I can see how it would be nice to simply flick your eyes to an icon to do something, or to make it easier to dig through your in-game inventory."
I can't see that being nice at all.
I may look at an icon, then decide not to use it, when considering my options. Or what If I want to activate an icon but need to activate it at a precise moment... I have to avoid looking at it until the right moment? That's not user friendly.
On the flipside, presumably there will be some method of preventing spurious activations
Re: (Score:1)
Re: (Score:1)
Basically the eye becomes the part of the system that moves the mouse and you have buttons or voice commands for accessing areas if you want. So you move your eye over to something and click it or you quickly adjust between five things and order a command that changes how the five things interact. This is way better ergonomically than sliding a mouse or using your hand on a touchpad because no part of your body is actually pushing against a slide-surface.
Re:yuck (Score:4, Insightful)
This is way better ergonomically than sliding a mouse or using your hand on a touchpad because no part of your body is actually pushing against a slide-surface.
Is it? I for one, do not want the mouse following my eye around. Have you ever played an FPS? or RTS? Do you really want the selection cursor/targeting reticule following your eyes?
I'm scanning the screen, looking at the mini map, checking the build status of units, glancing at ammo levels, ... providing cover fire in a FPS while scanning windows for activity...
I can't imagine ever wanting the 'cursor' to follow my eyes around like that. Not only would it be distracting to have some sort of eye tracking cursor flitting around the screen like a deranged idiot but it would be counter productive since we often want independent control of where we are looking and where something is happening.
e.g. I'll have the cursor positioned over an icon, but I'm looking at any enemy health meter or waiting for them to be in a certain position to time when I activate it.
Re: (Score:3)
I for one, do not want the mouse following my eye around. Have you ever played an FPS? or RTS? Do you really want the selection cursor/targeting reticule following your eyes?
Why would you need a cursor icon following your eyes? The cursor is there to provide a visual feedback for your hand movements. If the cursor was your eye, the icon is redundant.
The only reason you would need a visible cursor icon in a eye-tracking system is if the eye-tracking is crappy or laggy, and you need to give the user a way of seeing when it has arrived. In which case, few people who aren't paralysed would really want to use such a system, and certainly not gamers.
Re: (Score:2)
If the cursor was your eye, the icon is redundant.
Even if it was perfect you'd need the feedback to know where you are looking. You line up a shot, and you are checking your health meter by the time you register a click, or already scanning for the next target.
Easily solved. (Score:3)
Re: (Score:2)
Left wink for select, double blink for execute and right wink for a context menu.
I can't wait to try it out in a public place.
Re: (Score:1)
You're making the mistake of assuming technology will advance within the confines of our current designs only and you're ignoring the fact that whenever technology advances it often crashes through the confines of walls assigned to try and keep it the way a select few want it to be. Mouse cursor translucency could fix the problem or a simple cross-hair. Another possible fix is that there would be multiple cursors that would be manipulated in polygonal patterns. This polygon method is probably the best appli
Re: (Score:2)
You're making the mistake of assuming technology will advance within the confines of our current designs only
I'm taking the statement "eye tracking coming to video games" at face value. Because that is how I see it being implemented in the near term.
you're ignoring the fact that whenever technology advances it often crashes through the confines of walls assigned to try and keep it the way a select few want it to be
I'm not ignoring that. That's something else entirely. "Novel user interfaces for eye trackin
Re: (Score:1)
Re: (Score:2)
Basically the eye becomes the part of the system that moves the mouse
This would be the absolute worst possible way to use the technology for the reasons others have stated. Keep in mind that this thing reported has accuracy in the centimeter range, considering that I routinely click buttons on my 22 inch screen that are millimeters in physical size, this would work out terribly as a mouse replacement.
Where it would be helpful is as an augmentation to provide additional data (something like tooltips being the most obvious and simplistic form) when the users gaze lingers ov
Re: (Score:2)
Re: (Score:1)
I agree. Tooltips would be the limit of what I'd want to activate with my eyes.
I'm sure you'll change your mind once they find a way to integrate this with porn ;-)
Re: (Score:2)
Re: (Score:2)
So don't make your eye the button/trigger, just the cursor/reticle.
Your eyes are scanning the whole screen, checks ammo, health. It would be distracting to have the reticule due that.
Imagine you are sighted on a window, you glance down at your health... the whole screen moves as your 'avatar' looks at his feet... then madly scrambles back up to the window.
Or ... you are sighted on one window, but scan other areas for enemies... your reticule bounces around like an idiot.
Thanks but no thanks.
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
Imho the real advantage of eye-tracking is in wearable computing. If you track the eye quickly and precisely enough it becomes much easier to do all kinds of nifty mediated reality tricks.
Re: (Score:2)
I may look at an icon, then decide not to use it, when considering my options. Or what If I want to activate an icon but need to activate it at a precise moment... I have to avoid looking at it until the right moment? That's not user friendly.
A wink or a blink would serve as a decent substitute for a right/left or middle click.
I myself am ambivalent in seeing it applied to games, but if doing so serves to inject a little capital into technology that could help so many people with various disabilities it seems like a win-win.
Re: (Score:2)
--Personally, I think tapping your left/right big toe to act as a mouse click would be more useful (and quicker) then winking or blinking a certain way. Plus, I'm one of those people who gets a damn eyelash caught in my eye on a semi-frequent basis, causing me to blink furiously and my eye(s) to tear up. Not to mention allergies...
Re: (Score:2)
I'm not against it though, maybe it will let one handed people play more games, or some paraplegic will be able to play doom by blinking at the screen and it betters his life...
Having this technology at the consumer level would be huge for affordable adaptive technology. Games/shmames (though why not, people without limbs should be able to enjoy entertainment too).
Most adaptive equipment is horrifically expensive. The more accessible consumer-level stuff I can get to help my son, the better.
advertising effectiveness (Score:2)
I remember watching a show 20 years ago about gauging the accuracy of advertising by using eye tracking. They took people off the street into a special truck (it was bulky equipment), and they could very precisely see how the men's gazes never quite reached the lingerie brand name in the ad's corner.
There is no way anyone would ever want to use an updated version to track your ad viewing while browsing. Nope. Not gonna happen. Totally not running to the patent office (if Samsung doesn't have that one yet, s
Re: (Score:2)
[...] they could very precisely see how the men's gazes never quite reached the lingerie brand name in the ad's corner.
Not to mention how the gaze patterns were profoundly different for 5-10% of the men, who reliably tracked to male backsides instead of female backsides or breasts. We're getting closer to recognizing Nothing Wrong With That as a society, but there are still a lot of men who'd prefer not to be outed by their sidebar ads.
Re:advertising effectiveness (Score:5, Funny)
And some of us will just be stuck on the terrible kerning the ad uses on its font, either way.
Re: (Score:2)
And some of us will just be stuck on the terrible kerning the ad uses on its font, either way.
Dear Lord, it's all in Comic Sans!!!!
MY EYES! AIIIIIEEEE!!!!
Not until I see rock-solid containment of the data (Score:4, Insightful)
Eye-tracking is a big honking window into the subject's subconscious mind. I will be extremely reticent about releasing that data into an information system.
It's creepy enough seeing the amount Google knows about me already just from searches and cookies. I sure as hell don't want advertisers to get fine-grained feedback about which ads attract my attention, never mind cranking up the distracting peripheral-vision movement to force my gaze.
And don't even get me started on the evil tricks you can play by keeping things just outside the user's central vision, no matter how hard you try to look directly at them...
Context (Score:1)
Context is always missing from computer tracking. Without context there is always a shadow of a doubt.
Re: (Score:2)
Eye-tracking is a big honking window into the subject's subconscious mind. I will be extremely reticent about releasing that data into an information system.
It's creepy enough seeing the amount Google knows about me already just from searches and cookies. I sure as hell don't want advertisers to get fine-grained feedback about which ads attract my attention...
And the NSA wonders why I wear sunglasses indoors at night while using my computer these days... :)
Re: (Score:1)
Gaze-specific focal plane and depth of field? (Score:3, Insightful)
Re: (Score:2)
However, I'm not sure if that form of artificial focal depth will enhance the illusion or create more potent migraine material for someone in my position.
Re: (Score:3)
Almost there.... (Score:5, Interesting)
I don't care about games, but I would love it when they can adapt this technology to give X magic focus powers.
I already use 'Focus follows mouse' on fluxbox, but too often I am looking at a differnt terminal window when I start typing away, and I had forgotten to move the mouse.
[Paranoid Mode] imagine how much more advertising money Google could make if they tracked WHERE you were looking. [/mode]
Re: (Score:2)
Sometimes, you need to look at one thing and type into another, and that would just get in the way.
Lara Croft (Score:5, Funny)
Re: (Score:2)
You mean they will tailor the amount of, um, "muscle wiggling" to what we want? Sounds like finally a useful application of eye-tracking!
Hey, here is a business-idea: Make a Google Glass App that does just this type of tracking for the female wearer. Then she can get an immediate overlay for "creep", "one-night-stand material", and "boring possible husband"... ;-)
Re: (Score:2)
New game mechanic: The more focused the stare, the larger the 'muscles' get!
Re: (Score:2)
Allow it to realistically throw off her mass and center of balance and you could have one doozy of a challenging game.
Does this already exist in military? (Score:2)
Pan and tilt the camera! (Score:3)
More usefully, tracking eye movement could be used to control the camera. Tilting and panning to bring anything the user concentrates on to the centre of screen would be useful in a lot of games, and would get around the poor camera placement algorithms we've all been annoyed by.
It would have not to be too sensitive - the user glancing at a status display or an incoming message should not move the camera - but I could imagine this being really useful.
The motivation for this... (Score:3)
game pads are crap compared to the keyboard and mouse.
All these new interfaces are just means by which an inferior input system can compete.
Its frustrating for game devs to be limited by the gamepad.
Strategy games are totally impossible. FPS games are awkward. Even RPGs need to be simplified and MMOs which are already pretty dumbed down have to be dumbed down farther.
So here is my suggestion for how to improve the situation.
Allow console users to plug in a keyboard and mouse. By all means, streamline it a bit if you must. But that in and of itself would resolve most of the problems.
Re: (Score:2)
I guess you still haven't gotten around to replacing the shitty speakers your computer came with by a decent surround sound set yet. My computer speakers are the surround sound system for my living room. The center channel speaker just happens to sit under the monitor.
And I'd far rather sit on my $400 multi-way-adjustible office chair than on an immobile couch with no back support.
Re: (Score:2)
I have to disagree - a wiimote is superior to a gamepad, but it doesn't begin to compare to a mouse.
Re: (Score:2)
Depends, games like Risk of Rain are much better off played with a gamepad, but your mileage may vary. You only list a minimal number of genres that have been traditionally mouse-driven, though.
But as much as I like playing with pad (where convenient: fighting games like Skullgirls, sidescroller action games like Risk of Rain or Rogue Legacy, console emulation, etc), I wouldn't play, say... Doom, with one.
As for RPGs, depends. Skyrim with a pad is a bit of a pain, but there used to be games like Legend of M
Re: (Score:2)
It isn't a singularity at all.
Microsoft keeps pulling PC devs into the console. And what is the result? You get a dumbed down PC game with crappy controls. Why?
As to lap desks and couches... I'd explore air mice before I used a dpad for a lot of games.
Microsoft experimented with letting PC gamers play against xbox gamers with Halo 1. Guess what the result was?... The Xbox gamers were slaughtered. No contest.
Why? The controls are awkward for FPS games. Useless for strategy. And don't have enough buttons for
Re: (Score:2)
The problem is you need a flat, stable surface for a keyboard and mouse, i.e. a table or desk. Games consoles are for the living room/bedroom, most people play them sitting on the sofa or on the floor.
Allowing keyboard/mouse control in a single player game is okay I suppose, but most games have an online element now. It's bad enough with cheat devices (you can modify an XBOX pad to have autofire or even macros for performing complex combos in fighting games) and no-one wants a ground of people with a massiv
Re: (Score:2)
Well they're inferior. They provide an inferior degree of control. That is not an opinion but an experimentally proven empirical fact.
They are bad for anything but the most rudimentary of games. And more frustrating their influence on the PC community is to impose those design limitations on PC gaming to allow for easy porting between platforms. This means PC games are frequently simplified to be made compatible with this inferior system.
Ultimately this is what annoys me. I am a PC gamer. I don't really car
I don't know who "you" are, but ... (Score:2)
... "I" am doing no such nonsense. Looks more like a blatant slashvertisement for yet another worse-than-useless gadget than anything useful.
Eyes don't stand still... (Score:2)
The brain rarely lets the eyes stand still. Eyes are actually quite poor resolution anywhere except for the very centre of their visual space. The eyes will dart around often more than 3 times a second when you're not paying attention to anything specific and the brain puts the pictures together to build an image of what the surrounding environment looks like. The bigger the screen the worse the movement.
The idea of using the eyes focus point for selection criteria is borderline absurd. You open a screen an
Re: (Score:2)
The brain rarely lets the eyes stand still. Eyes are actually quite poor resolution anywhere except for the very centre of their visual space. The eyes will dart around often more than 3 times a second when you're not paying attention to anything specific and the brain puts the pictures together to build an image of what the surrounding environment looks like. The bigger the screen the worse the movement.
That's the first thing I thought of. Perhaps thats where the "... proprietary image processing algorithms, and ... physiological 3D model of the eye..." comes in. This definately doesn't seem like the kind of thing that could be used for the mouse cursor, but rather to determine which part of the screen is being looked at, generally.
Why do I want this again?
You don't. But I'm sure they'll come up with some spurious use or another just to get at all that juicy eye tracking data for ad targeting data or what ever other nefarious p
Re: (Score:2)
no sense of humor... i give it a +2 funny
Already using head tracking... (Score:2)
Not 30 minutes ago I finished building a point model (sort of like this one [gibthis.com]), which I'll use for head tracking in combat flight sims. I tried using face tracking, but it wasn't reliable enough.
IMHO, head tracking is better than eye tracking. For head tracking, a multiplier is configured so that e.g. turning your head 20 degrees turns the game POV all the way around. Once you're used to holding a controller with your face, it's pretty easy / natural to use. On the other hand, it takes a lot of effort to keep
This is not new technology. (Score:3)
Tobii has been using it in adaptive communications devices for year. My late wife suffered from ALS, and she used a Tobii C17 with Ceye for speech, since she was no longer able to talk.
Re: (Score:2)
It may not be new technology, but I'lll bet it's a new price point for the technology.
Assistive device manufacturers are not exactly known for being generous to those who need their tools.
Adaptive quality (Score:3)
You could increase polygon tessellation at an area of focus and remove detail from the peripheral. It would be very useful for triple 120Hz 4K 3D. ;)
Re: (Score:2)
You could increase polygon tessellation at an area of focus and remove detail from the peripheral. It would be very useful for triple 120Hz 4K 3D. ;)
I've long imagined that was the performance trick God uses on it's VR system.
Better for work than for play (Score:2)
But man, would it ever be useful for real work. Simple things, like making whatever windown I'm focusing on become active.
A good eye-tracking system could replace a mouse, with maybe a pair of buttons right below the space bar on the keyboard or something.
I would think blink or wink tracking would be more annoying than useful, though...
Integrate it inside an Oculus Rift (Score:2)
For those asking "what is this good for?" or complaining "why would I want this?", immersive virtual reality is in desperate need of new input methods. A convincing, immersive 3D world is wildly incompatible with our favorite pointing device, the venerable mouse. And "air mouse" is both obnoxiously inaccurate and obnoxiously fatiguing to use.
Gaze tracking integrated inside a VR head mounted display helps address the new user interface problems Rift users are already suffering. A 2D mouse cursor in your 3
Re: (Score:2)
Re: (Score:2)
Very good point. It would go a long way towards preventing motion sickness by helping the virtual world look more correct to the human brain, which is peculiarly sensitive to the oddest cues, depth of field among them.
Hope this gets hacked for usability studies (Score:2)
Eye tracking in the usability world has been in the 4- and 5-figure range for a while. At least one cheap ($1200) option required that you wear heavyish headgear.
Tobii makes some of this usability eyetracking equipment, and it's my hope that this device will be adapted to work with the popular Morae usability recording software.
Eyetracking will be part of nearly everyone's usability toolkit, vs. an expensive luxury, if so.
It might hold possibilities for eyetracking studies on mobile software as well.
What could possibly go wrong? (Score:1)
Yes, my eyes followed the woman in the red dress, so what?
Better Graphics Where You Look (Score:1)
This can be used for simply concentrating the graphics resources in the area that the eyes are focused. Microsoft Research was showing something like this off at last years SIGGRAPH conference. They are claiming an acceleration factor of 5-6 when using the eye tracking.
Their paper can be found here: http://research.microsoft.com/apps/pubs/default.aspx?id=176610 [microsoft.com]
Having stood by and watched while someone was using the system, it was interesting to see their eye dart around the screen and the little high-qua
Forget control. Let's optimize video processing (Score:1)
Pointless (Score:1)
Eye Control! (Score:2)
Because soon the children will be too fat and lazy for joysticks and buttons.
Old Tech (Score:1)
Inside the device there are two cameras and an infrared light source. Throw in a healthy serving of Tobii's proprietary image processing algorithms, and a physiological 3D model of the eye, and you can work out the position of the eye and the direction of the gaze with high accuracy.
Throw a healthy dosage of high intensity laser and get an even higher accuracy.
An identical technology (Score:1)
Not for me. (Score:1)
http://en.wikipedia.org/wiki/Nystagmus [wikipedia.org]
Ill just leave this here.
This has huge usages in HDR. (Score:2)
The magic bullet for eye tracking is using it for adaptive HDR. Monitors for good reason can't show bright lights. This makes it difficult to fit all the luminance information on the screen at once.
By tracking the eye we can adjust luminance information locally. This can have a huge impact on image quality.
many such demos at past two siggraphs (Score:2)