Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Project-Natal-Style Interface For Mobile Phones

CmdrTaco posted more than 3 years ago | from the future-is-nowish dept.

Input Devices 49

An anonymous reader writes "The University of Tokyo is developing a Project-Natal-style interface for mobile phones, which allows you to interact with a phone by waving your finger around. Members of the Ishikawa-Komuro Lab have used a high-frame-rate camera to track the motion of a finger and recognise input gestures. The touchless interface can be used to navigate around a phone's screen and even to type."

cancel ×
This is a preview of your comment

No Comment Title Entered

Anonymous Coward 1 minute ago

No Comment Entered


I foresee problems. (3, Funny)

wpanderson (67273) | more than 3 years ago | (#31983224)

The danger for users of porn is clear and present.

Re:I foresee problems. (1)

need4mospd (1146215) | more than 3 years ago | (#31984770)

I say it's more translucent than clear. It's not quite present either, but you could say it's coming soon....

Air-Typing? No thanks (0)

Anonymous Coward | more than 3 years ago | (#31983250)

I can barely type well with an onscreen keyboard, and soon they'll want me to air-type? No thanks.

Re:Air-Typing? No thanks (2, Insightful)

psnyder (1326089) | more than 3 years ago | (#31983548)

The cool, practical implications for hand-held devices start coming to mind if this is combined with a touch screen.

(The camera would also need to be less obtrusive, and able to detect fingers just above the touch screen.)

Re:Air-Typing? No thanks (1)

CarpetShark (865376) | more than 3 years ago | (#31983848)

I want a mobile with a laser projected keyboard, and a light-protected screen. With the normal screen and touchscreen keyboard as a backup when there are no good surfaces around, of course. Seems the most obvious (and cool) way to get the necessarily large IO devices "into" a small form factor.

I haven't actually tried a laser keyboard though. Anyone know if they're actually usable for serious work?

How about parsing ASL? Or finger-spelling. (1)

Ungrounded Lightning (62228) | more than 3 years ago | (#31988194)

I can barely type well with an onscreen keyboard, and soon they'll want me to air-type? No thanks.

How about parsing ASL (American Sign Language)?

Then again, finger spelling might be easer - since that only involves one hand, rather than both and the body, letting you hold the phone in the other and reducing the camera requiremens. Also a much more limited "vocabulary" consisting mainly of the letters and numbers.

Wow, that's stupid (4, Interesting)

drinkypoo (153816) | more than 3 years ago | (#31983268)

I can understand the appeal of Natal because it lets four people interact with a piece of hardware without holding anything. But if you already have to hold the phone, it seems more than a bit ridiculous to be finding ways to not touch it with your other hand. It's like finding a way to chew your food without it touching your tongue...

Re:Wow, that's stupid (0, Troll)

purpledinoz (573045) | more than 3 years ago | (#31983386)

Why is this stupid? This is just a cool demonstration of the results of research done at a University. You just lack creativity and imagination to see any use for this. The applications for these types of technologies are not yet explored, so I'm sure we might see some really cool applications for this.

Re:Wow, that's stupid (1)

The End Of Days (1243248) | more than 3 years ago | (#31988856)

It's stupid because Slashdot is populated with basement geniuses who know everything. This is why everything invented is obvious and simple to implement, yet useless.

Re:Wow, that's stupid (1)

TheGeekologist (1798018) | more than 3 years ago | (#31983508)

This is just like a super alpha prototype. You need to stop thinking about the uses it has now, and more in the uses its going to have later on. Maybe in the future you won't be holding a piece of thick hardware, instead, some ultra nano hardware built-in on an advanced OLED-type screen. I can definitely see the use on something like that.

  I don't know man, just be imaginative.

Re:Wow, that's stupid (1)

delinear (991444) | more than 3 years ago | (#31984124)

The first thing that came to mind for me - those touch-screens in public places, like for booking train tickets, or even pub trivia games or cash machines. Maybe I'm overly prone to OCD, but I shudder to have to touch those, not knowing who has used them before me. They're generally smeared with other people's greasy finger marks. If I could air type just in front of the device, I'd feel a lot better. Maybe an alternative would be allowing us to shrink netbooks and laptops by effectively turning the whole of the keyboard area into a big trackpad when your hand is hoverwing over it, and eliminating the UI nightmare of trying to integrate a pad and keyboard while minimising space. I agree on a phone it's difficult to see the point right now, but the research in general is certainly not a waste of time.

Re:Wow, that's stupid (1, Insightful)

Anonymous Coward | more than 3 years ago | (#31983518)

It let's you easily draw in 3D space. Tell me of another interface which lets you do that easily.

Re:Wow, that's stupid (1)

gman003 (1693318) | more than 3 years ago | (#31983912)

And how often do you do any 3d interaction on your main computer, let alone your cell phone?

Hacking.. (1)

skgrey (1412883) | more than 3 years ago | (#31983272)

The implications on security in regards to hacking are interesting - you could in theory try to brute-force someone's phone from a slight distance, say next to them on a city bus. Not having to leave fingerprints on someone's device as well is another positive, as it is based on gestures. Especially handy for devices where wireless is protected or not enabled! Even if it took a picture to see who was gesturing, I'm sure you could get out of the way and just have your hand in the frame.

Re:Hacking.. (1)

purpledinoz (573045) | more than 3 years ago | (#31983328)

Security is completely irrelevant to this story. This is just a really cool prototype of being able gesture 3D input. No one is suggesting that you type your passwords purely with gestures.

Re:Hacking.. (0)

Anonymous Coward | more than 3 years ago | (#31983338)

If a six-figure-a-year executive is stupid enough to fall asleep on public transit with her phone face up on her lap then she deserves to be hacked.

Re:Hacking.. (1)

Jurily (900488) | more than 3 years ago | (#31983524)

executive is stupid enough [...] she

When was the last time you've seen a stupid female executive?

Re:Hacking.. (2, Insightful)

ViViDboarder (1473973) | more than 3 years ago | (#31983602)

Most of the time you have to wake a device before you can do any input. That would probably involve hitting a physical button.

As for brute forcing... You must be fluent in sign language or something. It's not plausible that if someone was using this to detect some kind of pattern on a device that you'd be able to sneak your little hands over there and try every possible combination in 3D. This detects depth too. So it's not going to be something like Android lockscreens where you slide Left, Left, Right, Up, Right, Down or something. You'll have Up, Down, Left, Right, Front, Back all as possibilities. It would take a very long time to "brute force" with your fingers.

Also, the device is limited to a fairly short usage distance as well. The contrast between a hand 5 feet away will probably too small to allow the software to define a good finger outline when switched to Black and White. You'll notice that the camera is surrounded with really bright LED's to ensure that, at a short distance at least, the finger is distinguishable.

So essentially, there is no more security flaw to this as there is any practical reason to use it on a phone. :p

I could probably get used to it, but... (4, Insightful)

IndustrialComplex (975015) | more than 3 years ago | (#31983280)

I guess I understand some of the 'neato' factor that this could have, but isn't one of the big issues introduced with simple touchscreens the LACK of haptic feedback?

I love my iphone, but I miss being able to just reach into my pocket and dial a call, or skip a song/rewind, etc. Having to pull my phone out every time I want to do anything other than change the volume is damned annoying.

I could just see this kind of interface being more annoying to use.

Re:I could probably get used to it, but... (1)

vivaoporto (1064484) | more than 3 years ago | (#31984654)

I love my iphone, but I miss being able to just reach into my pocket and dial a call, or skip a song/rewind, etc. Having to pull my phone out every time I want to do anything other than change the volume is damned annoying.

If you are using the default iPhone earbuds, you can do all that using the middle button (the one between the up and down volume buttons):

One click to play/pause, two clicks to skip to next, three clicks to go back to previous, click and hold to activate Voice Control (where you can make calls, play albums, playlists, etc.).

That is a life saver feature that should be more publicized, took me a while to figure them out.

Re:I could probably get used to it, but... (1)

IndustrialComplex (975015) | more than 3 years ago | (#31985834)

I didn't know about Voice Control, does it require any special apps or iPhone version (3gs?)

Also, is there a non-apple version of earbuds available with those buttons? I'd use mine, but they don't fit my ear well and let in a lot of ambient noise. I've just been using some nice earbuds (no buttons) that give a good seal.

Wow, now that is COOL! (1)

purpledinoz (573045) | more than 3 years ago | (#31983282)

I was really impressed by the smoothness and accuracy that the camera was able to track the finger movements. Usually, things like this are really jumpy and horribly inaccurate. Although it's not the best way to type things in, I'm sure there will be a ton of cool applications of this.

In Japan it could make more than enough sense... (1)

sznupi (719324) | more than 3 years ago | (#31983334)

...if all those flamboyant choreographies from anime are any indication.

Public interface (5, Insightful)

need4mospd (1146215) | more than 3 years ago | (#31983504)

This would be pretty cool to slap into some public use areas like train stations, airports, movies, etc... Half the touchscreens I've used in public areas are always broken(or half working) after a couple years from people pounding on the screen when it won't respond. Slap a piece of plexiglass in front of it that you can replace when it gets damaged, saves the expense of a whole new touchscreen.

Re:Public interface (1)

Saint Gerbil (1155665) | more than 3 years ago | (#31985164)

And then comes the law suits when someone inevitably slaps a passer-by while trying to find their way to ann-summers.

Hope for Mankind (1)

NEDHead (1651195) | more than 3 years ago | (#31983636)

I was watching the demo video, and all of a sudden my finger started moving around matching what was on the screen! Then I realized that at last we have a glimmer of hope for a future where our politicians actually do what we want them to do....

Great... (1)

fragMasterFlash (989911) | more than 3 years ago | (#31983674)

We already have enough Bluetooth headset users walking around giving the appearance that they are talking to themselves. Now these same people are going to look like they are having their solo discussions in sign language.

Less accurate (1)

Enderandrew (866215) | more than 3 years ago | (#31983696)

The iPhone 4 prototype that leaked has second camera that faces the user. It could go gesture recognition as well. However, the last thing I want is a less accurate means to control my phone. Why wave my fingers in front of the phone, when I could just extend my fingers slightly more and touch the screen?

Natal? (0)

Anonymous Coward | more than 3 years ago | (#31983922)

Stop calling it "Project-Natal-style", because as usual Microsoft didn't invent anything.

Call it "EyeToy-style", as it should be.

Re:Natal? (0)

Anonymous Coward | more than 3 years ago | (#31984220)

There is significantly more to Natal than the Eyetoy part: it tracks multiple actors (more-or-less the moral equivalent of "multitouch" eyetoy), tracks in 3 dimensions, voice recognition, natural language processing, facial/object recognition, etc.. The eyetoy was not really sufficient for a full on natural interface. Calling it "Eyetoy-style" is like saying your new HTC Dream has a button-style interface, since you can press the screen. Or that a truck has a walking-style interface because you use your feet to move it. When in fact a truck's interface might better be compared to a car, or the HTC Dream has an iPhone-style interface. There's just a huge gap beyond one obvious similarity.

Plus, it's not like the Eyetoy was the first thing to recognize images on even a 2D plane anymore than Natal would be.

Re:Natal? (0)

Anonymous Coward | more than 3 years ago | (#31984348)

Well, okay then. But can we agree that "Natal" sounds as dumb as "Zune"? Seriously, what's wrong with the marketing department at Microsoft? They used to be the best ones out there.

Natal Is Garbage (0)

Anonymous Coward | more than 3 years ago | (#31985592)

LOL! Are you a fucking moron.

Every showing of Microsoft's shitty EyeToy ripoff has shown it to be a laggy, glitchy turd of a technology.

It was so bad last E3 Microsoft was caught faking their demos of it by the gaming press.

Re:Natal? (1)

IrquiM (471313) | more than 3 years ago | (#31988490)

Are you telling me that my multiplayer eyetoy games doesn't work? That Eyetoy Pet doesn't work? What are you trying to say?

Natal Style? (0)

Anonymous Coward | more than 3 years ago | (#31985494)

It's a single camera, not stereo cameras. It's nothing like Natal. It's more like the Playstation Eye, which predates Natal by a decade.

Natal hasn't even launched... (1)

sherriw (794536) | more than 3 years ago | (#31985760)

Considering that Natal hasn't even launched yet, other groups might want to hold off and see what the reception is to Natal before they start copying it. I for one bought a Wii and was sorely disappointed in the non-precision of the wii-mote. I also have an Xbox and I fear that Natal will be just as annoying. I hope some cool uses of it are created though.

Sign language (0)

Anonymous Coward | more than 3 years ago | (#31986174)

how is it the first thought is for faster interaction to a mobile device and not for helping handicapped people to communicate with the world.

Seems a shame we look at new technologies with helping ourselves, rather then helping others.

An obvious use: larger keyboard (1)

robi5 (1261542) | more than 3 years ago | (#31995382)

For those asking how it is an improvement over a touchscreen: There can be a ratio of finger movement vs. cursor movement, that is, a mid-air keyboard can be made much larger than an on-screen keyboard.
Check for New Comments
Slashdot Account

Need an Account?

Forgot your password?

Don't worry, we never post anything without your permission.

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>
Sign up for Slashdot Newsletters
Create a Slashdot Account