Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Robotic Hand Translates Speech into Sign Language

ScuttleMonkey posted more than 8 years ago | from the look-ma-no-hands dept.

Robotics 135

usermilk writes "Robot educators Keita Matsuo and Hirotsugu Sakai have created a robot hand that translate the spoken word into sign language for the deaf. From the article: 'A microchip in the robot recognizes the 50-character hiragana syllabary and about 10 simple phrases such as "ohayo" (good morning) and sends the information to a central computer, which sends commands to 18 micromotors in the joints of the robotic hand, translating the sound it hears into sign language.'"

Sorry! There are no comments related to the filter you selected.

How to sign First Post? (3, Interesting)

A Dafa Disciple (876967) | more than 8 years ago | (#14489536)

Good lord! I imagine the Japanese language [wikipedia.org] with its 1945+ character alphabet is hard enough to learn; learning Japanese sign language must really suck.

You know what would really spoil those deaf kids is, instead of a robot doing sign language, a robot that shows images or words based on what a speaker says. I know, I know; creating a robot to do this is a feat within itself and impressive in its own right, but perhaps there are better ways of communicating with a robot if it can already perform more than adequate speech recognition.

Re:How to sign First Post? (1)

Kokuyo (549451) | more than 8 years ago | (#14489659)

Uhhh, isn't sign language universal? I thought it didn't depend on the spoken language. I might be wrong, of course :).

Re:How to sign First Post? (2, Informative)

Anonymous Coward | more than 8 years ago | (#14489678)

I believe it's a joke... but just to point out, there's ASL and BSL and...

Re:How to sign First Post? (2, Informative)

Jugalator (259273) | more than 8 years ago | (#14489693)

Unfortunately, no [wikipedia.org] ;-)

Universal Sign Language (1)

SeanDuggan (732224) | more than 8 years ago | (#14489817)

Unfortunately, no ;-)
Yup, much the same as how, unfortunately, no one's come up with a universal spoken or written language. Gosh, let alone trying to get a universal programming language...

Re:Universal Sign Language (1)

cayenne8 (626475) | more than 8 years ago | (#14491380)

"...no one's come up with a universal spoken or written language..."

Hmm...isn't the middle finger signifying "Fuck You" pretty much universal?

:-)

I wonder if the robot hand translates that properly?

Re:How to sign First Post? (0)

Anonymous Coward | more than 8 years ago | (#14489660)

Why would learning Japanese suck anymore than learning BSL or ASL?

Re:How to sign First Post? (4, Informative)

tpgp (48001) | more than 8 years ago | (#14489672)

Good lord! I imagine the Japanese language with its 1945+ character alphabet is hard enough to learn; learning Japanese sign language must really suck.

The relationship between a language & sign language does not work like that.

From the wikipedia sign language page [wikipedia.org]
A common misconception is that sign languages are somehow dependent on oral languages, that is, that they are oral language spelled out in gesture, or that they were invented by hearing people
and
On the whole, deaf sign languages are independent of oral languages and follow their own paths of developmental. For example, British Sign Language and American Sign Language are quite different and mutually unintelligible, even though the hearing people of Britain and America share the same oral language.
You know what would really spoil those deaf kids is, instead of a robot doing sign language, a robot that shows images or words based on what a speaker says.

That doesn't really sound like a robot, but speech recognition software connected to a teleprompter (or monitor)

Re:How to sign First Post? (2, Informative)

lewp (95638) | more than 8 years ago | (#14489698)

Japanese has a whole bunch of kanji, but the various words in the language can be formed from a much smaller (hiragana, mentioned in TFA) character set that represents the various syllables in the words. These syllables are always pronounced consistently, unlike languages like English where sometimes it seems like nothing is consistent (and I'm a native speaker). Thus, the first thing that came to my mind was that teaching a robot spoken Japanese is probably quite a bit easier than teaching one English (though neither is a trivial task, obviously).

I know nothing about Japanese sign language, and practically nothing about American sign language, but I believe American sign language shares a similarity to written Japanese in that there are signs for common words most any competent signer knows (similar to kanji), and any particularly uncommon words can be signed out with the letter (or in the Japanese case, hiragana syllable) signs. Thus, I doubt teaching a robot enough Japanese sign language to be understandable wouldn't be any harder than teaching a robot American sign language. Which, once you've turned the speech into letters/syllables in the speech recognition part and programmed in the gestures, would be pretty much trivial. Japanese children's TV and manga aimed at kids (I'm told) mimmicks this behavior by mixing the simple kanji school children will have learned at a young age with the hiragana for the words that aren't expected to be known.

I'm shooting from the hip here based on what little experience I have with this stuff, so feel free to correct me, experts.

Re:How to sign First Post? (2, Informative)

magefile (776388) | more than 8 years ago | (#14489993)

I know nothing about Japanese sign language, and practically nothing about American sign language, but I believe American sign language shares a similarity to written Japanese in that there are signs for common words most any competent signer knows (similar to kanji), and any particularly uncommon words can be signed out with the letter (or in the Japanese case, hiragana syllable) signs.

Sorta, but not quite. You can fingerspell words you don't know, and some words are derived from their associated letters (i.e., one of the possible signs for "what" looks a lot like a "W" snapped into a "T", and one of the signs for toilet looks like a shaken "T"), but some of these are frowned upon culturally (cultural baggage due to decades of surpression of sign by hearing people). Too, if you depend on fingerspelling too much, you'll find it difficult to communicate; you won't be able to receive well, and while Deaf will put up with receiving it if they know you're learning, it's not sign language, and everyone knows it. Doesn't fit in all that well with the syntax and grammar of ASL, either.

Re:How to sign First Post? (0)

Anonymous Coward | more than 8 years ago | (#14490132)

Not only that, but sign is extremely dependent on context. A single spoken English word (I typically use 'run') will have numerous signs depedent on context. I.e. 'run to the store' is a different sign from 'the car is running', and from 'I have a run in my stocking', or 'my nose is running'.

In short, this is neat, but there's still a long way to go to replace real, human interpreters.

JSL vs ASL (1)

SeanDuggan (732224) | more than 8 years ago | (#14490144)

Interestingly enough, Japanese Sign Language has a trait which makes it less appropriate for this application than American Sign Language. Ambiguous signs are generally distinguished by mouthing [deaflibrary.org] the letter in JSL versus the finger-signed letters in ASL.

The next question which I have is the significance of body positioning of signs in JSL. Most ASL signs have migrated to the face and upper-chest region, but I know some sign languages have a great amount of significance in the body positioning and it may range all over the place.

Re:How to sign First Post? (1)

Andrzej Sawicki (921100) | more than 8 years ago | (#14490021)

That would not require a robot, just a screen to display the messages.

Re:How to sign First Post? (1)

Zaatxe (939368) | more than 8 years ago | (#14490149)

This robot could be used by deaf dogs, which (I'm not sure it's a myth or not) can't see flat images.

Re:How to sign First Post? (1)

Zaatxe (939368) | more than 8 years ago | (#14490175)

You think signing language could be hard to learn? Imagine blind people trying to learn japanese in Braille!!

ATTENTION SCUTTLEMONKEY (-1, Offtopic)

Anonymous Coward | more than 8 years ago | (#14489537)

After reading this letter, you will never again be able to trust Mr. Scuttle Monkey, and you will see with crystal clarity the way that reason, not make-believe, is the best way to deal with the real evils of our world. Before I begin, let me point out that his flock appears to be growing in number. I indeed pray that this is analogous to the flare-up of a candle just before extinction yet I keep reminding myself that many people think of his prudish insults as a joke, as something only half-serious. In fact, they're deadly serious. They're the tool by which myopic devotees of conspiracy theories will dress up Mr. Monkey's profit motive in the cloak of selfless altruism by next weekend. A second all-too-serious item is that perhaps one day we will live in a world where good people are not troubled by fear of semi-intelligible, crass half-wits. Until that day arrives, however, we must spread the word that Mr. Monkey has a knack for convincing self-serving gadflies that advertising is the most veridical form of human communication. That's called marketing. The underlying trick is to use sesquipedalian terms like "internationalization" and "roentgenographically" to keep his sales pitch from sounding inaniloquent. That's why you really have to look hard to see that Mr. Monkey's overweening dream is starting to come true. Liberties are being killed by attrition. Vandalism is being installed by accretion. The only way that we can reverse these satanic trends is to speak up and speak out against Mr. Monkey. To be precise, I would be grateful if he would take a little time from his rigorous schedule to compile readers' remarks and suggestions and use them to shout back at his propaganda. Of course, pigs will grow wings and fly before that ever happens. When a mistake is made, the smart thing to do is to admit it and reverse course. That takes real courage. The way that Mr. Monkey stubbornly refuses to own up to his mistakes serves only to convince me that he either is or elects to be ignorant of scientific principles and methods. Mr. Monkey even intentionally misuses scientific terminology to replace intellectual integrity with ridiculous sloganeering.

Mr. Monkey acts as if he were King of the World. This hauteur is astonishing, staggering, and mind-boggling. Make special note of that point, because if we are to expose his double standards for what they really are, then we must be guided by a healthy and progressive ideology, not by the mindless and macabre ideologies that Mr. Monkey promotes.

According to the latest scientific evidence, Mr. Monkey's attempts to let ruthless nincompoops run rampant through the streets are much worse than mere lexiphanicism. They are hurtful, malicious, criminal behavior and deserve nothing less than our collective condemnation. It is similarly noteworthy that we must understand that Mr. Monkey works from the false assumption that most people actually want meretricious agitators to sully a profession that's already held in low esteem. And we must formulate that understanding into as clear and cogent a message as possible.

Mr. Monkey is addicted to the feeling of power, to the idea of controlling people. Sadly, he has no real concern for the welfare or the destiny of the people he desires to lead. All I can tell you is what matters to me: I like to speak of him as "crapulous". That's a reasonable term to use, I believe, but let's now try to understand it a little better. For starters, it's undoubtedly a tragedy that Mr. Monkey's goal in life is apparently to condemn children to a life of drugs, gangs, drinking, rape, incest, verbal abuse, physical abuse, and a number of other horrors. Here, I use the word "tragedy" as the philosopher Whitehead used it. Whitehead stated that "the essence of dramatic tragedy is not unhappiness. It resides in the solemnity of the remorseless working of things," which I interpret as saying that in a recent essay, Mr. Monkey stated that he has achieved sainthood. Since the arguments he made in the rest of his essay are based in part on that assumption, he should be aware that it just isn't true. Not only that, but if he thinks that women are crazed Pavlovian sex-dogs who will salivate at any object even remotely phallic in shape then maybe he should lay off the wacky tobaccy.

A small child really couldn't understand that what I take much more seriously than libidinous jackanapes are blasphemous slaves to fashion. But any adult can easily grasp that Mr. Monkey recently stated that his biases epitomize wholesome family entertainment. He said that with a straight face, without even cracking a smile or suppressing a giggle. He said it as if he meant it. That's scary, because the key to his soul is his longing for the effortless, irresponsible, automatic consciousness of an animal. Mr. Monkey dreads the necessity, the risk, and the responsibility of rational cognition. As a result, I find that I am embarrassed. Embarrassed that some people don't realize that his equivocations will have consequences -- very serious consequences. And we ought to begin doing something about that. While some information provided by Mr. Monkey's secret police may be factual, other material is unsubstantiated rumor or money-grubbing whinges. This letter has gone on far too long, in my opinion, and probably yours as well. So let me end it by saying merely that Mr. Scuttle Monkey has no conscience and therefore no feelings of guilt for wanting to inculcate the hermeneutics of suspicion in otherwise open-minded people.

Scutters! (2, Funny)

LiquidCoooled (634315) | more than 8 years ago | (#14489548)

They only need to put it on wheels and it can become a scutter.

Additional warning:
Do not let this robot pat you on the back whilst near the top of the stairs.

Re:Scutters! (1)

IAstudent (919232) | more than 8 years ago | (#14489901)

Only a smeghead like you would come up with such thoughts. ;)

Re:Scutters! (1)

meringuoid (568297) | more than 8 years ago | (#14489999)

Do not let this robot pat you on the back whilst near the top of the stairs.

But you must be protected. You must go down the stairs...

Re:Scutters! (1)

AndroidCat (229562) | more than 8 years ago | (#14490003)

Yeah, the scutters were pretty good at sign language, weren't they?

Over Kill? (4, Insightful)

xoip (920266) | more than 8 years ago | (#14489552)

Call me culturally insensitive but, why not simply translate speech to text?

Re:Over Kill? (1)

chr1sb (642707) | more than 8 years ago | (#14489613)

I agree. This is an example of something that is fun but impractical. I assume they did it because they could, and because it would pique the students' interest in robotics, which is a worthy goal. As a real-world application however, a text-to-speech unit would be cheaper, much more robust, much smaller and more portable, most likely faster, useful to a wider market, etc. etc. But it wouldn't look as cool.

Text is not the same (2, Insightful)

QMO (836285) | more than 8 years ago | (#14489674)

Since sign languages are different all over the world, I don't know if there is the same problem in Japan, but:

American Sign Language is not English (American or other).

Thus, translating speech to ASL would reach people that that understand ASL but don't read Englih.

Re:Text is not the same (1)

emmadw (768195) | more than 8 years ago | (#14489786)

I suspect that the numbers of ASL users who can't read is small. However, as sign languages are linguistically very different from written languages, many profoundly deaf signers aren't good readers. (To start with, we have a lot of pretty meaningless words in text ... a ... the ... etc) So, signing makes it much easier to get the richness of speech. When you textualise speech, you have to cut words out, and loose a lot of nuances. (Have a look at the closed captions on the news sometime).

Re:Text is not the same (1)

magefile (776388) | more than 8 years ago | (#14490013)

Literacy is very low among people whose first language is ASL. I believe the commonly quoted statistic is that much of the deaf population reads at a 5th grade level? Thus, part of the job description of an ASL interpreter can be to translate English text into ASL.

Re:Text is not the same (1)

emmadw (768195) | more than 8 years ago | (#14490850)

Yes, I think that the statistic that I read (in the UK) is that a profoundly deaf adult (who's never had hearing), is about age 8-9 - which is I guess about the same.

Having said that, the average reading age of the average adult isn't that great - for example http://news.bbc.co.uk/1/hi/talking_point/3166967.s tm [bbc.co.uk] cites a study that say "millions of adults do not have the skills of the average 11 year old" (implying that it's because their skill set is lower, not higher!).

So, certainly in the UK, though profoundly deaf adults aren't always proficient readers, there's a significant proportion of adults who aren't deaf, who also aren't great at reading.

Hence, speech or signing benefits is likely to benefit more people than text.

ASL users and bad reading (1)

SeanDuggan (732224) | more than 8 years ago | (#14491420)

I heard a similar statistic in my ASL class. A lot of it, particularly with the older generation, is because deaf people were either put into ASL-only schools who often could not attract the better teaching talent or into speaking schools where they were actively discouraged from signing, often by tying their hands to the desks, and therefore could not properly partake in the learning process. I wouldn't be surprised if some of the bad writing processes have propagated in a manner like ebonics; as a tight community that's a definite minority and surrounded by people who largely can't understand the life they live, they likely have a taboo against too much correction of bad grammar and spelling within their community.

The grammar is, of course, different from English, but many children learn multiple languages growing up. So long as you're exposed to fluent speakers and forced to use the languages, anyone can pick up a language. The only reason very little children are seen to pick up languages easily is a) we excuse a lot of grammar and spelling on their part due to their age and b) lacking a language to fall back on, they have to learn quickly. There's a small segment of tonal languages which are easier to pick up as a child, but that has more to do with perfect (or near-perfect) pitch getting fixed at an early age.

Re:Text is not the same (0)

Anonymous Coward | more than 8 years ago | (#14489794)

Thus, translating speech to ASL would reach people that that understand ASL but don't read Englih.

1) There is virtually noone who understands ASL, but doesn't read English.

2) I think it might translate to Japanese sign language hey?

Signing, but not reading (2, Interesting)

QMO (836285) | more than 8 years ago | (#14489851)

After reading a couple of replies to my parent post, I was thinking about people that might understand signing, but not read or hear.

It is my understanding that children can learn to sign before they can learn to read. (In fact hearing children can learn to sign before learning to speak.)

Similarly, developmentally challenged people, such as certain people with Down's Syndrome, never learn to read, but can sign just fine.

Reading takes certain specific brain functions, and it is not inconceivable that there are people who have had head trauma that damaged the reading part of their brain, and the hearing part, but can still understand sign language.

These are just quick thoughts and may have lots of holes, and little sense. Please feel free to expand/correct/flame/whatever.

Re:Signing, but not reading (1)

zsau (266209) | more than 8 years ago | (#14491215)

There's also the fact that as sign languages are typically independent of spoken ones, people who know (say) Auslan will need to learn to read and write English as a second language, and the process is (I'm told) difficult.

Re:Over Kill? (4, Informative)

tpgp (48001) | more than 8 years ago | (#14489701)

Call me culturally insensitive but, why not simply translate speech to text?

Because signing is the native 'tongue' for most deaf people - and it is easier for them to communicate using sign language (over text) - just as its easier for you to understand speech (over text).

Basically - the same reason that some British TV (and undoubtedly many other channels around the world) have a signer translating the news rather then scrolling text.

Meh - then just have video of someone signing... (1)

JMZero (449047) | more than 8 years ago | (#14490622)

In the time it takes to program the robot to do a bad simulacrum of someone doing each sign, they could have just video'd someone doing all the signs. Then it's more visible to a bigger audience, too.

I'm not saying it's not an interesting project, but it's not a practical solution to the problem.

More than just words (1)

SeanDuggan (732224) | more than 8 years ago | (#14491480)

In the time it takes to program the robot to do a bad simulacrum of someone doing each sign, they could have just video'd someone doing all the signs. Then it's more visible to a bigger audience, too.
Sign languages tend to be more than just words. the positioning and motion of a sign conveys location and tense of nouns and verbs. It would be like speaking English without being able to conjugate any of the nouns or verbs.

They could, perhaps, dynamically generate pictures of the signs to convey more information, but that means you have to get more information out of the original language (often the much tricker part) and even then, you still have an issue of the signing only being visible from one direction and a fixed distance (which I suspect would be even worse for a field unit which would probably have an LCD screen, which tend to be pretty unviewable from any direction but straight on.)

OK, you're just being silly. (1)

JMZero (449047) | more than 8 years ago | (#14491607)

the positioning and motion of a sign conveys location and tense of nouns and verbs

You're just being silly. All this robot does is take words and map them to gestures. It doesn't convey all this crap you're imagining. And if you're going to do signs which require relation to body parts - as many do - you're going to need a big f'ing robot body to make it visible to lots of people - and you're back to viewing from one direction.

In the very worst case scenario, they could have a 3d representation of a hand on the screen (or on a giant screen, or on ten screens - either way still many times cheaper than a robot hand with this kind of articulation). It could then carry out exactly the same motions as the robot hand - only way, way cheaper, easier and more easily replicable. No maintenance, easier viewing, everything.

Again, this thing is cool - but 100% not practical.

Re:Over Kill? (1)

JesseMcDonald (536341) | more than 8 years ago | (#14491604)

just as its easier for you to understand speech (over text).

I realize I may be a bit unusual, but I tend to understand text faster and more accurately than speech. It is certainly faster to speak than it is to write out the text, but I can read an entire paragraph in a second or two, whereas communicating the same information as speech would take much longer. Also, I usually visualize the speech as text in my mind and then "read" it, rather than interpreting the words directly.

Visibility? (1)

SeanDuggan (732224) | more than 8 years ago | (#14489799)

A good sign language interpreter can read signs from a fair distance, well across a board room at least. How far away can the PDA be before you stop being able to read the text on the screen?

blind and deaf, teaching, and other uses (1)

snooo53 (663796) | more than 8 years ago | (#14490309)

Someone could be blind and deaf. But then why not use braille? The situation I can imagine is maybe a person knew sign language but then became blind later in life. So that would be one of the only ways to communicate. From what I understand a lot of older people have eyesight problems, so for the deaf this is even worse.

The other use could be for teaching sign language. There's a lot of people that know a little sign language, but perhaps not enough to teach someone. Seeing a robotic hand do it in three dimensions might help.

Also if you develop the technology for a sign language hand there's probably other uses for it. Imagine a robotic hand on the end of a stick that you could use to grab fragile things from high shelves. There's thousands of things it could be used for if you use your imagination

Re:Over Kill? (0)

Anonymous Coward | more than 8 years ago | (#14490368)

The idea that Deaf people could just read something and that it's "good enough" can be a rather dangerous assumption to make. In the US at least, the use of sign language varies greatly. One one end you will have people using American Sign Language which is quite different from English -- it has it's own structure/syntax/rules. On the other end you'll have people using a type of sign language that is heavily based on English word order (some of the signs are actually different in order to code the English language). And you'll even have people that float between these two extremes.

Why does this matter? Because from my experience (I worked as a sign language interpreter for 5 years - up until last year), there are a lot of Deaf people out there that don't sign using strict English word order because they don't have a strong grasp of the English language. There's been many times when a Deaf person has come up to me and asked me to interpret a news paper article because they don't understand the English. There's been many times I've had to read a Deaf person's writing and the only way I could understand it was to sign it word for word because they are doing their English is much close to American Sign Langauge than the language that we use everyday.

As for my statement I made earlier about it being dangerous to assume that Deaf people can read and fully understand English...I went to a job at a clinic. The Deaf person had been going there for 20 years and never had an interpreter (the doctor thought it was good enough to write back and forth, because all Deaf people can read English, right?). During the appointment it came out that they had many "discussions" (really writing back and forth) about the high chance that the Deaf person has cancer -- and that a major problem was this person's lack of motivation to do anything about it. After a bit, it came out that the Deaf person had no idea what the word "cancer" meant.

Re:Over Kill? (1)

Aragorn17 (946572) | more than 8 years ago | (#14491391)

I thought the same thing initially... but then I thought about deaf *children* who don't know how to read. I have a neice who falls within this category and she can sign quite a bit... but can't read a single word yet (she's only 2).

Does it also distinguish... (2, Interesting)

MadJo (674225) | more than 8 years ago | (#14489562)

... Does it also distinguish between different 'dialects' in sign language?
I seem to recall that sign languages differ between countries, same as 'natural' language.

However this is really great for deaf people.

Yes, but... (-1, Troll)

Anonymous Coward | more than 8 years ago | (#14489573)

I'm also blind, you insensitive clod!

But... (-1, Troll)

xtracto (837672) | more than 8 years ago | (#14489581)

Can it translate:

"FUCK YOU"

Into a finger sign?

ha...ha

More Useful As Software (3, Insightful)

Afty0r (263037) | more than 8 years ago | (#14489584)

Would this not be more useful as software, able to render simple 3d hands with low microprocessor overheads, and preferably available for mobile phones and PDAs?

Deaf people could carry a PDA, and when they need to find out what someone is saying, they can hold the PDA up like a microphone, and watch the screen, assuming the translation is at least reasonable accurate...
Of course they could lipread too but some find that harder than others, and this could also be used eventually to cross language barriers?

I imagine it's extremely hard to lipread a foreign language.

it's about form factor (4, Funny)

commodoresloat (172735) | more than 8 years ago | (#14489629)

Would this not be more useful as software?

Yes but not nearly as intimidating. Who's going to get their lunch money taken -- deaf kid with a PDA, or deaf kid with a giant robot hand?

Re:it's about form factor (1)

dancallaghan (890674) | more than 8 years ago | (#14489681)

Parent is not kidding! From TFA:

An 80-centimeter robotic hand

!!!

Re:it's about form factor (1)

meringuoid (568297) | more than 8 years ago | (#14490056)

Who's going to get their lunch money taken -- deaf kid with a PDA, or deaf kid with a giant robot hand?

Remember, though, that this is Japan. Kid with PDA probably merges with the Wired. Kid with part of giant robot merges with... well... pretty much everything, after a while.

Once that happens, your lunch money is the least of your concerns.

Re:More Useful As Software (-1, Redundant)

Anonymous Coward | more than 8 years ago | (#14489646)

> Would this not be more useful as software, able to render simple 3d hands with low microprocessor overheads, and preferably available for mobile phones and PDAs?

Why not render the bloody text?? If you can read sign language you probably can read text as well can't you?

Re:More Useful As Software (0)

Anonymous Coward | more than 8 years ago | (#14489654)

or just use speech to text on the pda

Re:More Useful As Software (1)

John Hasler (414242) | more than 8 years ago | (#14489861)

> Would this not be more useful as software, able to render simple
> 3d hands with low microprocessor overheads...

There is no need for 3d. I know a woman who makes her living as an ASL translator. She spends most of her time sitting sitting in front of a monitor and camera wearing earphones.

The robot hand is pointless. The computer could just as well generate cartoon images.

Re:More Useful As Software (1)

whimdot (591032) | more than 8 years ago | (#14490693)

With a real hand it could be used by deaf-blind people who need to touch the signing hand to understand.

Yes, render (1)

Poromenos1 (830658) | more than 8 years ago | (#14491136)

So, the steps are these: Recognize language, use translator (of the babelfish kind) to translate to sign language, render signing hand.

Why not just type it out to the screen? :(

Talk to the hand (0)

99luftballon (838486) | more than 8 years ago | (#14489601)

The possibilities for this are endless - converting 'wanker' into an off the wrist gesture, raising one or two fingers for the US or UK symbol for f*** off, the list goes on... On a practical level however this is surely of limited use. Conversion to text would be far more useful and allow deaf people to talk to non-deaf folk. Cute application mind you can could have some spin offs for better robotic hands in the future.

Re:Talk to the hand (1)

AllahsAvatar (887555) | more than 8 years ago | (#14489683)

I am pretty sure this is a Japanese sign for friend, but I can't find any linkage to back me up on this.

I've gotta hand it to these scientists (2, Funny)

commodoresloat (172735) | more than 8 years ago | (#14489603)

The robotic hand was shown at a two-legged robot tournament

So it's not just a hand, but a hand with two legs!

Re:I've gotta hand it to these scientists (0)

Anonymous Coward | more than 8 years ago | (#14489707)

No - the two-legged robot tournament means that the second leg is the return fixture in a couple of weeks?

foriegn language (0)

Anonymous Coward | more than 8 years ago | (#14489605)

To them, its not a foriegn language.

Other uses? (1, Funny)

Anonymous Coward | more than 8 years ago | (#14489607)

Anyone else thinking there might be some other 'alternate' use for this device?

Re:Other uses? (1)

KiroDude (853510) | more than 8 years ago | (#14489721)

yes!!! use it and ask repeatedly to translate "wanker" ...

This gives a whole new meaning (5, Funny)

sikandril (924466) | more than 8 years ago | (#14489611)

This gives a whole new meaning to the phrase "talk to the hand"

A different dept. (1, Insightful)

Anonymous Coward | more than 8 years ago | (#14489642)

This might as well be from the "doing-it-just-because-we-can" dept. As many slashdotters have already pointed out, this is pretty impractical.

Mentifex Artificial Intelligence (0)

Anonymous Coward | more than 8 years ago | (#14489669)

Mentifex AI [sourceforge.net] might benefit from this technology of sign language as a form of speech output.

Microsoft [cryptome.cn] will probably find a means of censorship for users in China.

Wonderful (1)

solanum0 (868316) | more than 8 years ago | (#14489686)

Finally deaf people can use a computer too.. err.. wait a second..

This is great and all... (1)

romiir (874939) | more than 8 years ago | (#14489702)

Without a camera that translates sign language into spoken language.. This is kinda useless isn't it?

You can talk to the hand, sure but, that doesn't help you read the deaf persons hands..
--
In retrospect: ...and yes.. I know some deaf people can talk sorta.. So I guess it helps there.

Re:This is great and all... (0)

Anonymous Coward | more than 8 years ago | (#14489747)

What deaf person? I wanted to use it as a wireless router.

The article makes no sense (2, Interesting)

MobyDisk (75490) | more than 8 years ago | (#14489712)

...that can covert spoken words and simple phrases into sign language...
Ignoring the spelling, this implies that it has speech recognition. It converts SPOKEN words.
...recognizes the 50-character hiragana syllabary and about 10 simple phrases such as "ohayo" (good morning)...
Hiragana is the Japanese phonetic alphabet, so it READS. Huh? Now, does it read only 10 simple phrases, or does it read anything plus it recognizes 10 simple audio phrases. I guess the breakthrough is the hand articulation and the idea, not the rest.

Deaf Wife/Son (1)

SirASCII (694759) | more than 8 years ago | (#14489717)

Awesome! This could be very cost saving for when my son starts school. Just train the robot on his teachers voice to sign what they are saying... It looks like the only thing I'll have to figure out how to do is to translate SEE - Signed Exact English to ASL - American Sign Language...
Hope they make a good SDK...

Universal Signs (1)

digitaldc (879047) | more than 8 years ago | (#14489720)

Can you flip off someone in Japanese? Give them the OK sign? Give them a stop with the full palm?

It would be interesting to know how these motions translate, if at all.

Re:Universal Signs (1)

Boccaccio (762644) | more than 8 years ago | (#14489771)

I teach in Japan and I've found some gestures to be the same whilst others are quite different. The OK sign in the west is often misinterpreted as it is similar to the the sign the Japanese use for money. When you call someone over to you or hail a cab you use your hand palm down and make a kind of gesture that looks like a western childs goodbye gesture. These are just a couple of examples.

Re:Universal Signs (1)

Boccaccio (762644) | more than 8 years ago | (#14489795)

Oh yeah, if you give someone the bird in Japan then, well, you're giving them the bird. Might be good to check they don't have a lot of tattoos or are driving a car with black tinted windows first ;-)

Re:Universal Signs (1)

2obvious4u (871996) | more than 8 years ago | (#14491141)

Then there is classic expample of this when Richard Nixon visited Australia and gave the sign for Victory. Well that wasn't exactly what he was doing in their minds... http://en.wikipedia.org/wiki/V_sign [wikipedia.org]

what if (1)

jaimz22 (932159) | more than 8 years ago | (#14489727)

if i tell someone to f-off will it flip them off for me?

moD down (-1, Offtopic)

Anonymous Coward | more than 8 years ago | (#14489841)

for all practical moans and groans FreeBSD showed one common goal - legitimise doing Contributed code achievemEnts that = 1400 NetBSD Cycle; take a say I'm packing

Japanese Sign Language link (1)

SeanDuggan (732224) | more than 8 years ago | (#14489845)

For the sake of being informative, here's a good page on Japanese Sign Language [deaflibrary.org] . It's not the same as American Sign Language, which isn't the same as British Sign Language as someone's sure to post eventually. *sigh* Short of Gestuno [wikipedia.org] , there is no universal sign language, no more than there is a universal spoken or written language. *rolls eyes* Except, of course, Esperanto [wikipedia.org] , which everyone speaks by now, right?

Re:Japanese Sign Language link (1)

nblender (741424) | more than 8 years ago | (#14491515)

This has always been something of sheer amazement to me. I've always been personally keen on ASL and have taken an ASL course. It hasn't done me much good because unless you practice and use it; it just goes away... But the deaf community has really lost an opportunity here. If, in retrospect, the community had developed a common sign language across cultures around the world, then I believe this USL (Universal Sign Language) would have become the lowest common denominator for communication among the non-hearing impaired. As a consequence, more people would know sign language and there would be fewer barriers for the hearing impaired as a result. Currently, when I travel to a place like Stockholm, I often see situations where a non-swedish speaking tourist, say German, will walk into a restaurant and will use very broken english to speak to the swedish waiter (who is actually Japanese and was just minutes prior speaking Japanese to his wife in the kitchen). English is the common denominator in these situations. Imagine what the world would be like if that common denominator were USL?

We tought our child sign language when he was 6mos old. By 8 mos, he was asking for 'more milk'. By 1 year, he had about 45 words in his vocabulary; long before he was able to use oral speech; he was able to tell us what he wanted for lunch; or that he was looking for a specific toy... I urge all of you geeks to look into this should you be lucky enough to have children some day.

This book is what got us started. http://www.amazon.com/gp/product/0966836774/sr=1-1 /qid=1137518491/ref=pd_bbs_1/102-4440407-6915304?_ encoding=UTF8 [amazon.com]

Recognziing Sign Language (4, Interesting)

SWroclawski (95770) | more than 8 years ago | (#14489862)

There's a researcher [gallaudet.edu] at Gallaudet working on the other side of this equation with a system designed to recognize sign langauge, which seems like a much harder problem.

ASL isn't like English in that there are always specific words- a lot of it has to do with spacial context (where in the signing space the sign was made) and a whole class of signs that don't translate directly into words (they're hand shapes which can translate into an event or a description of an object or set of objects).

And, as the research page shows, facial expressions and even facial movements can be part of a sign.

Of course, this is American Sign Language, Japanese Sign Language may be very different.

Re:Recognziing Sign Language (1)

cp.tar (871488) | more than 8 years ago | (#14489971)

And, as the research page shows, facial expressions and even facial movements can be part of a sign.

Actually, AFAIK facial expressions, body tilt etc. have a para-linguistic meaning, much like the tone of voice and facial expressions in hearing communication.

Therefore, they are not - strictly linguistically (structuralistically) speaking, part of the sign itself, but rather a part of the co-text and context.
Anyway, that's what I remember from that one lecture a year and a half ago...

Re:Recognziing Sign Language (1)

SWroclawski (95770) | more than 8 years ago | (#14490291)

That's *partially* true.

They have para-linguistic meaning but they're an important part of signining. It's not equilivant to the tone of voice as it would appear in a Western language (English, French, German), but it's often part of the sign itself in that certain facial expressions or mouthings should accompany certain signs.

They're also *vital* for things like questions, where eyebrow position are the indicator that the statement is a question. translated to English, it might be more akin to tone: "Going to the store." vs "Going to the store?"

Re:Recognziing Sign Language (0)

Anonymous Coward | more than 8 years ago | (#14490828)

The basis of the problem from the paper [gallaudet.edu] (the only one not downloadable)

"The major challenge that faces American Sign Language (ASL) recognition now is to develop methods that will scale well with increasing vocabulary size. Unlike in spoken languages, phonemes can occur simultaneously in ASL. The number of possible combinations of phonemes is approximately 1.5 x 10^9, which cannot be tackled by conventional hidden Markov model-based methods. Gesture recognition, which is less constrained than ASL recognition, suffers from the same problem."

Wait... (1)

void24601 (942027) | more than 8 years ago | (#14489897)

So now I have to learn Japanese to communicate with the deaf?

this has been done dozen of times (1)

peter303 (12292) | more than 8 years ago | (#14489949)

I remember reading about Stanford grad student project doing this ten years ago and a winner of the National Science Fair doing this three or four years ago.

i wonder... (1)

ladyKae (945309) | more than 8 years ago | (#14489956)

... will a version be demonstrated at future Ann Summers parties...

Very large? (1)

necro81 (917438) | more than 8 years ago | (#14489972)

The article states that the hand is 80 cm large (doesn't specify, but I'm guessing that's height). 80 cm is almost three feet for non-metric types. My own hand is only about 12 cm long. Is this the largest communicating hand on the planet? Or, as is more likely, the 80 cm takes into account the massive box of micromotors and computing. Pay no attention to the man behind the curtain.

Re:Very large? (0)

Anonymous Coward | more than 8 years ago | (#14490469)

I kind of doubt the 80 cm is the size of the hand, I suspect it's the length of the arm... In many languages the word for "hand" can mean the entire arm, so it might be just a translation issue.

Been There, Done That... (1)

Carcass666 (539381) | more than 8 years ago | (#14489976)

There is an article Evolution of Mechanical Fingerspelling Hands for People who are Deaf-Blind [stanford.edu] that talks about the development of this technology since 1977.

There are a couple of challenges with this type of technology. Sign language does not depend only on finger movements but gestures and facial expressions to convey emotion and context. Finger-spelling hands, being mechanical, can only accept data so fast before they start "choking" and sezing up/breaking (we tried hooking one up to a teleprompter application, and its middle finger got stuck - go figure).

This technology can be exciting on a small scale, but is not meant (not able) to act as a replacement for sign language or even closed captioning.

Not the first... (1)

necro81 (917438) | more than 8 years ago | (#14490046)

There have been other efforts to develop speech-to-sign robots. I recall one being featured on the Discovery channel many years ago that was able to fingerspell a variant of ASL that is used by persons that are both deaf and blind. That was nearly 10 years ago. In that case, the person "listens" by placing their hand over the signer's hand, and feels the different handshapes.

On another note, this sort of translation is actually more difficult than a voice-to-text, text-to-sign translation. As someone who studied sign language for several years, I can categorically state that such a direct translation is not the same as ASL, which has a different grammar and word order. The way things are described and conveyed in ASL is often not just a matter of stringing different signs that represent different words together into a sentence. Oftentimes it uses spatial and directional relations as well. Humans are able to understand these meanings quite naturally, but it is difficult to program that kind of style into a computer. The translation of voice-to-sign is just as difficult as translating between, say, spoken english and chinese. A more difficult task than voice-to-sign would be to go the other way around: a camera or motion-based system that extrapolates what a signer is saying, and then translate that to normal English.

Can't see?, you can feel the fingers... (2, Informative)

eMpTyBeitler (123072) | more than 8 years ago | (#14490191)

In the early 90's I worked with the robotic finger spelling hands called "Dexter" & "Ralph". Those devices were intended for individuals who are both deaf and blind. An individual with this kind of disability must rest their hand on the back of someone's hand (or on the back of the robotic hand) and feel letters as they are signed by the hand/fingers.

Re:Can't see?, you can feel the fingers... (1)

2obvious4u (871996) | more than 8 years ago | (#14491324)

Ding. We have a winner. Then again you could always use a braille reader for this as well. http://www.tiresias.org/equipment/eb7_b.htm [tiresias.org] But the robot hand would be more fun than a Braille Monitor.

That's Cool (1)

Greyfox (87712) | more than 8 years ago | (#14490221)

Now I can get a freaking ROBOT HAND attached to my HEAD so I can WRITE GOOD MORNING and have aforementioned ROBOT HAND sign it to a deaf guy. That's a LOT EASIER than just writing good morning and SHOWING IT TO THE DEAF GUY!

MST3000 (2, Funny)

GigsVT (208848) | more than 8 years ago | (#14490233)

Bah, Joel invented this on MST3000 years ago. Where's my edible sneakers?

Why didn't they just use a monitor with 3D image? (1)

iamacat (583406) | more than 8 years ago | (#14490445)

It's not like that robotic hand actually has to manipulate anything. That way, the program would be actually usable by any deaf person with a notebook that has a microphone.

Great! (1)

bcnu (551015) | more than 8 years ago | (#14490677)

Can I get one of these for the back of my car?

interesting but how practical? (1)

iamhassi (659463) | more than 8 years ago | (#14490694)

Wouldn't it just be easier to have the computer type the words to a screen? I mean if you have the equipment to carry around a robotic hand I'd imagine a LCD screen would be much cheaper and it could probably print more words to a screen than the hand could sign and do so faster.

guess this is more for the sake of doing rather than being practical

Yuor fail it?! (-1, Redundant)

Anonymous Coward | more than 8 years ago | (#14490966)

It's cool, but... (1)

nicholaides (459516) | more than 8 years ago | (#14491014)

Why a robotic hand? Why not simply text on a screen?

Huh... (0)

Anonymous Coward | more than 8 years ago | (#14491064)

So when someone tells a deaf person 'f--- you,' does the robotic hand give the deaf person the finger?

now if it could just translate morse code (0)

Anonymous Coward | more than 8 years ago | (#14491115)

then slashdot readers would really get excited.

Why use a robot? (1)

sbaker (47485) | more than 8 years ago | (#14491205)

Why on earth use something as complex as a robot? What's wrong with using ultra-cheap computer graphics instead? Surely the effect must be identical for the viewer. Anything with that many motors has got to be expensive and unreliable.

Oh for heaven's sake... (2, Insightful)

viksit (604616) | more than 8 years ago | (#14491516)

Don't you guys ever consider the fact that some of these breakthroughs are not built for commercial applications?
Instead of trying to analyze these achievements in the rather constricted mould of "Why not 3D graphics" or "Why not text on a screen", consider the use of this technology in the future - when say, the robots to help disabled people finally get off the assembly lines. By then, this process would've been refined to the point of being able to do an excellent job in communications.
As a researcher in the field of robotics, a lot of work which I do or goes on around me, has definite implications - if not now, atleast in the next decade or so. And don't we owe to ourselves to look at developments such as this just for the *sake of the development itself*?
Load More Comments
Slashdot Login

Need an Account?

Forgot your password?