Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Computer Program Allows the Blind To "See" With Sound

samzenpus posted about 8 months ago | from the flipper-vision dept.

Science 56

sciencehabit writes "Scientists have developed an algorithm that converts simple grayscale images into musical soundscapes. Even people blind from birth can use the technology to 'see' their surroundings and navigate around a room. Equally intriguing, the part of the subject's brain responsible for vision was active during these tasks, suggesting our thinking about how the brain works may be wrong. Instead of a 'vision center' of the brain, for example, we may actually have a region that helps us 'see', whether that input comes from sight or sound."

Sorry! There are no comments related to the filter you selected.

LOL (-1)

Anonymous Coward | about 8 months ago | (#46424847)

Do you hate beta? Do you hate Dice.com? Do you hate the technically-ignorant newfags that are infesting Slashdot? Do you hate niggers, jews and fags? Well don't fret! Join SoylentNews! A site by straight, white nerds for straight, white nerds!

But can you (3, Funny)

Anonymous Coward | about 8 months ago | (#46424947)

play a mean pinball?

Re:But can you (1)

chispito (1870390) | about 8 months ago | (#46425979)

play a mean pinball?

Not if you're deaf dumb and blind. It conveys visually information auditorily.

Re:But can you (1)

davester666 (731373) | about 8 months ago | (#46426025)

This will totally rock Daredevil's world.

Re:But can you (1)

nuckfuts (690967) | about 8 months ago | (#46426793)

Sure!

Re:But can you (1)

Matheus (586080) | about 8 months ago | (#46428595)

I think Ben Affleck starred in that one...

Re:But can you (1)

lazy genes (741633) | about 8 months ago | (#46432391)

Finally be a contestant on the Wheel of Fortune. I really wish they would introduce a system that would allow them to play.

Grayscale may not be best (4, Interesting)

ShanghaiBill (739463) | about 8 months ago | (#46424951)

I am not sure if "grayscale" is the most useful information to a blind person. A few years ago I tried on some ultra-sound goggles designed for the blind. The cool thing about it was that, with practice, by listening to the pulses, I could not only tell how close an object was, but also how rigid or dense it was. A pillow would sound very different from a rock. Just by listening, I could "look" at two soda cans on the shelf, and tell which one was empty. Of course, it gave no information about color.

Re:Grayscale may not be best (1)

Anonymous Coward | about 8 months ago | (#46425687)

If you continue past the summary they are teaching patients to interpret color as well (see the video attached to the article).

Re:Grayscale may not be best (1)

mindriot (96208) | about 8 months ago | (#46426815)

Except that video doesn't prove anything of that sort. It merely shows that the participant has learned to recognize specific sound patterns. For all we know from the video, he may have been told the following:

  • First sound pattern playing
  • Interviewer: This is John. He is bald and has black eyes and a goatee.
  • Second sound pattern playing
  • Interviewer: This is Lisa. ...
  • ...

...as a result of which, the participant is able to recognize the pattern and recite what he has learned.

If they wanted to demonstrate that the participant could really recognize the faces from the sound patterns, they should have presented him with a face he hadn't seen/heard before (i.e. didn't know the name of the "person"), and asked him to describe what he "saw".

Anyone with access to the Science article, did they at least report such findings in the actual paper? Because the video is worthless.

Re:Grayscale may not be best (1)

mikael (484) | about 8 months ago | (#46427385)

Vision works in the following. Take a stereoscopic picture. The two images give you depth information. You can use edge detection algorithms to determine what pixels belong together as an object (segmentation) and reconstruct a cardboard cutout view of the world. From each 2D cutout figure, the brain finds the closest matching known 3D object and constructs an internal 3D representation of the scene with information consisting of two things; the object and it's orientation relative to the person (distance, scale, rotation).

Now, you could replace the stereoscopic picture with the sound input. Then brain makes a closest match between the type of sound and known objects. Another method is to place an electrode grid on the tongue and a similar form of vision becomes possible.

Re:Grayscale may not be best (1)

mindriot (96208) | about 8 months ago | (#46428047)

Vision works in the following. Take a stereoscopic picture. The two images give you depth information. You can use edge detection algorithms to determine what pixels belong together as an object (segmentation) and reconstruct a cardboard cutout view of the world. From each 2D cutout figure, the brain finds the closest matching known 3D object and constructs an internal 3D representation of the scene with information consisting of two things; the object and it's orientation relative to the person (distance, scale, rotation).

I get that, I work with 3D point clouds and stereo and RGBD sensors every day.

Now, you could replace the stereoscopic picture with the sound input. Then brain makes a closest match between the type of sound and known objects. Another method is to place an electrode grid on the tongue and a similar form of vision becomes possible.

I get that too. That is the idea and the claim, anyway. The point of my post was: The video posted with the ScienceMag article [sciencemag.org] does not prove that this is what happens! The task shown could have been accomplished by memorizing the sounds and the descriptions given by the interviewer. That's still fairly impressive. But from the video, we do not know whether the participant actually does what the authors, and you, say he does – namely, actually recognizing and understanding the picture. As I said, the video is worthless.

In different terms, what the video shows is that the participant reproduces the classification on the training data. What's missing is the test data that the participant didn't use during training.

As I don't have access to the full article, I would like to know if anyone could tell us about whether the conclusion is at least supported from the actual data presented in the paper. It definitely is not supported from the video.

Re:Grayscale may not be best (4, Informative)

hozozco (856621) | about 8 months ago | (#46426485)

My Daughter is legally blind. She has rod monochromatism often called achromatopsia (http://www.achromatopsia.info/) and doesn't see any colour (only grey scale). She seems to think greyscale is quite useful. Now obviously colour would also be useful, but greyscale allows you to 'see' most things. In some states in the USA people with achromatopsia can drive using bioptic glasses (http://www.biopticdrivingusa.com/achromatopsia/). Of course the rest of the world sees this and thinks 'only in America'. Anyway, this is interesting if not entirely new. Other students at my daughters school (for the visually impaired) also learn echo location - this is from my daughters school: http://www.abc.net.au/btn/stor... [abc.net.au] :-)

Wonderful (1)

guygo (894298) | about 8 months ago | (#46425057)

Geordi's glasses can't be far behind. Excellent.

Re:Wonderful (0)

Anonymous Coward | about 8 months ago | (#46425227)

Daredevil as well for that matter. Although it might take a while to retrain Ben Affleck after his role as Batman...

Development in Science (4, Interesting)

jawnah (1022209) | about 8 months ago | (#46425069)

I'm glad to see a post that positively promotes development in science. For some reason, our "science" has stagnated lately in my opinion with "scientists" taking hard-line approaches to situations - they are no longer thinking out of the box and force everyone to think in the box or be ostracized and labeled "stupid". I agree that there is more likely a part of the brain that helps us "see". I believe that the data used by that brain center can be different to produce different results: 1) You see with your eyes the events happening in front of you. 2) You see with your mind when you recall a sequence of events, situation, or even dream.

Re:Development in Science (2, Insightful)

narcc (412956) | about 8 months ago | (#46425191)

It's not science or scientists so much as it is the thundering hoard of scientifically illiterate science cheerleaders -- the self-appointed elite defenders of "science".

They're worse than creationists.

Re:Development in Science (1)

Anonymous Coward | about 8 months ago | (#46425541)

Who says science has stagnated? I look around and I'm seeing rapid advances in science and technology and it's just getting more rapid. The only thing stagnating is people's ability to keep up and comprehend what's really happening.

Re:Development in Science (0)

Anonymous Coward | about 8 months ago | (#46429263)

He means innovative advances. Revolutionary things like the steam engine was in it's age.
But you're right, we have plenty of innovation, not just improvements of old technology. And maybe not like the steam engine, but we're not slowing down at all.

Re:Development in Science (0)

Anonymous Coward | about 8 months ago | (#46426727)

""Equally intriguing, the part of the subject's brain responsible for vision was active during these tasks, suggesting our thinking about how the brain works may be wrong.""

I'm sorry but your post is way off, your not on a far right wing site.

This has been known for years, there's nothing "intriguing" about it. The brain depending on a persons age, can re-program itself if something like eye sight is lost, same with hearing. /. articles are becoming repetitive anymore reporting things that have been long known, however a new generation of kids maybe will learn to listen to those that have been around, and read articles that will cause them to learn something as well, that they didn't know.

No shit! (1)

Jmc23 (2353706) | about 8 months ago | (#46425095)

'Visual' cortex is just a bunch of pattern processors. This becomes obvious to anybody who repurposes them to do math.

Re:No shit! (0)

Anonymous Coward | about 8 months ago | (#46425319)

If this is "obvious" to you then I think you jump to conclusions way too quickly. The most I would ever say is that it's plausible.

Also, the idea that you know that you repurpose the visual cortex to do math (as opposed to thinking you do that) without having shown that is pretty bold.

plus 3, 2Troll) (-1, Redundant)

Anonymous Coward | about 8 months ago | (#46425121)

Not only gray scale (1)

roman_mir (125474) | about 8 months ago | (#46425189)

If you actually watch the video and RTFA (OMG, what a noob, obviously) you'll notice that they now can even encode and transmit and successfully read colours within image, which is even more impressive. Nicely done. But it doesn't seem to be possible to use this with motion, you can see a static image, I don't believe this can be done with a movie, they would have to read the entire image many times over, frame by frame (the image is played as a series of audio notes, sounds like a melody) and each simple image takes a few seconds to play.

But I think it's possible to modify this idea, so that instead of fine features, you get information about objects coming closer or moving away from you, take 2 snapshots, figure out the difference in motion, play the difference rather than a snapshot.

Can you do it with pressure? (1)

GoodNewsJimDotCom (2244874) | about 8 months ago | (#46425225)

I think wearing laser range finders around, and having pressure around your body depending on how close it is to objects could be more information too. I could be wrong, but I would like some feedback on this reddit post [reddit.com] I made a day or two ago. Can't hurt to discuss this stuff.

Re:Can you do it with pressure? (0)

Anonymous Coward | about 8 months ago | (#46428337)

Any haptic feedback would work.

[Per your specific idea: having a laser ranger or ultra-sonic device in a small torch-like handle, with a haptic "display" under the finger-tips, would give greater feedback than a simple cane. Wearing force-feedback devices all over your body is kind of excessive (and would look retarded.) You just need it on areas that have a high number of nerve-endings (such as the finger tips), and the brain will adapt.]

What bothers me about the tendency to use sound feedback (because it's easier for device makers to include a speaker than a haptic "display") is that I would expect that sound would already be central to a blind person's ability to understand their surroundings. Intruding on that seems counter-productive. Haptic "displays" would make more sense, touch-at-a-distance. The only time the device would be in the way is if you're close enough to touch something directly, which is when you don't need the device anyway.

Somewhat old news (2, Insightful)

Anonymous Coward | about 8 months ago | (#46425231)

I have a friend who is working on his PHD in experimental psychology at the moment, and we had a discussion about this 4 or so years ago. Apparently this is the new paradigm of thinking in the neuroscience world, or has been on the verge of becoming the new paradigm for some time now. The brain is just a pattern recognition engine, in much the same way that we are able to wire a monkeys brain to a robotic arm, and over time have that brain adapt to control this new appendage, the brain will take any signal that seems like it might be visual and process it as such.

the blind to see and the lame to walk (1)

turkeydance (1266624) | about 8 months ago | (#46425295)

science, believe it or not, involves just a little faith that it might, just might, work.

Once again, not new (0)

Anonymous Coward | about 8 months ago | (#46425299)

This is hardly new technology, I remember reading an article in Popular Science back in the 60's about a walking cane for the blind that scanned the area immediately in front and produced a series of tones informing the user as are to obstacles and distances.

Re:Once again, not new (0)

Anonymous Coward | about 8 months ago | (#46428019)

The point of this article isn't the tonal "display" device, it's demonstration of the brain to adapt purely visual processing areas of the brain to purely aural ones when using a tonal display.

[I know it's hard to believe, but there's an occasional non-Slashvertisement that slips in when the editors are tired.]

There are people who can echolocate (2)

Khashishi (775369) | about 8 months ago | (#46425431)

I saw an episode of a documentary/show Extraordinary People on Ben Underwood, and it was incredible what he was capable of, including riding a bicycle and playing basketball using echolocation. This is without any computer program or aid. Unfortunately, he died in 2009. As far as I know, only a few people possess this skill, but it can apparently be taught. The world expert is probably Daniel Kish.
http://en.wikipedia.org/wiki/H... [wikipedia.org]

Re:There are people who can echolocate (1)

camperdave (969942) | about 8 months ago | (#46426147)

There are people who can echolocate

Anyone else read that as e-chocolate?

Re:There are people who can echolocate (1)

cellocgw (617879) | about 8 months ago | (#46428599)

There are people who can echolocate

Anyone else read that as e-chocolate?

No, because "chocolate" is a color, not greyscale (you insensitive clod).

Portal? (0)

Anonymous Coward | about 8 months ago | (#46425441)

I hope this is just like pointing at the underside of a light bridge in Portal 2.

Pah! This blind guy *cycles* without any aids (1)

umafuckit (2980809) | about 8 months ago | (#46425493)

Check out this guy: completely blind but can cycle by making clicking noises [youtube.com] . i.e. he's a bat.

Re:Pah! This blind guy *cycles* without any aids (1)

ketomax (2859503) | about 8 months ago | (#46425629)

I remember that from the Stan Lee's Superhumans episode.

Re:Pah! This blind guy *cycles* without any aids (1)

Buchenskjoll (762354) | about 8 months ago | (#46426407)

My bicycle makes clicking noises. I wasn't aware it was part of the navigational system.

Some blind people can see with echolocation. (0)

Anonymous Coward | about 8 months ago | (#46425531)

It's called human echolocation.

http://en.wikipedia.org/wiki/Human_echolocation

There is a institution for the technique.

http://www.worldaccessfortheblind.org/

Military Application's? (0)

Anonymous Coward | about 8 months ago | (#46425723)

I can see DARPA's next project. Ground forces total darkness/incapacitating light sonar, this would be very useful for ground forces imagine a urban environment war constant light/sound disruption from artillery or grenades (deployed by ground troops for room clearing) constant thunder-flashes/flash-bangs and soldiers with this technology able to see/acquire targets throughout any visual disruption.

Except It Doesn't (3, Insightful)

Baby Duck (176251) | about 8 months ago | (#46425743)

This does not change how we think of the brain. We've know for decades from synesthesia that sensory cortices have no choice but to process any inputs they receive.

You're maybe not incorrect (2)

ubergeek (91912) | about 8 months ago | (#46426701)

But they're wrong in a more important way: We've believed for years that the visual cortex is actually a visualization center! It just happens that when we're awake and looking with our eyes the visualization is constrained by sensory inputs. Sensory (and even association) cortices are basically simulators, that contain our best models of the world (what we expect the world to be like, based on prior experience), and the parts of those models that are active are dynamically constrained in real time by sensory data. When we have no sensory input at all, those models can run freely (this is essentially what is happening when we dream or have out of body experiences).

When individuals completely lose (or are born without) input for one modality, there's no reason to think they couldn't still use the corresponding cortical hierarchy for modelling (visualizing) that aspect of the world. The reported research is good evidence that this is exactly what happens.

Re:Except It Doesn't (0)

Anonymous Coward | about 8 months ago | (#46427293)

It might change how say doctors think brain works, it certainly doesn't change how we think neural networks(including brains) work. The basic principle is pretty much the same, regardless of scale or implementation. Neural network is neural network, it doesn't matter if its human brain, ant brain or a computer program.

An old, old idea (1)

Ozoner (1406169) | about 8 months ago | (#46425805)

This was an old idea even when it was "invented" back in 1992.

Re:An old, old idea (1)

gnupun (752725) | about 8 months ago | (#46425961)

But will "sound" as vision be tolerable over the long term? Imagine having to listen to non-stop sounds/music your entire waking hours in a day. That's bound to cause earache and sound weariness.

Re:An old, old idea (1)

mark-t (151149) | about 8 months ago | (#46426117)

No more so than keeping your eyes open and subjecting them to photons every waking moment.

Re:An old, old idea (1)

Ozoner (1406169) | about 8 months ago | (#46426133)

An interesting thought: Especially as we can shut our eyes, but not our ears?

Re:An old, old idea (0)

Anonymous Coward | about 8 months ago | (#46426405)

An interesting thought: Especially as we can shut our eyes, but not our ears?

Then pull the power plug from the vision-to-sound system.

Re:An old, old idea (1)

mikael (484) | about 8 months ago | (#46427413)

No different from watching a TV show with lots of flashing lights. My womenfolk can no longer watch some of those talent shows because the program makers completely overdid the lighting effects - they had spotlight patterns moving up and down behind the singer, spinning patterns all over the floor and the spotlight patterns moving sideways to each side of the stage.

Fovea? (0)

Anonymous Coward | about 8 months ago | (#46426403)

Seems like it would be more effective if it scanned radially around a central point. The higher frequencies can be more concentrated similar to the fovea of the eye.

Maybe a quadtree indexed together with the repeating 12 notes of the scale?

Yawn (0)

Anonymous Coward | about 8 months ago | (#46426449)

You are aware that bats have a working visual cortex? It's not just for their rather underused eyes. And in fact, so have blind mole rats.

converts simple grayscale? (1)

Benzainload895 (3510209) | about 8 months ago | (#46426777)

Eventually this may elevate the works of Science. This aims the blind one to be being capable to see though their own vision.

New but old (1)

clickclickdrone (964164) | about 8 months ago | (#46426923)

Back in the 70's at least they had white walking sticks with echo location devices on them that allowed blind people to 'see' objects in the room. It was in a 'how it works' book I bought as a kid.

Re:New but old (1)

Ravaldy (2621787) | about 8 months ago | (#46429409)

But like anything, it evolved to the next level so it's news worthy. People had to work to achieve the level they are at now. The concept isn't new but the achievement is.

The idea and theory on how to go on the moon existed way before we could actually do it.

Old news? (0)

Anonymous Coward | about 8 months ago | (#46427011)

So the actual news are about the brain responses, the software has been around since 1992...

vision center (1)

dkman (863999) | about 8 months ago | (#46428381)

Instead of a 'vision center' of the brain, for example, we may actually have a region that helps us 'see', whether that input comes from sight or sound."

How about instead of a 'vison center' you call it a 'spacial awareness center'. Then it fits the bill no matter where that information is coming from. Because vision can be broken down into "this is over here" and "that is over there". Now it would be interesting to know if the 'vison center' is active in the blind people who use echo-location, that would solidify my claim.
Just because I devised my claim based on the above quote doesn't make it invalid, only not well tested.

We've just been mislabelling our brain region (1)

Wolfier (94144) | about 8 months ago | (#46430779)

If the "vision" center had been named "geometry" center, its activation would be very natural and not a surprise at all.

Check for New Comments
Slashdot Login

Need an Account?

Forgot your password?