Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Extracting Audio From Visual Information

samzenpus posted about 3 months ago | from the what-the-bag-says dept.

Science 142

rtoz writes Researchers at MIT, Microsoft, and Adobe have developed an algorithm that can reconstruct an audio signal by analyzing minute vibrations of objects depicted in video. In one set of experiments, they were able to recover intelligible speech from the vibrations of a potato-chip bag (video) photographed from 15 feet away through soundproof glass.

Sorry! There are no comments related to the filter you selected.

Not surprising (4, Insightful)

Z00L00K (682162) | about 3 months ago | (#47599079)

Measuring the vibrations of windows or other items was used already 40 to 50 years ago by spy agencies, so I wonder if this isn't something that has been re-discovered?

Re:Not surprising (5, Informative)

Z00L00K (682162) | about 3 months ago | (#47599099)

To follow up, look at the Electromax Laser Listening Systems [electromax.com] .

Re:Not surprising (1)

Anonymous Coward | about 3 months ago | (#47599129)

Bet that works really well on video.

Re:Not surprising (5, Insightful)

Hamsterdan (815291) | about 3 months ago | (#47599181)

The countermeasure for laser listening was to install the windows inside a pipe *frame* and play music in the pipes. Using an object inside the building to extract audio defeats that countermeasure. This is 2014, do not expect any privacy, especially from government agencies...

Re:Not surprising (4, Funny)

fuzzyfuzzyfungus (1223518) | about 3 months ago | (#47599297)

Clearly, if your work is that important having a window office becomes a sign of extremely low status and institutional nonimportance, rather than professional advancement...

(At least until they discover the guy spying on the basement dwellers with sophisticated seismometers)

Re:Not surprising (2)

Jeremy Erwin (2054) | about 3 months ago | (#47600455)

From Top Secret America: the rise of the surveilance state [google.com]

As important to a man's self image as the power of his car's engine or his motorcycle's rumble, SCIF size had become a symbol of status. "In DC, everyone talks SCIF, SCIF, SCIF," said Bruce Paquin, owner of a construction company that builds SCIFs for the government and private corporations. "They've got the penis envy thing going. You can't be a big boy unless you're a three letter agency and you have a big SCIF.

(A SCIF [scifsolutions.com] is a room that has been certified to be impenetrable to various types of surveillance techniques.)

Re:Not surprising (1)

skovnymfe (1671822) | about 3 months ago | (#47600883)

Is it certified to be impenetrable by human stupidity?

Re:Not surprising (0)

Anonymous Coward | about 3 months ago | (#47599691)

> government

Piffle. Wait til the advertisers do this routinely.
Hell, they will just put recording chips on every package along with the theft-detection things and pick up their profiling data from your appliances.

You won't even need to leave home to shop. They'll just deliver crap to you as long as your bank account lasts, guaranteed to be exactly what you desire.

Re:Not surprising (0)

Anonymous Coward | about 3 months ago | (#47599701)

I'd Rick Roll their ass, it's too bad they cannot turn sound into visual because they'd be getting the ol meat spin.

Re:Not surprising (4, Insightful)

timeOday (582209) | about 3 months ago | (#47599233)

Well, even a normal microphone is "just" measuring the linear displacement of a membrane over time, so clearly the important distinction is how you measure it. A laser range-finder is different from a microphone, and a video camera is different from a laser range-finder.

Re:Not surprising (1)

Anonymous Coward | about 3 months ago | (#47599869)

I heard about this system sometime in the 1980's -- I'm sure it was "secret" back then, but dudes seem to leak "cool stuff" -- so there you go.

The counter to this technique is to put two speakers set to different radio stations and aim them at the window -- or go into a shower stall and whisper in your hand with the water turned on (no, really).

I hope I don't get on a "Snowden" watch list, but as a ten-year-old, I think I could figure out a counter to every possible spook tool. It's not my fault they aren't more creative than a 5th grader with ADHD.

FYI: Tin Foil Hats are ridiculed, because THEY DON'T WANT you to wear them. It really improves the reception for my space messages ...

Re:Not surprising (4, Informative)

JazzHarper (745403) | about 3 months ago | (#47599195)

There is a very significant difference: this involves detecting vibrations in images of objects in a video recording rather than the objects themselves. However, not just any video will do; it requires a very high frame rate.

Re:Not surprising (2, Funny)

Anonymous Coward | about 3 months ago | (#47599221)

When high frame rate cameras are outlawed, only outlaws will have high frame rate cameras..............

Re:Not surprising (2)

Z00L00K (682162) | about 3 months ago | (#47600007)

The method is the same, it's just a different tool involved on the way.

It's enough to measure the image of an object, you don't need to record it first and you actually don't need a laser either, even though it may help.

Re:Not surprising (0)

Anonymous Coward | about 3 months ago | (#47600305)

Wouldn't the frame rate need to be something like 5000 frames per second to capture a normal human voice?

Re:Not surprising (4, Informative)

doublebackslash (702979) | about 3 months ago | (#47600677)

FTFA

In other experiments, however, they used an ordinary digital camera. Because of a quirk in the design of most cameras’ sensors, the researchers were able to infer information about high-frequency vibrations even from video recorded at a standard 60 frames per second. While this audio reconstruction wasn’t as faithful as it was with the high-speed camera, it may still be good enough to identify the gender of a speaker in a room; the number of speakers; and even, given accurate enough information about the acoustic properties of speakers’ voices, their identities.

They don't go into detail on the algorithm but reading between the lines it seems that they are using the spatial nature of video and the fact that not every pixel is captured at exactly the same moment (let alone each line) to ferret out higher frequency information. I have other guesses, but they are wild speculation. Either way VERY cool.

Re: Not surprising-Interesting constitutional rami (1)

Anonymous Coward | about 3 months ago | (#47600857)

Woth this discovery, I wonder what the ramifications of takng public video wothout sound are...with this technique its possible to take any video from a spy camera and reproduce the audio...what are the ramifications on wire tapping laws?

Not surprising (0)

Anonymous Coward | about 3 months ago | (#47599203)

That was done live, not from a video.

Re:Not surprising (1)

fuzzyfuzzyfungus (1223518) | about 3 months ago | (#47599289)

The general notion that all sorts of things will vibrate in the presence of ambient noise is definitely not new. Even perfectly ordinary mics depend on it, though they bring their own specialized vibrating surface in order to make the problem considerably easier.

However, there's very little similarity, aside from the use of available objects rather than specially designed surfaces; between using an interferometer to measure vibrations and using a machine vision algorithm to do so.

Re:Not surprising (0)

Anonymous Coward | about 3 months ago | (#47599349)

I think calling this "machine vision" is a little generous. Detecting these vibrations in the high speed video requires only trivial image processing.

It certainly is interesting that someone is demonstrating this. But, really, this is little more than an undergrad project for a lab that has a bit too much money on its hands.

Re:Not surprising (1)

Zeromous (668365) | about 3 months ago | (#47599293)

Agreed, this is more of a bench mark scale capability I think. Also uses video capture, not lasers which is more passive technology.

Re:Not surprising (1)

Zeromous (668365) | about 3 months ago | (#47599301)

Clarify, video is the more passive tech obv.

video frame rate = 2x higest audio freq (0, Informative)

Anonymous Coward | about 3 months ago | (#47599375)

You need a good 500 fps to recover audio from video. This has not been standard, but is possible with some cameras.

Now that's a bit disturbing.... (1)

PortHaven (242123) | about 3 months ago | (#47599105)

Puts the potato chip bag back into his lunch bag.

(How much did DHS pay for this research?)

Been there done that (3, Funny)

Anonymous Coward | about 3 months ago | (#47599109)

Sorry but that is so 2004.

- NSA

Re:Been there done that (4, Funny)

gsslay (807818) | about 3 months ago | (#47599277)

Are they not doing this already in CSI? I'm sure I saw them enhance an office security video of a post-it note, reflected off a monitor screen, magnified a couple of times, and there they had it; complete dialog in stereo, with accompanying analysis of voice stress so they knew who was lying. Isn't science wonderful?

Possible NASA method (2, Funny)

Anonymous Coward | about 3 months ago | (#47599117)

Could this be used by NASA to look for intelligent life on other worlds by measuring objects in the same fashion?

Re:Possible NASA method (1)

Anonymous Coward | about 3 months ago | (#47599149)

Depends, does it work on objects of sub-pixel size?

Re:Possible NASA method (1)

fuzzyfuzzyfungus (1223518) | about 3 months ago | (#47599363)

Depends, does it work on objects of sub-pixel size?

Just to make life more fun, a planet with life would likely have an atmosphere and those make a bit of a mess of light that passes through them(which might be handy if we are looking for atmospheres; but substantially less so if we are trying to look through them).

Re:Possible NASA method (0)

Anonymous Coward | about 3 months ago | (#47599219)

Hopefully nobody used this method on Earth, all they will hear is the noise of a million farts every day.

"Ah yes, it appears these creatures communicate using loud gaseous noises, we have managed to decipher a few of them, some seem playful, others seem vengeful and hostile in nature.
We found other noises as well, but they seem unintelligible."

Re:Possible NASA method (1)

Calinous (985536) | about 3 months ago | (#47599235)

This is best used at very high frame rates (50,000 frames per second I think) - and the "pictures" of alien planets are made with exposures of hours.

Re:Possible NASA method (1)

jellomizer (103300) | about 3 months ago | (#47599261)

In theory... However you will need to get a really good video image of a planet.
Right now most of the planets outside of our solar system is extrapolated mathematically not actually seen directly with a camera.

Then you will need to get a really really good resolution (To a point where you can probably see the life on the planet anyways) Then you will need to send it back, to earth for translation. So by the time we say hello. to them it may just be a lost language, or at least sounding quite out of place.

Re:Possible NASA method (1)

gtall (79522) | about 3 months ago | (#47600127)

I think it depends on whether the intelligent life has developed potato chips and bag in which to hold them while being eaten...under the assumption that potato chips are not toxic to the intelligent life as they sometimes are to terrestrial humans.

Scary (3, Interesting)

Anonymous Coward | about 3 months ago | (#47599121)

This is cool, yet scary stuff.

I wonder how loud the original audio has to be in order to be recovered in this manner? It sounded to me like the spoken words were being shouted, and we have no way of knowing how loud the music was played. I didn't see any mention of that in the linked article.

The linked article has additional technical(ish) information that's not in the video.

Re:Scary (0)

Anonymous Coward | about 3 months ago | (#47599211)

It sounded to me like the spoken words were being shouted, and we have no way of knowing how loud the music was played.

We don't know how loud the spoken words were played either. You can play a shouted recording with a low volume too.

Now my tin-foil hat... (5, Funny)

Anonymous Coward | about 3 months ago | (#47599137)

...Needs a tin-foil hat!

Re:Now my tin-foil hat... (4, Insightful)

JackieBrown (987087) | about 3 months ago | (#47599223)

The hat is a trick!

The reason they want you to wear foil is so that the sound can bounce off it.

Re:Now my tin-foil hat... (4, Insightful)

fuzzyfuzzyfungus (1223518) | about 3 months ago | (#47599495)

Worse than that. If there's a metal foil involved, vibration measurement should be doable with RF as well as light. Only with a next generation reduced radar cross section geometries and RF absorbent materials can a truly secure tinfoil hat be constructed.

Unfortunately, walking around with what appears to be a small F-117 attached to your head offers limited visual camouflage potential and may prove counterproductive in your attempts to avoid Their surveillance.

Re:Now my tin-foil hat... (1)

fustakrakich (1673220) | about 3 months ago | (#47599725)

What's that, a hockey joke?

To properly secure your tin foil hat, you need to cover it with cork. Quarter inch should do it.

Requires a very high speed camera (4, Interesting)

tepples (727027) | about 3 months ago | (#47599139)

The YouTube video captions state that this technique requires a camera capable of a few thousand frames per second. Thus this is pretty much using a camera to follow the vibrations, little different from a laser mic [wikipedia.org] . What would impress me more is if they were able to pick up different frequencies from different parts of the bag with different resonant frequencies and reconstruct from standard 30 fps video using the bag as a transducer.

NFL call stealing (0)

Anonymous Coward | about 3 months ago | (#47599163)

So does this mean that someone with that equipment could measure the vibrations on the play sheets coaches commonly hold in front of their face while calling in plays? In the past teams have hired lip readers to try to determine the play, this might be more accurate - or at least lead to people waving their play sheet in front of their face while they call in the play.

Re:NFL call stealing (1)

king neckbeard (1801738) | about 3 months ago | (#47599385)

I'm not sure if that would a practical usage of this. With the hardware needed for this, it would probably be cheaper to get a sensitive enough shotgun mic.

Re:NFL call stealing (1)

ColdWetDog (752185) | about 3 months ago | (#47599739)

Radar! What is the general saying?

Re:Requires a very high speed camera (4, Insightful)

interiot (50685) | about 3 months ago | (#47599241)

30 Hz is far below the Nyquist rate [wikipedia.org] (6800 Hz, going by POTS specs), so no, that wouldn't be possible without some fundamental changes in our understanding of information theory and physics.

Re:Requires a very high speed camera (2)

Uecker (1842596) | about 3 months ago | (#47599333)

No, it could work. He wants to capture different information from different parts of the bag. This is a multi-channel problem so you can go below Nyquist. Also you might have a model for speech and you can use to reduce the amount of required information. Finally, you co not need perfect recovery.

Re:Requires a very high speed camera (1)

Anonymous Coward | about 3 months ago | (#47599751)

The problem is that you would be able to pick up a maximum frequency of 15Hz.
It doesn't matter how many channels you have if all of them are completely out of your frequency range.

Re:Requires a very high speed camera (2)

Kristian Brigman (3387951) | about 3 months ago | (#47599973)

Well, it might be theoretically possible - but you'd need to get the bits from somewhere. Think of an ocean wave, and you want to measure the height of the water at a given point in time. But waves on water move in fairly predictable ways, so a single picture will tell you both the height of the water at the time the picture was taken, as well as a good approximation of what it was for a short time before and after the picture.

Another possibility is if there are multiple video streams from the same event, they are probably all 30 fps, but probably didn't catch the exact same samples - overlay them and you may be able to reconstruct a higher-frequency signal.

This doesn't make it an easier problem, or even possible - now, instead of having to capture at a frequency above the nyquist rate, you have to capture video at a resolution that can tell the micro-topology of a potato chip bag from 15 feet away. After all, you have to extract the information from somewhere. But there are ways to get beyond nyquist sometimes.

Another example which feels related but i'm not sure how yet: Roland has a patent on electronic drums. They have a single sensor in the middle of the drum, yet within a quarter-wave of a hit anywhere on the drum, they can tell both that it was hit and how far away from the center it was hit, based on the shape of the wave.

Re:Requires a very high speed camera (2, Informative)

SydShamino (547793) | about 3 months ago | (#47600147)

No, you can pick up something higher than Nyquist, as long as you understand your sources of information and noise. It will alias down into the measurable range, and you can extract useful information from the alias. We have a system that operates up to 1 MHz using a 1.8 MHz ADC. When we know the signal is at 1 MHz, we extract the information at 800 kHz and use that.

What the GGP was talking about, though, was finding resonance on the bag where unique 30-Hz-width bands higher frequencies were being naturally modulated to baseband. If you had 100 points on the bag that each modulated a different frequency (30 Hz, 45 Hz, 90 Hz, ... 1500 Hz), you could extract the data from each sub-band separately and reconstruct the original signal. See http://en.wikipedia.org/wiki/F... [wikipedia.org] and assume the source isn't one 1500 Hz conversation but instead one hundred 15 Hz conversations. And also assume that is one amazing bag of chips.

Re:Requires a very high speed camera (0)

Anonymous Coward | about 3 months ago | (#47600175)

The reason you normally can't use a 30 Hz sampling to go higher than 15 Hz, is because higher frequency signals will alias, and look like lower frequency signals. You then can't tell if a signal is the apparent frequency, or is a higher frequency that is being aliased to look lower (and if you don't have appropriate filters, you won't even be able to measure below 15 Hz if there is too much high frequency stuff around too...). If you know before hand what narrow frequency range to look at, then you can tell that a signal is being aliased and still reconstruct it. If you have several channels that only can emit very narrow ranges each (about 15 Hz bandwidth), then you will know how each would be aliased and reconstruct each from 30 Hz measurements, and then combine them together. This does require that such signals are around long enough to be picked up by several samples at 30 Hz though.

Re:Requires a very high speed camera (0)

Anonymous Coward | about 3 months ago | (#47600607)

This works only if there we're resonances scattered throughout the audible range. Given the extremely low weight of the material in a potato chip bag, I doubt there would be any. Also, given the strong coupling of the bag to the air, the Q of any resonances would probably be very low.

Now, you might have more luck with objects other than a potato chip bag. In fact, it would be cool to try to construct an object that would do this - provide resonances at reasonably high Q across the audio band on a visible surface.

Sounds like a thesis project.

Re:Requires a very high speed camera (2)

jones_supa (887896) | about 3 months ago | (#47599443)

30 fps would allow a maximum frequency of 15 Hz.

Re:Requires a very high speed camera (3, Insightful)

tepples (727027) | about 3 months ago | (#47599961)

In theory, if you can find different targets in the frame with resonant frequencies spaced no more than 15 Hz apart, you can read a different 15 Hz off each target.

Re:Requires a very high speed camera (1)

jones_supa (887896) | about 3 months ago | (#47600049)

Yes, that is true. :)

Re:Requires a very high speed camera (3, Insightful)

Anonymous Coward | about 3 months ago | (#47599989)

Oh dear. You even linked to Wikipedia (although not to the Wikipedia page "Nyquist Rate"). Does it not occur to you that OP understands those things better than you do?

To start with you need to understand what the Nyquist rate means. Sampling is like wrapping a signal around a cylinder. Just because parts are overlaid ("aliasing") doesn't mean you can't untangle the original signal. For instance, if a single audio source contains only pure harmonics, so the frequencies are known to be N, 2N, 3N, 4N, and so on, and if you have the range of possible N down to a smallish range (e.g. you know it's a voice) and you know that higher harmonics are always smaller than lower harmonics, then you can, from a massively sub-Nyquist sampling like this, extract both N *and* all the coefficients of all the harmonics. It's just like determining the dimensions of a triangle after it's wrapped around a cylinder. No, the triangle doesn't have to fit within one revolution of the cylinder, that's just the trivial case that obviously works.

What OP is proposing is that because different parts of the physical system have different resonances, when you look at that part of the image you are seeing a strongly filtered version of the original signal - basically a single frequency. You can measure the size of this signal using an aliased sampling - there's no problem with that whatsoever, it just works, an aliased sampling has the same energy as a non-aliased sampling, the samples are just in a different order. Then if you know different image areas have different responses, you can build up an image of the signal by patchwork. It would be a bloody hard job for a crisp packet in arbitrary configuration, but if you get to design the object you're looking at you can make this as sensitive as you like, and even use really crappy cameras to do it.

Nyquist rate isn't the be-all and end-all people think it is, it's just a limit for *perfect* reconstruction of *arbitrary* signals. The naive approach is to restrict yourself to sub-Nyquist signals and use the easy algorithms everybody knows. The fun stuff (read: the stuff you might get paid for) involves at least flirting with the Nyquist range, or even fully embracing that aliasing is happening and figuring out the consequences from first principles. Once you do this, you can do amazing things that seem impossible to Signal Processing 101 students ... the only problem then is you get SP101 students telling you you're an idiot for thinking that's possible. Oh, well.

BTW, sampling rate on telephony is 8000Hz as standard. Pro-tip: if you want to sound like a signal processing expert, know common sample rates.

Re:Requires a very high speed camera (1)

Anonymous Coward | about 3 months ago | (#47600967)

What OP is proposing is that because different parts of the physical system have different resonances, when you look at that part of the image you are seeing a strongly filtered version of the original signal - basically a single frequency. You can measure the size of this signal using an aliased sampling - there's no problem with that whatsoever, it just works, an aliased sampling has the same energy as a non-aliased sampling, the samples are just in a different order.

Figuring out the spatial distribution of resonant frequencies in the surface of an arbitrarily crumpled bag of potato chips to let you can invert the aliasing of a mixed-frequency human voice against 30 Hz video is so much more difficult than just buying a kilohertz framerate camera that I don't even know how to begin.

Re:Requires a very high speed camera (0)

Anonymous Coward | about 3 months ago | (#47600365)

The frame rate of video is only a few tens of Hz, but the video signal itself has a much higher band width. It may not be completely impossible to do using one of these "shaky" camera's that don't freeze a complete frame at once and using some very smart stuff. I would be impressed too by that.

Re:Requires a very high speed camera (0)

Anonymous Coward | about 3 months ago | (#47600529)

In the DSLR camera mentioned lines of pixels are recorded sequentially in time so, if the picture has 1080 lines (a standard HD image) the sample rate is actually 1080*30 (or (1080*60 depending on the format, even SD would work) which is more than sufficient a sample rate for audio as long as the sound waves are big enough to move enough of the bag (or other target) together at the same time and the target takes up sufficient space in the image.

Re:Requires a very high speed camera (2)

doublebackslash (702979) | about 3 months ago | (#47600743)

That assumes that you only are getting one sample per frame. FTFA

In other experiments, however, they used an ordinary digital camera. Because of a quirk in the design of most cameras’ sensors, the researchers were able to infer information about high-frequency vibrations even from video recorded at a standard 60 frames per second. While this audio reconstruction wasn’t as faithful as it was with the high-speed camera, it may still be good enough to identify the gender of a speaker in a room; the number of speakers; and even, given accurate enough information about the acoustic properties of speakers’ voices, their identities.

Remember that video has two spatial dimensions with 3 channels (which themselves are in different spatial locations within each pixel) each and that each line isn't captured at the same instant. There is a lot more information there than a single sample at a given rate. Nyquist doesn't apply to the frame rate here. Nyquist is stil lrelevant to the problem, of course! They didn't break Nyquist, they just found a way to get more information than intuition implies.

Re:Requires a very high speed camera (2, Insightful)

sunderland56 (621843) | about 3 months ago | (#47599267)

reconstruct from standard 30 fps video

Dear sir: what you are asking is impossible.

Sincerely yours,

Harry Nyquist

Re:Requires a very high speed camera (2)

silfen (3720385) | about 3 months ago | (#47599373)

It's not "impossible", and he even told you how to do it. Incidentally, your ear works the way he suggested.

How this get modded up? (1)

Anonymous Coward | about 3 months ago | (#47600093)

If you are going to sign your name as someone, maybe you should understand what they said first. The Nyquist limit only applies when you are trying to reconstruct an arbitrary waveform you know nothing about. If you can constrain the waveform with previous knowledge, or make some assumptions, you can go well beyond the Nyquist limit. If you know something resonates at a particular frequency (or even a particular small range), you can measure the intensity of that frequency using measurement rates way below where you would if you had no prior estimate of the frequency. Of course measuring frequencies higher than the Nyquist limit will alias, but you can still reconstruct things if you have an appropriate frequency range and know it is aliasing.

Re:Requires a very high speed camera (1)

SydShamino (547793) | about 3 months ago | (#47600165)

If you use a 1 kHz ADC to measure a 1.1 kHz signal, what do you measure at 900 Hz?

Re:Requires a very high speed camera (1)

aitikin (909209) | about 3 months ago | (#47600789)

If you use a 1 kHz ADC to measure a 1.1 kHz signal, what do you measure at 900 Hz?

450Hz. Just like, in your example of 1.1kHz measured by a 1kHz ADC, your measurement would be .650kHz or 650Hz.

Re:Requires a very high speed camera (0)

Anonymous Coward | about 3 months ago | (#47600301)

They do it with a 60fps camera by considering the row timing differences produced by the rolling shutter artifact. In this way there is no violation of the Nyquist rate.
I highly doubt that this will work equally well on a camera with a lower frame rate, though.

Requires a very high speed camera (1)

TwentyCharsIsNotEnou (1255582) | about 3 months ago | (#47599305)

From your own link:

However, countermeasures exist in the form of specialized light sensors that can detect the light from the beam. Rippled glass can be used as a defense, as it provides a poor surface for a laser microphone.

You say "little different from a laser mic". Yes, innovation is incremental, this is an increment.

Re:Requires a very high speed camera (5, Informative)

blincoln (592401) | about 3 months ago | (#47600149)

For some reason, the person who posted the article or the Slashdot editors linked to a bad knock-off video that removed 3/4 of the details instead of the actual researchers' video [youtube.com] . The real video makes it clear that they can also get results from a standard DSLR 60 FPS video by taking advantage of the rolling shutter effect. There's a fidelity loss, but it's a lot better than I would have expected.

Needs video filmed at 2000-6000 FPS (0)

Anonymous Coward | about 3 months ago | (#47599141)

Which significantly tones down the 'omg they will hear what we say in the background of youtube videos' effect.

Re:Needs video filmed at 2000-6000 FPS (1)

bobbied (2522392) | about 3 months ago | (#47599625)

Which significantly tones down the 'omg they will hear what we say in the background of youtube videos' effect.

Not to mention that you need fairly high resolution images to be able to detect the movement too. If you cannot do high resolution at reasonable frame rates you are not going to have anything to hear...

Resolution and sensor noise (2, Informative)

BitZtream (692029) | about 3 months ago | (#47599145)

The sensor and optics must have been ridiculously high quality and resolution for this to work. Sensor noise alone would almost certainly rule this out for any COTS consumer package. They certainly aren't doing it with CNN footage or old CCTV surveillance tapes.

In which case, it's of no practical value since a laser mic would be far cheaper and more discrete.

Cool from an academic perspective that they can use DSP now, but it's just more fun with a laser mic, same principals and theories, new less workable application.

Re:Resolution and sensor noise (1)

Calinous (985536) | about 3 months ago | (#47599251)

This seems to work through soundproof glass... On the other hand, how big would be a camera able to record at this resolution and frame rates, and how close it must be?

Re:Resolution and sensor noise (1)

bobbied (2522392) | about 3 months ago | (#47599693)

This seems to work through soundproof glass...

Glass that is pretty CLEAN.. You need really fast frame rates (6Khz will get you phone quality audio) and pretty high optical resolution. I'm just guessing, but you are going to need 3-4 pixels for any kind of reasonable S/N ratio that's listenable, so if the object you are looking at only moves a few nanometers with the sound, that means you need a minimum of two pixels per nanometer. To do that kind of resolution at say 10 feet, is going to require some pretty good optics. The resolution of the video itself, doesn't matter all that much, but you are going to need some serious optics and some really fine focus.

Re:Resolution and sensor noise (0)

Anonymous Coward | about 3 months ago | (#47600237)

you are going to need 3-4 pixels for any kind of reasonable S/N ratio that's listenable, so if the object you are looking at only moves a few nanometers with the sound, that means you need a minimum of two pixels per nanometer

No, no, no. Using image processing algorithms you can distinguish movement to way higher resolution than 1 pixel. That's kind of the point, and why this is news.

Let me tell you a story. I'm no expert but I once wrote (hacked?) a webcam barcode scanner that can decode a barcode without even enough pixels to distinguish every pixel of the barcode. Yeah, it surprised me too; Nyquist must have been turning in his grave! But if you think about it, there's plenty enough raw bits to get a product code out, they're just in inconvenient forms like you have greyscale where you want more black/white pixels, or you have multiple frames where you want better colour discrimination, or you have a set of similar scanlines and you want better discrimination of a single scanline. My algorithm (intended to deal with poor focus) accidentally exploited some of that extra data and BANG! I beat the (naive) Nyquist limit. Somewhat in awe, I researched that field, and what's being done with this is AMAZING. Like, take a "still life" video at 720p and reconstruct a 4Kx2K image from it. Trade temporal resolution for spatial resolution, or trade Y resolution for X resolution, or trade greyscale depth for spatial resolution. The more you know about your input image, the better, which is why the barcode problem is an accessible way into thisc. But the cutting edge is exactly the sort of thing being discussed here - things like measuring *miniscule* displacements over time, that common sense says the camera can't even see. The output isn't a 1/0 square wave, it's more like a Bayesian probability wave, and you can get *incredible* SNR, just way beyond what intuition says, because you have a ton of data from a video feed, and you can use it ALL.

This kind of thing is only the beginning, believe me.

Re:Resolution and sensor noise (1)

gtall (79522) | about 3 months ago | (#47600161)

Or Wolf Blitzer broadcasts. It is a fact that Wolf can actually suck information out of the ambient room and it is never seen nor heard from again. Think of him as an acoustical black hole.

Yeah, only if one speaks in extremely low tones... (1)

American Patent Guy (653432) | about 3 months ago | (#47599153)

After all, video normally has an update rate of 24 - 30 fps. The sampling rate will be half that at about 15 Hz. If you have to have a video camera that can take pictures at audible sampling rates (very expensive), why not just bounce an IR laser off that potato chip bag?

Re: Yeah, only if one speaks in extremely low tone (2, Funny)

Anonymous Coward | about 3 months ago | (#47599183)

Because if your target is eating SunChips you'd risk hearing loss.

Re: Yeah, only if one speaks in extremely low tone (0)

Anonymous Coward | about 3 months ago | (#47599197)

There doesn't need to be a bag. Only a very high frame rate video of a bag.

Re:Yeah, only if one speaks in extremely low tones (2)

silas_moeckel (234313) | about 3 months ago | (#47599279)

Because your emitting something sending that IR laser to do it. This is completely passive.

Re:Yeah, only if one speaks in extremely low tones (1)

American Patent Guy (653432) | about 3 months ago | (#47599453)

Yeah ... except that your spy van in your target's parking lot housing the high speed camera and its zooming lens will be obvious. With an IR laser you could do it from blocks away and nobody in the room would be the wiser.

Re:Yeah, only if one speaks in extremely low tones (1)

silas_moeckel (234313) | about 3 months ago | (#47599585)

And with good optics the camera could be just/nearly as far away. Laser is old know tech so places will have countermeasures and detection setup. Miniaturization can also come into place where you can shove the camera into a cell phone etc.

Re:Yeah, only if one speaks in extremely low tones (1)

mpe (36238) | about 3 months ago | (#47600675)

Yeah ... except that your spy van in your target's parking lot housing the high speed camera and its zooming lens will be obvious.

Why should a high speed camera look any different from a regular one?

Re:Yeah, only if one speaks in extremely low tones (1)

Anonymous Coward | about 3 months ago | (#47599423)

RTFA. They are using a high speed camera.

Why not bounce an IR laser off that potato chip bag? Because you may not be able to. Windows have IR-reflective coating, the bag is not very reflective, the surface may be at the wrong angle. This is useful.

High speed cameras are not that expensive. You can get 1000fps in a consumer camera for around $1000 these days.

Re:Yeah, only if one speaks in extremely low tones (1)

American Patent Guy (653432) | about 3 months ago | (#47600567)

You can get 1000fps in a consumer camera for around $1000 these days.

Not one with a telephoto lens that will zoom in from yards away to resolve a potato chip bag. Your consumer camera spy equipment is a pipe dream...

You'd be better off hiring a lip reader. Of course, all of this could be averted by closing the blinds in front of the window where the subject wishes his conversation to be private... I'm filing this one under "Dumb ideas".

I'm usually a cynical bastard but... (1)

Assmasher (456699) | about 3 months ago | (#47599209)

...that's pretty effin' amazing. From video, 15 feet away. Not using a laser, FROM VIDEO! Lol.

It's curtains for privacy (1)

hippo (107522) | about 3 months ago | (#47599253)

The minimalist architects are in league with the spooks!

Breached! (1)

Anonymous Coward | about 3 months ago | (#47599287)

The Cone of Silence has been breached Max!

Re: Breached! (1)

bill_mcgonigle (4333) | about 3 months ago | (#47599533)

only because the Adobe software was subject to a buffer overflow encoded in the Ruffles halftone.

Stop talking (1)

Anonymous Coward | about 3 months ago | (#47599291)

that glasshole is staring at your crisps.

The inverse would be truly phenomenal (0)

Anonymous Coward | about 3 months ago | (#47599355)

extracting video from audio.

Our Jewish 'masters' will love this... (-1)

Anonymous Coward | about 3 months ago | (#47599391)

... they can spy on us even more, and make sure we don't say anything 'anti-semitic' (like the truth).

http://balder.org/judea/Hate-Speech-Laws-Immigration-Jewish-Influence-Britain.php

The first thing (1)

kryliss (72493) | about 3 months ago | (#47599483)

This is the first thing I though of when reading the title.

SFW

https://www.youtube.com/watch?v=6i3NWKbBaaU

Class and Security (1)

khr (708262) | about 3 months ago | (#47599515)

Not only is it classier, but now serving your potato chips in a nice bowl is more secure.

Um... (1)

Ronin Developer (67677) | about 3 months ago | (#47599679)

How did they isolate the speech from the crunch of potato chips???? And, if this possible, there is no hope for anyone with the munchies!

tl;dr: (2)

CaptainStumpy (1132145) | about 3 months ago | (#47599917)

Yelling MARY HAD A LITTLE LAMB, ITS FLEECE WAS WHITE AS SNOW at a houseplant, bag of chips, and glass of water is now research.

"photographed" (0)

Anonymous Coward | about 3 months ago | (#47599931)

Is "photographed" now a synonym for "recorded" (video)? Because for a second I thought that they were claiming to have extracted a slice of audio from the ripples in a photograph... and THAT would be very impressive.

Using a high-dollar, high-resolution, high-speed video camera to capture frames of video and then processing vibrations into sound? Meh, it's basically been done before, only in real-time and using equipment that costs a whole lot less.

laws on recording (1)

Anonymous Coward | about 3 months ago | (#47599991)

The reason I find this interesting is that there are a number of locations (in the U.S.) where it is legal to record video but not audio. I happen to be against some of the laws forbidding audio recordings, because I think that it usually is used to protect against the recording of corruption. (try to open a bar in new york city, with live music, see what kinds of visits you get, and notice the correlation with fire, police, and food inspection after varying responses to those asking for money.) Would be interested in how the video gets treated in law, if one were to subsequently extract the audio for it.

Observers (1)

Cmplctd_Smplcty (2743093) | about 3 months ago | (#47599995)

Reminds me of technology used in the show Fringe by the Observers. Surely, no one will use this tech for nefarious purposes. Don't worry about it! Now, where's that Carbon Monoxide generator?

It's nothing more than laser mics (1)

LostMyBeaver (1226054) | about 3 months ago | (#47600137)

All they did was remove the laser to make the ease dropper unobservable. The tech is useless unless the camera has an insanely fine pitch resolution or the speech is so loud it causes large vibrations.

Otherwise, they're just sampling motion at 2000-6000fps. It seems ridiculously processor intensive for something which could be better achieved using a high performance light meter.

Eagle Eye (1)

advantis (622471) | about 3 months ago | (#47600637)

This is what I instantly thought of: Eagle Eye [wikipedia.org] . The scene with the soundproof room.

Next step - extract recorded sounds (1)

Anonymous Coward | about 3 months ago | (#47600655)

Amazing accomplishment! I remember using a laser interferometer decades ago to hear audio at a distance, although the quality was low. For years I've wondered whether a material setting up such as a ceramic or brick or anything that turns hard over time might be capturing nearby sounds in its micro structure. An analysis of micro density variations on an old brick or piece of pottery might be able to recreate the sounds of voices in some ancient culture - we could actually hear what Sumerian or ancient Egyption really sounded like, maybe even hear the voices of Socrates and Ramses!

Load More Comments
Slashdot Login

Need an Account?

Forgot your password?