Slashdot is powered by your submissions, so send in your scoop

 


Forgot your password?
Close
typodupeerror
×
Science Technology

Brain Imaging Reveals the Movies In Our Mind 141

Posted by Soulskill from the coming-to-a-tsa-checkpoint-near-you dept.
wisebabo sends word that scientists from UC Berkeley have developed a method for scanning brain activity and then constructing video clips that represent what took place in a person's visual cortex (abstract). The technology is obviously quite limited, and "decades" away from any kind of sci-fi-esque thought reading, but it's impressive nonetheless. From the news release: "[Subjects] watched two separate sets of Hollywood movie trailers, while fMRI was used to measure blood flow through the visual cortex, the part of the brain that processes visual information. On the computer, the brain was divided into small, three-dimensional cubes known as volumetric pixels, or 'voxels.' ... The brain activity recorded while subjects viewed the first set of clips was fed into a computer program that learned, second by second, to associate visual patterns in the movie with the corresponding brain activity. Brain activity evoked by the second set of clips was used to test the movie reconstruction algorithm. This was done by feeding 18 million seconds of random YouTube videos into the computer program so that it could predict the brain activity that each film clip would most likely evoke in each subject. Finally, the 100 clips that the computer program decided were most similar to the clip that the subject had probably seen were merged to produce a blurry yet continuous reconstruction of the original movie."

This discussion has been archived. No new comments can be posted.

Brain Imaging Reveals the Movies In Our Mind More

Brain Imaging Reveals the Movies In Our Mind

Comments Filter:

  • This is late, House did it (Score:3)

    by joaommp ( 685612 ) on Friday September 23, 2011 @09:35AM (#37490538) Homepage Journal

    A couple seasons ago...

    • I remember that episode, thinking, gee, that sounds plausible, but can they really make it work? It's pretty amazing to see real world results.

  • Recording (Score:3)

    by binkzz ( 779594 ) on Friday September 23, 2011 @09:37AM (#37490556) Journal
    Does this mean I can record my dreams with Scarlett Johansen and Natalie Portman and view them at a later date? Or sell them on ebay?

    • Wake me when I can record on my PC dreams with Scarlett and Natalie, then view them in my dreams when I can fully interact with them.

      On second thought, don't wake me then ;).

      • I suppose that in principle, if you can synthesise an image from a set of neural patterns you can do the reverse. Give the computer your video, and it'll give you the neural activity associated with that video. Just get your brain wired up at a sufficiently fine resolution and let the computer stimulate your visual centres to create the appropriate visual pattern. (Of course this is probably so coarse-grained that you'd just wind up recreating the "sex" scene in Demolition Man.)

        • Re:Recording (Score:4, Interesting)

          by Doc Ruby ( 173196 ) on Friday September 23, 2011 @10:40AM (#37491430) Homepage Journal

          Well, what this system is doing is selecting from a library of video clips the 100 clips whose brain signatures when viewed were closest to the brain signatures sensed later. So the video is lo-fi because of averaging of the images. More signal processing, particularly a stats model for excluding red herrings, will give this system higher specificity in selecting the video to match the sensed signature.

          It will take another breakthru to generate a 3D image (more likely than a 2D image, which is really just a derived artifact of the 3D brain activity) directly from the sensed brain activity rather than selecting a correlating video. But that breakthru is at least as likely to occur in the SW/data video realm as in the brain/sensor realm. Because we might be able to use large video libraries, and swatches within their 2D images, as primitives from which to synthesize the 3D visual image, rather than building the recreated images from raw voxels. Which is, I believe, precisely how the brain does it.

          What will also come along is reading and stimulating other brain sensory (apperceptory, really) regions that are active when we think we're having a hi-fi memory. Often the details are not remembered, but we "remember" that we are remembering the details, in the "metadata". That level of resynth will require the breakthru of stimulating those brain regions, not just reading mappable sensory regions. But with this research at UC Berkeley I think we are now in the (very long and difficult) phase of "working out the details". We are over the watershed into the new age.

        • Re: (Score:1)

          by Wattos ( 2268108 )

          Sex is one thing, but think of the big picture:

          You could finally play WoW 24/7 without missing your sleep!

        • imagine GOATSE in 5D!

    • You've been reading my mind, haven't you?

    • Re:Recording (Score:4, Insightful)

      by twistedsymphony ( 956982 ) on Friday September 23, 2011 @10:36AM (#37491380) Homepage
      Watch out the MPAA might sue you for having dreams about their precious content.

  • There are no images in the mind. And no dogs in dog biscuits.

    • Bah, the voices in my head show me pictures all the time.

    • Any no baby in baby oil.

      • Re: (Score:1)

        by Anonymous Coward

        You'd think that wouldn't you...

    • There are no images in the mind. And no dogs in dog biscuits.

      There was a debate [lesswrong.com], in the late 1800s, about whether "imagination" was simply a turn of phrase or a real phenomenon. That is, can people actually create images in their minds which they see vividly, or do they simply say "I saw it in my mind" as a metaphor for considering what it looked like?

      The debate was resolved by Francis Galton, a fascinating man who among other achievements invented eugenics, the "wisdom of crowds", and standard deviation. Galton gave people some very detailed surveys, and found that some people did have mental imagery and others didn't. The ones who did had simply assumed everyone did, and the ones who didn't had simply assumed everyone didn't, to the point of coming up with absurd justifications for why they were lying or misunderstanding the question. There was a wide spectrum of imaging ability, from about five percent of people with perfect eidetic imagery to three percent of people completely unable to form mental images.

      Perhaps you are one of the people who do not see images in your mind. Then you would think that nobody saw them and post what you did here.

  • In the not to distant future we will have REAL thought police!
  • how quickly the MPAA sued the scientists for infringement of copyright

    • Re: (Score:3)

      by vlad30 ( 44644 )
      Actually the MPAA are investing in the technology so that every time you think of any movie they can charge you

      • Finally the RIAA can charge for every repeat of that song that's stuck in your head.

        • Finally the RIAA can charge for every repeat of that song that's stuck in your head.

          Damn. I would be soooooo screwed if that happened. Ever since I recently discovered Hungry Like the Wolf (yes, I know I am about 3 decades too late on this one, but even though the 80s were my prime years, this one somehow managed to escape my notice back then), the song has been playing in my brain several times a day for a couple hours straight.

          I would owe the RIAA a gajillion dollars or so, give or take a few.

          • Finally the RIAA can charge for every repeat of that song that's stuck in your head.

            Damn. I would be soooooo screwed if that happened. Ever since I recently discovered Hungry Like the Wolf (yes, I know I am about 3 decades too late on this one, but even though the 80s were my prime years, this one somehow managed to escape my notice back then), the song has been playing in my brain several times a day for a couple hours straight.

            I would owe the RIAA a gajillion dollars or so, give or take a few.

            Thank you so much...

            ...I'm on a hunt I'm after you... Aaagh! Shut up!

        • Can I charge them instead? After all, their song is occupying space in my head - they really ought to be paying rent for that space.

      • Actually the MPAA are investing in the technology so that every time you think of any movie they can charge you

        The plan is almost complete.

        Card transaction data can be sold and monitored; movies one has watched can be plucked out of their head. Ones who didn't purchase a movie but DO think about images and scenes from that movie are targets for MPAA's ultimate message to the world: "Told ya we had bigger balls. IN YO' FACE!"

        Shhhhhh.. Don't tell.

  • If I go watch a movie in the theater, then replay it to my friends later from my mind.. Would that be an illegal bootleg?

    • Re: (Score:3)

      by joaommp ( 685612 )

      That would be a very cool way to make spoilers.

    • If I go watch a movie in the theater, then replay it to my friends later from my mind.. Would that be an illegal bootleg?

      It would only be a bootleg if the transmission came from your leg instead of your head, and only if you are wearing boots...and then only if the boots are stolen.

    • MAFIAA is pleased to announce that in their effort to combat piracy, lobotomy is now included in the list of remedies for three-strikes infraction. Naturally any attempt to recollect any part of any film is considered as one strike.

      You could, at your option, pay a ... modest fine instead of receiving the lobotomy.

  • misleading demo (Score:5, Informative)

    by Thud457 ( 234763 ) on Friday September 23, 2011 @09:42AM (#37490636) Homepage Journal
    The video showing the original source and "output" is misleading.
    The output is not synthesized directly from the fMRI data.
    Rather, they take a bunch of samples from youtube and try to find a sample that generates the closest match the fMRI data.

    Still impressively neat. It's just that they need to more explicitly explain what they're doing.

  • Starting at @0:06 and the words. Who the heck did they test this on and why do they apparently subtitle their thoughts?

    @0:07, the words "Powershot" 2 secs in the bottom right.

    "Lot4Life" in the elephants.

    This is going to be a RTFA story it appears.

    • Its output is generated by interpolating together YouTube clips. The computer has a model of how a video maps onto a pattern of fMRI activity. Given a pattern of fMRI activity, it can attempt to generate an interpolation of YouTube clips that closely recreates that pattern.

      • For definitions of closely. That looks a lot more like a painting of a headcrab zombie on a grassy field in front of a lake of blood than a parrot.

        • True. When I first looked at it, I thought, "This is pretty nifty." Come to think of it, maybe I still think that... ...but upon further examination of the two images in the summary, I think the only direct correlation I can see between the images is the very general color of the bird and that the parrot's tail is on the correct side of its body in the "result" picture. Maybe if I were to RTFA, it would explain that the image had to be flipped to correspond or something.

          • It's also limited to the training data. Even if the brain activity can be accurately correlated to the closest training images, averaging 100 of the closest matches still has to come reasonably close to the actual scene for you to recognize it.

            • Re: (Score:2)

              by g4b ( 956118 )

              i am still skeptical, if the youtube data was really completely off limits to the reconstruction, or if the clips only show where things were similar.
              we speak about tons of data - its enough if the color definitions is taken from the clips "to find them again" as one factor and you are screwed.

              it all depends on the theory, that we know how the brain maps the seen things in neurons, and if fmri data does really show this data. i do not think, that is the case. so it seems too biased to say it really was a su

  • I think I still got it somewhere at the bottom of a paper box...

  • they want to know when you'll be sending your royalty checks to them. Please make the check out to "All Your Thought Belong to Us"...

  • nightmarish (Score:3)

    by notgm ( 1069012 ) on Friday September 23, 2011 @09:52AM (#37490788)

    that video validated every single nightmare i've ever had

  • This science is the watershed in human/machine interfaces. An improved version of this tech will give computers direct reads of our visual mind. We will imagine scenes that computers will interpret to execute.

    How far along are we getting in cheap, low-power SQUID [wikipedia.org] caps (or alternative gear) that we can wear to express to our Internet and personal processors what we imagine happening, so our machines can make it so?

  • so what now? What kind of an encrypted link of - image to brain (or sound or text to brain), are music and movie studios going to require government mandates? Because clearly, this shit is an unauthorized copy right there.

  • The fact that this is compiled from superimposed youtube clips in no way detracts from how absolutely awesome this is. If I were to tell you what I'm seeing, you would also compare my description with images that you know and imagine my vision via metaphors.... which, all things considered, works quite well in everyday life, by the way.

    • I was going to say there's no reason why they needed to use YouTube footage, and they could've used abstract shapes and patterns as the basis for reassembling the image. Then it occurred to me that given our visual system is an evolved artefact, you might actually be able to work more efficiently using naturalistic images.

  • The future is a scary place. I'm reminded of the Dune universe, in which there are no computers because the populace rose up, overthrew their AI robot masters, and banned them. I wonder when that will happen on Earth. Anybody think some day we're not going to want this sort of technology around?

    • Re: (Score:2)

      by edraven ( 45764 )

      Why would you post this sentiment online, from a computer?

    • There's a lot of technology we don't want around but are stuck with. Who would not want to put the nuclear genie back in it's bottle, if only we could? Alas, man does not often unlearn technologies.

  • the quality sucks. But honestly, BRAVO! Keep at it, guys!

  • On a whim I submitted this story (as you can see I'm the submittor) with the URL and a ONE WORD summary:

    "Unbelievable"

    The Slashdot editors either have a sense of humor and/or they don't mind doing some background research. They produced the paragraph long summary you see above and they went to the original article to clip out sone text and pictures.

    So thanks! Do you get paid? :)

    • I am glad you submitted, and glad it made it to the top. I think this is pretty incredible- the thought of one day, truly being able to see through another's eyes... And I know there are already cybernetic eyes out there, but this seems like this research could also lead to some big leaps forward in that area.

      Maybe they can help out some aging astronauts [slashdot.org], too..

    • Re: (Score:3)

      by samzenpus ( 5 ) * Works for Slashdot

      I could be getting paid? We actually do this with a lot of stories, and you're welcome!

  • While increadibly cool, I see it leading to a lot of divorces...

    Man: Honey, I had this hilarous dream last night, youv'e got to watch it! *plugs himself into the tv*
    Woman: Why is your secretary wearing that skimpy dress in your dream?!?!?
    Man: I.. uh.. that's a weird coicidence!

    • Woman: Why is your secretary wearing that skimpy dress in your dream?!?!?

      Man: Damn, honey, you're right! Why in the hell IS she wearing it? Giggity giggity giggity!

  • This is extremely impressive. That's better quality than I got on my first TV.

  • In other words, is the scanning that is done only valid
    for the test subject, or can you now use that same data
    to analyze another test subjects visual cortex ?

    • The summary strongly implies that they had to calibrate it for each of the three study participants.

      • The summary strongly implies that they had to calibrate it for each of the three study participants.

        Collectively or separately? :)

    • Re: (Score:2)

      by Ksevio ( 865461 )

      I imagine a lot of calibration would be needed - though with time that could be automated, unless it's some fundamental wiring difference in the way the data is transmitted.

      Would be interesting to see them go the other direction and transmit images into the brain (who needs physical screens?), but that would likely be quite invasive.

  • "ERROR DATABASE CONNECTION"

    I was just thinking that.....amazing.
  • I see this more of an innovative VJ technique, which is cool indeed, rather than a reliable brain reader (or even vision center reader). Nevertheless, the output could be made more psychedelic, colorful and various. The idea and implementation is indeed very fun to us, IT people, because we usually love data crunching and we are enthusiastic about sci-fi, but, seriously... The title of the post is badly chosen.
  • This is fantastic, I hope they make further progress on this soon. This is big!

  • Hook me up to this and let my wife plug in a monitor whenever she's around. Then I won't ever have to answer, "What are you thinking?" again!!

    She NEVER believes me when I give answers like, "If we give robots hands, will they pick their noses? And would running oil through air intake ducts do the same job as snot?"

    Dammit! Those are LEGITIMATE engineering questions, and yet she thinks I'm just making things up. This could just save us the whole argument, and after a while she would just give up and quit

  • Others have already mapped vision to pixels years ago [wikipedia.org]. This variation has more to do with pattern association than biology, and it's not even particularly interesting. Someone just wants more funding for their research.

  • wipe (Score:1)

    by Msdose ( 867833 ) *
    After the police get tired of reading peoples minds to match their thoughts to the crime, they will recycle old criminals by inserting the memory of the crime into their heads so they always get a guilty plea.

  • That would be wild to record someone watching Brainstorm, especially if they died.

Slashdot Top Deals

If all else fails, lower your standards.

Close