×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Disney Algorithm Builds High-Res 3D Models From Ordinary Photos

samzenpus posted about 9 months ago | from the adding-a-little-depth dept.

Graphics 80

Zothecula writes "Disney Research has developed an algorithm which can generate 3D computer models from 2D images in great detail, sufficient, it says, to meet the needs of video game and film makers. The technology requires multiple images to capture the scene from a variety of vantage points. The 3D model is somewhat limited in that it is only coherent within the field of view encompassed by the original images. It does not appear to fill in data"

cancel ×
This is a preview of your comment

No Comment Title Entered

Anonymous Coward 1 minute ago

No Comment Entered

80 comments

Time Saver (2, Insightful)

Anonymous Coward | about 9 months ago | (#44352221)

This is great for scenery, it is amazing how much effort goes into the background scenery that no one will really pay attention to, but if you get it wrong everyone pays attention suddenly.

Re:Time Saver (0)

Anonymous Coward | about 9 months ago | (#44353887)

Meh, I'm not impressed. Why bother to report something like this when the product doesn't even exist? If I can't download or buy it right now, then they should have kept it to themselves.

Primitive, useless tech (3, Funny)

Anonymous Coward | about 9 months ago | (#44352285)

The 3D model is somewhat limited in that it is only coherent within the field of view encompassed by the original images. It does not appear to fill in data

Just have the CSI boys zoom and enhance. C'mon guys, they've been doing this for years.

Re:Primitive, useless tech (2)

ArcadeX (866171) | about 9 months ago | (#44352439)

Just have the CSI boys zoom and enhance. C'mon guys, they've been doing this for years.

Darkman did this in the early 90s, long before CSI was a glimmer in CBS's pocket book.

Re:Primitive, useless tech (5, Informative)

tom17 (659054) | about 9 months ago | (#44352653)

Excuse me?

http://www.youtube.com/watch?v=qHepKd38pr0 [youtube.com] (Bladerunner)

Bitch, please.

Re:Primitive, useless tech (2)

NatasRevol (731260) | about 9 months ago | (#44352865)

What am I supposed to bitch about? The video quality? The audio quality? The lighting? The blue blinking?

They all stink!

Re:Primitive, useless tech (0)

Anonymous Coward | about 9 months ago | (#44352881)

The Polaroid he receives when he's done, obviously!

Re:Primitive, useless tech (1)

tom17 (659054) | about 9 months ago | (#44353005)

THIS!

the sky was blue as a TV with no input (2)

Thud457 (234763) | about 9 months ago | (#44353395)

Hey gramps, what the hell is a "Polaroid"? r * sin (Hemorrhoid) ?
Next you'll go blathering on about irrational things like "phone books", when everybody all knows they're called Kindles.

Re: the sky was blue as a TV with no input (1)

z0 (69700) | about 9 months ago | (#44364759)

So you would misunderstand the meaning of, "The sky above the port was the color of television tuned to a dead channel," thinking that meant RGB blue? Gibson was referring to the grey of random noise.

I weep, no seriously (0)

Anonymous Coward | about 9 months ago | (#44353665)

I weep for this version of /.

It seems that all true geeks are gone, or if not gone they are now silenced.

Last night I made a comment about Snowcrash in the Mars Virus Crap Story (found it on idle) and nobody commented, not even the trolls.

I guess /. is too, well, (uggh, I hate to say it) mainstream now.....

Just like Bladerunner and everything else 80's.

Scheisse.

Re:I weep, no seriously (1)

NatasRevol (731260) | about 9 months ago | (#44353743)

Apparently, your humor died in the 80s too.

Re:I weep, no seriously (0)

Anonymous Coward | about 9 months ago | (#44353765)

not at all, I make fun of many things 90's, and have a particular hate for all things born after 1999. However, thanks for trying ;)

Oh yeah, I'm sure I hate you too. Now get off my lawn, kid.

Re:Primitive, useless tech (2)

JDevers (83155) | about 9 months ago | (#44353283)

To be fair, Bladerunner is set in a future world where technology is both far ahead of ours and seemingly behind ours in many ways (almost steampunk like...but forward thinking for a 1980s movie). If the camera that took that picture was more advanced than those today, it would be very possible for this to happen. Imagine a small snapshot taken with an 800 megapixel camera and this is very much possible, especially if one assumes that the actual "photo" uses might also contain an embedded memory fragment with the full resolution image.

Re:Primitive, useless tech (0)

Anonymous Coward | about 9 months ago | (#44356775)

That's just zoom. There's no enhance.

Re:Primitive, useless tech (0)

Anonymous Coward | about 9 months ago | (#44359017)

At first glance, it would appear to be a technological world where computing capability continued to increase for a period after interface peripherals stagnated. This is a sane way for an interface to be done using only a viewing screen and audio capture. Enhance is just a synonym for zoom in our lingo.

New thing same as the old thing (1)

icebike (68054) | about 9 months ago | (#44352307)

The technology requires multiple images to capture the scene from a variety of vantage points.

So simply the big difference here seems to be synthesis of properly spaced stereo cameras by using cameras positioned anywhere.

But it sounds less impressive than some of the stereo movies released from older 2D movies where there wasn't any additional
cameras, and someone simply assigned depths to parts of the image and put CGI to work on it.

Re:New thing same as the old thing (1)

interiot (50685) | about 9 months ago | (#44353131)

3D scanning is really important. Whenever we figure out how to do it faster/cheaper/easier, that's important. 3D scanning is useful for all kinds of future activities, from the maker movement (3D printer + 3D scanner = 3D copier), to gaming (eg. Kinect), to driving (eg. DARPA Grand Challenge), to mobile devices (eg. Google Glasses).

Already exists (0)

Anonymous Coward | about 9 months ago | (#44352313)

Disney's version is for making even cheaper knock-off game content.

I think this has been done for some time now (0)

TheLoneGundam (615596) | about 9 months ago | (#44352363)

See this page [pauldebevec.com]; the Campanile movie is from SIGGRAPH 97. How is Disney's tech different?

Re:I think this has been done for some time now (0)

Anonymous Coward | about 9 months ago | (#44352557)

It's automated, as it doesn't require someone to create a 3D model of the scene, which was done for that movie. There's more differences, if you'd bother to read about both.

Re: I think this has been done for some time now (0)

Anonymous Coward | about 9 months ago | (#44354277)

If you weren't a dick you'd realize several others have been automating this idea for years.

Re:I think this has been done for some time now (4, Informative)

MatthiasF (1853064) | about 9 months ago | (#44352643)

Autodesk has a service already available that does what the Disney does, it's called Recap.

http://usa.autodesk.com/adsk/servlet/pc/index?id=21350337&siteID=123112 [autodesk.com]

They have a cloud service that can make full 3D models from photos.

Re:I think this has been done for some time now (0)

Anonymous Coward | about 9 months ago | (#44355395)

I spent years making a film all from 3d scans of photos... you think I could ever get any mention on the front page of any good website, no way... vimeo.com/69136384 .... Disney does the same thing & doesn't even show it from other angles ++ they had a post a few months back like this. Perhaps I should try making a film for them?

Re:I think this has been done for some time now (1)

dacut (243842) | about 9 months ago | (#44354327)

See this page [pauldebevec.com]; the Campanile movie is from SIGGRAPH 97. How is Disney's tech different?

I saw similar technology at CMU in around that same timeframe (late 90s).

My memory will be obviously hazy here, but the resulting output was much less refined. A simple box-shaped house, for example, ended up having wickedly jagged walls. The technology showed promise, but it was far from realistic.

The Disney folks, while not inventing the tech itself, seem to have taken it a step further. Their key claim -- "Unlike other systems, the algorithm calculates depth for every pixel, proving most effective at the edges of objects" -- certainly jives with my memory.

Kinect (1)

BitZtream (692029) | about 9 months ago | (#44352377)

Yes, its fairly easy to build a 3d model given enough input and the right algorithms. Look at all the 3d scanner software that uses kinect and multiple frames to construct a higher resolution model.

Re:Kinect (2)

MickLinux (579158) | about 9 months ago | (#44353651)

Easy? Not at all. IIRC, to be able to theoretically get the model, no... let me try again: to even determine where your cameras are and how they are oriented, you need to be able to define something like 11 points in 7 photos.

At that, that just gets you to the point of having N equations, N unknowns. It doesn't give you the answer. Nor does it account for lens distortion. Throw in lens distortion, and you have that many more unknowns, therefore that many more points you'll need to define.

Having thought about it more, since then, I have decided that that isn't the way to do it. The proper way to do it is something more akin to relaxation... but you still need sufficient points. You also have to be able to define what the "same point" is. That's not easy.

That said, there are ways to make it easier. One is to first find which photos are closest to each other. To do that, you have to overlay the photos, and subtract the RGB values of each pixel. Then, run an FFT on the parts of the photos. The main frequency output of the FFT will tell you the probable shift-error in that part of the image. Try adjusting the photos that many pixels left/right/up/down (4 directions) until you find the best match, then rinse and repeat. Do this for all parts of the photo, and you will start to identify point alignments. Now work other photos together in a similar way, until you have a single network.

THEN you can use relaxation to try to find your camera positions.

THEN you can back-ray-trace, using I^4 correlation to get probable "glow spots", and then use that to generate your wireframe.

And somehow, you have to account for objects that moved, or people who were walking. Yes, it can be done by identifying different objects, but...

As I say, nothing easy about it.

Re:Kinect (1)

fatgraham (307614) | about 9 months ago | (#44354175)

You can roughly calculate camera extrinsic and intrinsic (including lens distortion) parameters from 3 world coordinates. (Assuming they're all in the photos)
One presumes with the amount of threshold and flexibility the system needs, that'll be plenty.

It's not that bad, just needs a lot of tweaking to get things to err, give decent persistent output

Re:Kinect (1)

MickLinux (579158) | about 9 months ago | (#44354661)

That's fine, if it's already known. But if it is unknown, then you've got a more difficult piece to chew on.

This is newsworthy why? (0)

Anonymous Coward | about 9 months ago | (#44352389)

Now, if it filled in data, that would be worthwhile.

All we got here is some pretty standard stuff

Re:This is newsworthy why? (1)

Tr3vin (1220548) | about 9 months ago | (#44352781)

It could still be very useful for compositing 3D effects or actors into another scene. I think that is the use case that they are targeting here.

If I've done it in Processing (0)

Anonymous Coward | about 9 months ago | (#44352517)

It's probably not new.

Reminds me of that Outer Limits episode (1)

Provocateur (133110) | about 9 months ago | (#44352583)

...where construction crews are responsible for building our universe every second. Somehow our hero accidentally falls into that realm, and sees some whitespace or voids. One of the crewmembers tells him, when nobody goes there, they don't really have to build it. That when one of their crew is negligent, and forgets to put stuff where they're supposed to be; this explains those incidents when you could have sworn you placed that hammer on that table a second ago...

Re:Reminds me of that Outer Limits episode (0)

Anonymous Coward | about 9 months ago | (#44352773)

O'h crap, I better get rid the big pile of photos of my ex I saved to use on the the bird-cage tray, just-in-case !

Re:Reminds me of that Outer Limits episode (0)

Anonymous Coward | about 9 months ago | (#44353163)

This sounds like it was based on the Theodore Sturgeon story "Yesterday Was Monday".

Affine (5, Informative)

tmarthal (998456) | about 9 months ago | (#44352599)

The name of the algorithm is called 'affine reconstruction' and is a fairly well studied algorithm in computer vision. It is great that Disney and co. are releasing software to semi-automate the data input and reconstruction.

Re:Affine (1)

gravis777 (123605) | about 9 months ago | (#44353605)

My question is, where is the software? All I saw on the Disney Research page was a Youtube video, and a link to white papers and links to download the original images. Just thinking how much I would love to take some old home movies and generate some semi-3D scenes from them. Could also be useful in 3D film conversions (although I thought that this was a similar approach to what they were already doing).

Re:Affine (4, Informative)

Anonymous Coward | about 9 months ago | (#44353655)

Possibly not for those particular use cases, but there certainly is already freely available software to do the "structure from motion" reconstruction trick; e.g., vSFM [washington.edu] -- an easy(FSVO)-to-use frontend for a couple of tools from different research projects.

Re:Affine (1)

gravis777 (123605) | about 9 months ago | (#44353851)

Oh cool, thanks, someone mod that up! I may play with this some when I get home. Now this leads to an additional quesion - both this, and the software from Disney, seems to focus mainly on buildings. I guess I can try myself, but I wonder how well this works with people? It would also be nice if I can feed in a video, but I guess I can always take my video, feed it into some video editing software and export as a jpeg or bmp securence or something.

Re:Affine (1)

Threni (635302) | about 9 months ago | (#44354235)

I think you're confusing Disney with some other company which does open source graphics software...

AutoCAD has a service for this (3, Informative)

bradgoodman (964302) | about 9 months ago | (#44352641)

It's called 123D Catch. They have an iPhone app and everything...

http://www.123dapp.com/catch [123dapp.com]

Re:AutoCAD has a service for this (0)

Anonymous Coward | about 9 months ago | (#44352757)

I believe you are thinking of Autodesk. AutoCAD is a product that is made by Autodesk, it is not the name of the company.

Re:AutoCAD has a service for this (1)

gl4ss (559668) | about 9 months ago | (#44353095)

It's called 123D Catch. They have an iPhone app and everything...

http://www.123dapp.com/catch [123dapp.com]

123D catch creates actual 3d models. in fact, any 123d catch demo is more impressive than this.. this is mostly useful for post process 3d effect to movies, it's one direction.

Ordinary Photos? (0)

Anonymous Coward | about 9 months ago | (#44352657)

"Densely sampled light fields in the order of 10^9 light rays allow us to capture the real world in unparalleled detail"

So the article abstract claims their input capture is not exactly "Ordinary Photos".

This was a cooler story when it wasnt old news. (0)

Anonymous Coward | about 9 months ago | (#44352877)

This was on here months ago, and it wasn't even that great then.

now all we need (0)

Anonymous Coward | about 9 months ago | (#44353079)

is hi res 3d naked models of miley cyrus and selena gomez. woot woot!

Dead actors, new movies (2)

chuckugly (2030942) | about 9 months ago | (#44353769)

I wonder if all those frames that stored say, John Wayne, could be used to create a fairly good 3D likeness. If not now, maybe soon. Also, who would own the rights to those performances?

Crappy gallery (0)

Anonymous Coward | about 9 months ago | (#44353781)

Gee, it would be great if they actually posted some 3D examples in their gallery. Instead we just get depth maps and reconstructions from the same POV as the original camera. I want to see the 3D models this thing generates!

I for one can't wait... (0)

Anonymous Coward | about 9 months ago | (#44354363)

I for one can't wait to buy Star Wars The Original Trilogy 3D on BlueRAY!!!!!!!!!!!!!!!!!! And if the box the new Trilogy with the old that's three threes. How could any not buy it.

Subliminal messages? (1)

Phibz (254992) | about 9 months ago | (#44354615)

Did anyone else notice flashing images on the screen while playing the video?

I can't get it to stop exactly on the image but it looks like some sort of white and red striped thing near 7 seconds in to the video.

Odd.

Re:Subliminal messages? (1)

Phibz (254992) | about 9 months ago | (#44354649)

I hate responding to my own comment but I finally got it downloaded and stepped through the video.

It's just a person in the shot wearing a white and red striped shirt.

photosynth (1)

thisisnotreal (888437) | about 9 months ago | (#44354837)

Wasn't this the first time? Photosynth demo.. http://www.youtube.com/watch?v=M-8k8GEGZPM [youtube.com]

Re:photosynth (0)

Anonymous Coward | about 9 months ago | (#44359539)

Actually it wasn't the first. ARC3D by KU Leuven already existed before that. http://homes.esat.kuleuven.be/~konijn/publications/2006/mva06.pdf
www.arc3d.be

Revisiting Stereoscopy (2)

aklinux (1318095) | about 9 months ago | (#44355249)

Overlap the photos you're taking by 60% & look at them through a Stereoscope... you get 3D.

http://en.wikipedia.org/wiki/Stereoscopy

Low-cost, accurate CNC machines? (1)

Dr. Spork (142693) | about 9 months ago | (#44355751)

One problem with cheap and homebrew CNC machines is that the cutting head loses track of its own position. The feedback from CNC machines comes to the computer from sensors on the position screws of the various axes. These can lose their calibration. I keep thinking that with this kind of technology, a computer can "visually" determine the precise position of the cutting head and also of the material being cut. It can update the computer constantly on the exact shape of the remaining material, and then compare it to the design and calculate what else needs to be cut away. Basically, all of the feedback can potentially be just optical, but of course, the computer can "eyeball it" much more accurately than a person. Because good cameras are cheap and CPU cycles are cheap, I think this could be the basis of a very flexible and affordable CNC machine. I suppose it would be important to have a good sawdust removal system, because that could potentially obscure the view - although I guess the cutting head itself could be used to sweep it away.

Re:Low-cost, accurate CNC machines? (1)

Richy_T (111409) | about 9 months ago | (#44358357)

Usually stepper motors are used. These *can* lose track of their position but it is typically not an issue. It would mean that something had jammed or bound.

already covered by slashdot... a month ago? (0)

Anonymous Coward | about 9 months ago | (#44356215)

http://tech.slashdot.org/story/13/06/23/168218/disney-research-creates-megastereo---panoramas-with-depth

Basic technique demonstrated years ago (0)

Anonymous Coward | about 9 months ago | (#44356553)

The basic technique for this was demonstrated at Siggraph some years ago, I don't remember which off hand but before 2003.

Remember Canoma? (1)

Hecatonchires (231908) | about 9 months ago | (#44356889)

From Wikipedia:
Released by MetaCreations Corp. in 1999, this application allowed users to create 3D models based on one or more photographs taken from various angles.

Great program. Never understood why it died. Assume it was corporate hijinks.

disney algorithm (2)

l3v1 (787564) | about 9 months ago | (#44358401)

In another news, the Sun is shining. I mean seriously, light-field based 3D reconstruction has been around for many years. Hell, even one of my colleagues has built a rotating table-based camera setup to capture images and create a full 3D model. Just google light fields 3D reconstruction or structure from motion and smell the coffee.

Yeah, great news.

The future is now? (1)

the_arrow (171557) | about 9 months ago | (#44358811)

So art imitating life, or life imitating art? Just saw the Futurama episode Forty percent leadbelly [wikia.com] in which Bender takes a 2-D photo of a guitar, and gets it duplicated by a 3-D printer. Seems we are in the future already!

Check for New Comments
Slashdot Account

Need an Account?

Forgot your password?

Don't worry, we never post anything without your permission.

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>
Sign up for Slashdot Newsletters
Create a Slashdot Account

Loading...