Beta

Slashdot: News for Nerds

×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

CMU Video Conference System Gets 3D From Cheap Webcams

timothy posted more than 5 years ago | from the little-bit-disorienting dept.

Communications 94

Hesham writes "Carnegie Mellon University's HCI Institute just released details on their "why-didn't-I-think-of-that-style" 3D video conferencing application. Considering how stale development has been in this field, this research seems like a nice solid step towards immersive telepresence. I was really disappointed with the "state-of-the-art" systems demoed at CES this year — they are all still just a flat, square, video stream. Hardly anything new. What is really cool about this project, is that researchers avoided building custom hardware no one is going to ever buy, and explored what could be done with just the generic webcams everyone already has. The result is a software-only solution, meaning all the big players (AIM, Skype, MSN, etc.) can release this as a simple software update. 'Enable 3D' checkbox anyone? YouTube video here. Behind the scenes, it relies on a clever illusory trick (motion parallax) and head-tracking (a la Johnny Lee's Wiimote stuff — same lab, HCII). It was just presented at IEEE International Symposium on Multimedia in December."

cancel ×

94 comments

Dundle Linux does this. (0, Informative)

Anonymous Coward | more than 5 years ago | (#26657421)

Dundle Linux has been supporting something like this since its first version, just a few months ago. Skype is already on board.

Re:Dundle Linux does this. (-1, Offtopic)

Anonymous Coward | more than 5 years ago | (#26657967)

How exactly is this a troll? It should be modded UP, as Informative, at the very least.

Re:Dundle Linux does this. (0, Troll)

harry666t (1062422) | more than 5 years ago | (#26658225)

Link please, or shut up.

Re:Dundle Linux does this. (0)

Anonymous Coward | more than 5 years ago | (#26665329)

GFE. If you can't find a link to Dundle on Google there's a problem.

Re:Dundle Linux does this. (1)

Impy the Impiuos Imp (442658) | more than 5 years ago | (#26660459)

It's not gonna give you a true 3D sensation since the image will appear identical to both eyes.

It's basically using your cam to track your head, then using software to munge up the incoming image from the other person's 2 cameras, as if your head was at that spot between the two, kind of like setting the fade and balance for audio, but for video.

But it'll still be a flat image.

But "it's very neat!"

Re:Dundle Linux does this. (1)

zippthorne (748122) | more than 5 years ago | (#26662447)

Agreed. The next step is to make a virtual mannequin head and map the face onto that. (with a very small number of knobs for fitting size and orientation) Like that Disney ride with the ghosts.

And after that, a few tricks to change the virtual viewpoint so it looks like you're looking at the camera and not the picture of the other person.

Re:Dundle Linux does this. (1)

Shotgun (30919) | more than 5 years ago | (#26666731)

No. The next step is to move the technology into a FPS. Imagine actually being able to look around the corner by...looking around the corner.

Plus, my wife will not longer be able to laugh at me for leaning around in my chair when I play.

Illin with the Panicillin? (-1, Troll)

Anonymous Coward | more than 5 years ago | (#26657429)

Is she illin in the panicillin?
Is she chillin in the panicillin?
Is she stealin in the panicillin?
Is she feelin in the panicillin?

Panka panka

Is she liable no suitifiable pliable style is so suitifiable
Is she liable no suitifiable im not on trial but its suitifiable
Is she reliable no suitifiable not just viable but real suitifiable
Is she try-able no suitifiable lying in the aisle im real suitifiable

Is she spillin in the panicillin?
Is she squealin in the panicillin?
Is she feelin in the panicillin?
Is she trillin in the panicillin?

Panka panka

Is it libel? no suitifiable pliable style is so suitifiable
Is it a style? no suitifiable im not on trial but its suitifiable
Is it a mile? no suitifiable not just viable but real suitifiable
Is it wild? no suitifiable lying in the aisle im real suitifiable

2.5D, not 3D (5, Insightful)

adam (1231) | more than 5 years ago | (#26657505)

The post title/summary is misleading -- this is actually 2.5D and not 3D at all. (It works on the premise that the background is static, and obtains a matte of the background, and using subtraction to dynamically key/mask the participant from the image, and then add the user as a second foreground layer; on the viewer side, headtracking is used to gently shift the user layer to reveal background hidden behind it)

For what it's worth, I really don't care for this effect at all. I am not denigrating its inventors in the slightest; this is a novel (read: low cost) approach, and I am sure some people would enjoy having this in their iChat/AIM/skype. To me, it's the equivalent of Apple's Photobooth filters (fisheye, inverted colors, etc) -- a cheap parlor trick that seems nifty for about 5 seconds, and then becomes precipitously distracting. True 3D has its own issues with distraction and visual anomalies (leading to headaches, etc). Even the best 3D cinematographers around have to be very careful to avoid these issues (for instance, Vince Pace, who shoots 3D for James Cameron (Titanic, Terminator, etc) has plenty of headache-inducing scenes in his demoreel, and this is a guy with state-of-the-art facilities who has as much knowledge as anyone about how to do stereoscopic cinematography). Frankly, I think video conferencing is best left 2D, and any efforts toward improving it should be spent increasing framerate/resolution (and reducing lag + dropped frames).

Re:2.5D, not 3D (2, Interesting)

MightyYar (622222) | more than 5 years ago | (#26657597)

I'm with you - while my inner geek wants to give the developers credit and is impressed, the result is not something I'd want to actually use short of screwing around with it for a few minutes.

Even if it were improved to the point where it was "perfect", it would still be just a cool trick and not a killer feature.

Re:2.5D, not 3D (1)

anilg (961244) | more than 5 years ago | (#26665037)

One thing I found was the the blackness around the edges was annoying.. it "gave" the impression of 2.5d-ness to someone who would otherwise have considered it 3D.. I'm talking Mr.LameO suddenly installing 3DChat-2009, and immediately recognizing it for a "parlor" trick.

One small thing that would go a long way in alleviating this would be cropping off edges on both sides (only in the viewing windows).. would make for a much more realistic experience.

Also, I disagree that its something you wouldn't want.. if it was included by default into a video chat, you'd be pleasantly surprised at the 3Dness of the other side. HCI works in these small unconscious way.

Re:2.5D, not 3D (1)

MightyYar (622222) | more than 5 years ago | (#26665289)

You don't think it would still look like "Viewmaster" 3-D, even if they trimmed the edges? To me it was very obvious that a flat person was being moved around on a flat background, making things more cartoonish rather than more realistic.

But maybe you are right and I would like the effect if it were polished.

Re:2.5D, not 3D (1)

anilg (961244) | more than 5 years ago | (#26666487)

With the background being a good distance away (> 10 or 15 feet) and the person within 2 feet, an object moving around a flat background is a good approximation of real 3D.

I've a very good hunch a blind test would make identifying a real 3D environment with this 2.5D would prove they look pretty much the same.

Re:2.5D, not 3D (0)

Anonymous Coward | more than 5 years ago | (#26667469)

This would be even better if there were multiple cameras. that way, when you moved your head around, it would change the perspective of the objects in the room. the images wouldn't look so flat then, and the parallax effect would give pretty convincing 3d for no glasses.

Re:2.5D, not 3D (0, Funny)

Anonymous Coward | more than 5 years ago | (#26657607)

Exactly! But people still have to publish papers about something.

Re:2.5D, not 3D (2, Insightful)

0100010001010011 (652467) | more than 5 years ago | (#26657661)

Why not just use 2 webcams a red/blue filters and a camera on the other end?

It'll be slightly annoying wearing the glasses, but it'll be much more 'real' than what this appears to be. Set the cameras eye width apart for realism or farther to make the effect more predominant.

Re:2.5D, not 3D (1)

0100010001010011 (652467) | more than 5 years ago | (#26657673)

glasses, glasses on the other end...

Re:2.5D, not 3D (1)

kramulous (977841) | more than 5 years ago | (#26658163)

Cause with Red/Blue glasses you only see black and white. I would take the colour video over the 2.5D/3D effect any day.

Re:2.5D, not 3D (1)

jack2000 (1178961) | more than 5 years ago | (#26658487)

I assume you meant grayscale, and you are wrong.

Re:2.5D, not 3D (1)

joeman3429 (1288786) | more than 5 years ago | (#26659771)

black and white is grayscale. That's why they're called black and white TVs, or black and white pictures.

Re:2.5D, not 3D (1)

jack2000 (1178961) | more than 5 years ago | (#26705577)

Black and white is different then grayscale in-my-book. Black and white uses Binary colors, gray scale has a "scale" from White through gray to black.

Re:2.5D, not 3D (4, Informative)

GameMaster (148118) | more than 5 years ago | (#26658167)

First off, the image would be an, ugly, red/blue mess. Secondly, even if you used one of the more advanced shutter glasses or polerized 3d techniques you'd still end up looking at someone wearing goofy 3d glasses abscuring eye contact. Don't get me wrong, I have no problem with wearing 3d glasses when playing games or watching a movie but not when I'm trying to converse, face to face, with someone.

Re:2.5D, not 3D (2, Funny)

blueskies (525815) | more than 5 years ago | (#26658511)

Red/blue contacts.

Re:2.5D, not 3D (1)

nextekcarl (1402899) | more than 5 years ago | (#26659991)

I've seen a couple of Real3D movies recently and I liked the effects, and I wonder if there is any reason they couldn't make contacts like that? I've never worn contacts (and my wife didn't like hers before she has laser surgery) so it might not be worth it, but I think that could be cool for certain uses.

Also, does anyone know if they can make games, etc using this tech? I tried to find something about it the other day and everything was about the red/blue system and that frankly sucks compared to the more modern techniques.

Re:2.5D, not 3D (1)

DiLLeMaN (324946) | more than 5 years ago | (#26664407)

Nike actually has MaxSight contacts [see2020now.com] that act like shades. Shouldn't be too hard to make something similar with red/blue, and it'd look even more freaky.

Dunno if polarisation lenses can be done as contacts, since those have to be exactly right, rotation-wise.

Re:2.5D, not 3D (1)

chihowa (366380) | more than 5 years ago | (#26668915)

Dunno if polarisation lenses can be done as contacts, since those have to be exactly right, rotation-wise.

Most modern polarized monitors use circular polarization, which could be easily implemented in contacts. Linear polarization was an issue even with fixed glasses, as a little tilt of the head would blend the views together and make you feel sick.

Re:2.5D, not 3D (1)

GameMaster (148118) | more than 5 years ago | (#26727893)

As the othe poster mentioned, modern 3D movie glasses from RealD use circular polarization. Not entirely familiar with ho that works but it sounds like it might do the trick. Also, my understanding is that in the case of costume/club contact lenses, like cat's eyes, they make minor changes to the shape of the contact lenses in order to keep them in the right orientation. I think that it would also be needed in order to make contact lenses to correct some vision problems like Astigmatism but I could be wrong about that.

Re:2.5D, not 3D (1)

geekoid (135745) | more than 5 years ago | (#26658835)

Who the hell uses 'red blue' 3d techniques anymore?

The DM is not always right.

Re:2.5D, not 3D (1)

GameMaster (148118) | more than 5 years ago | (#26659293)

Not many people. It was, however, included as a feature in the newer nVidia stereo 3d drivers. It was also, coincidentally, what the OP suggested and what I was responding to.

this was released at CES (1)

way2trivial (601132) | more than 5 years ago | (#26661541)

and I can't believe no one else has mentioned it
http://www.minoru3d.com/ [minoru3d.com]

it comes with red-blue glasses for the purchaser to send to people that they intend to use it with.

Re:2.5D, not 3D (1)

way2trivial (601132) | more than 5 years ago | (#26661559)

http://www.minoru3d.com/ [minoru3d.com]

I can't believe I'm the first to know this--
released at ces THIS year.. it uses red/blue and comes with crappy glasses

Re:2.5D, not 3D (0)

Anonymous Coward | more than 5 years ago | (#26658759)

You steroscopicists just don't get it. Not one wants to wear headgear of any kind. You might play around with it in the lab because it'll never amount to anything meaningful.

Re:2.5D, not 3D (-1, Offtopic)

moniker127 (1290002) | more than 5 years ago | (#26657787)

There is no such thing as 2.5D. Something is either a dimension or not.

Re:2.5D, not 3D (1, Offtopic)

RegularFry (137639) | more than 5 years ago | (#26657867)

Unless it's fractal. Actually, that's the definition.

About a million miles off topic, admittedly, but there you go...

Re:2.5D, not 3D (1)

moniker127 (1290002) | more than 5 years ago | (#26658705)

Up, down, and donut hole? What is half a dimension?

Re:2.5D, not 3D (1)

zippthorne (748122) | more than 5 years ago | (#26662383)

What is a complex number?

Math is all kinds of weird.

Re:2.5D, not 3D (2, Informative)

Chabo (880571) | more than 5 years ago | (#26657873)

Games with faked 3D are known as "2.5D" -- most notably, most side-scrolling fighting adventure games, like the Teenage Mutant Ninja Turtles series for the NES.

It's not pure 2D like the Mario/Metroid NES/SNES games, but it's not pure 3D either.

Re:2.5D, not 3D (0)

Anonymous Coward | more than 5 years ago | (#26658053)

Zuh? By that definition, isometric = 2.5D, which is nonsense.

The term has most commonly been used to describe DOOM, where any point on the map had exactly one height value. That is, you couldn't have a bridge that you could walk on top of and also cross under, or multi-level buildings.

This trick...is just utterly pointless crap. It's on par with isometric projection and the games you mention: a simple illusion.

Re:2.5D, not 3D (1)

Chabo (880571) | more than 5 years ago | (#26658283)

Pretty much any 2D game that uses some sort of trickery to emulate 3D gameplay is 2.5D.

I submit this review as evidence of the aforementioned NES/SNES games being considered 2.5D by the gaming industry:
http://www.escapistmagazine.com/videos/view/zero-punctuation/222-XBLA-Double-Bill [escapistmagazine.com]

Doom was a "3D game" in that all of the brush work was actually drawn in 3D, even though all of the entities were sprites. Each point on the map had only one height value, but the point is that different points on the map could have different values.

Re:2.5D, not 3D (1)

Curien (267780) | more than 5 years ago | (#26657931)

Someone should read about fractals [rice.edu] .

Re:2.5D, not 3D (1)

nomorecwrd (1193329) | more than 5 years ago | (#26658005)

Not really, think about games like Doom, you moved on a plane that happens to be deformed on 3 dimensions.
There was no real up/down... you could never pass under a bridge, you could only move on a 2D plane.
But this plane was deformed to give you the illusion that you where moving on 3D.. clever!... and, yes, 2.5D.

Re:2.5D, not 3D (1)

vux984 (928602) | more than 5 years ago | (#26660271)

Not really, think about games like Doom, you moved on a plane that happens to be deformed on 3 dimensions.
There was no real up/down...

Yes there was.

you could never pass under a bridge,

Yes. A map design and rendering engine limitation.

And you could only move on a 2D plane.

Not exactly. You could move freely in a 3D space limited to being an arrangement of connected non-overlapping rooms, with different floor and ceiling heights. But it was possible to freely move in 3D within that space, within the confines of the simple gravity model. But, for example, you could be below an incoming projectile, or above it and not be hit. You could move vertically by jumping off a ledge, or riding elevators. This would not be possible in a 2D plane, even a deformed one. In a 2D plane if the projectile passed through your x/y coordinate you would be hit. Doom had a z-coordinate. And it mattered.

But this plane was deformed to give you the illusion that you where moving on 3D.. clever!... and, yes, 2.5D.

It wasn't an illusion. *You* were moving in 3D. The 3D rendering engine had serious limitations, but it was 3D.

Doom was an honest to goodness 3D game. It had 2D sprites, and the level design was seriously constrained by the rendering engine, but it was as 3D as MS flight simulator.

Now wolfenstein-3D on the other hand... that would make for a better argument.

Re:2.5D, not 3D (1)

nomorecwrd (1193329) | more than 5 years ago | (#26664949)

Sorry to say, but you were fooled (just like all of us).
You could *never* dock a projectile... that's why you never need to aim up/down.
Ceiling effect was done in a similar way.
Doom was just an extension to Wolfenstein, but the rendering was similar, some sort of "ray trace" on a plane.
I insist, try to find *any* map where you could cross under *and* over a bridge.
I build a couple of deathmatch maps on my own, and that's how I remembered it... it's been a while, so I still can be probed wrong.
Do John Carmack read /.? He has the "ultimate" answer :-).

Re:2.5D, not 3D (1)

vux984 (928602) | more than 5 years ago | (#26666879)

Sorry to say, but you were fooled (just like all of us).

No, actually I wasn't.

You could *never* dock a projectile...

Of course you could. Find a monster with a slow moving projectile... an imp, or those things that threw the green balls, or the big floating heads... and find a steep staircase. They'd fire at you, and it would come right at you at whatever your elevation was (auto aim), and you could trivially dodge it simply by moving forwards down staircase, and the projectile would fly harmlessly over your head.

that's why you never need to aim up/down.

While its true you couldn't control your aim vertically, because it auto-aimed vertically, it didn't always work, and you often had to back up before it would kick in. If there was a ledge with a bad guy on it, and you ran too close to it and fired, instead of hitting the guy at the top, your weapon would harmlessly bounce off the wall *below* him.

I insist, try to find *any* map where you could cross under *and* over a bridge.

I *know* that you can't. I *know* that was a limitation of the rendering engine and map editor. But its immaterial. But that isn't a requirement for 3D. Its true the map was limited to a 'deformed plane', but it still carved out a 3D space that you could move in.

You don't need bridges to have a 3d world. If you made a half life counterstrike map of an empty field with some barrels and sandbags for cover, it would still be a 3D game.

I build a couple of deathmatch maps on my own, and that's how I remembered it... it's been a while, so I still can be probed wrong.

No. You are quite right about the map limitations. You are just wrong in that it wasn't 3D.

Re:2.5D, not 3D (5, Insightful)

JustinOpinion (1246824) | more than 5 years ago | (#26657881)

I agree with you: having this kind of 2.5D experience is neat but not particularly useful.

But I wonder if this software could be adapted to do something else... One of the things that most people dislike about webcam-conferencing is that the other person is never looking "at" you. They are looking on their screen at an image of you, so they are not looking directly at their camera, and so on your end they seem to be looking away from you. (And they see you looking away from them, too.)

While this may seem trivial, it is actually a significant roadblock to inter-person tele-communication. People rely on body language and eye contact to establish each other's moods, to really "connect". Webcam-conferencing forces us to violate social conventions (like looking into people's eyes), which can be anywhere from subconsciously bothersome, to somewhat distracting, or even perceived as insulting.

So what I would like is a multi-camera system that uses similar kinds of interpolation to rebuild the image of the person so that they are looking directly at the camera. So if I put one webcam on either side of my screen, they can combine their images to create a shifted image where I am looking directly at the viewer on the other end.

Though it is a rather small and subtle addition to tele-conferencing, I believe it would have a bigger impact than what TFA seems to be showing. I think it would make the interaction "more real."

Re:2.5D, not 3D (2, Interesting)

kramulous (977841) | more than 5 years ago | (#26658205)

Or, just put the stream of the conferenced person just below/above and centred on the camera. I've operated Access Grid a couple of times and this is the first thing that I do.

Re:2.5D, not 3D (0)

Anonymous Coward | more than 5 years ago | (#26658891)

I believe it would have a bigger impact than what TFA seems to be showing. I think it would make the interaction "more real."

...and if the camera captures enough pictures of me and my babble, my computer could simulate me too and I could stay in bed.

Re:2.5D, not 3D (2, Insightful)

Anonymous Coward | more than 5 years ago | (#26659501)

I did a study about this gaze problem and a possible algorithmic solution, for a videoconf specialist about one year ago.
My conclusion:
no algorithm was/is/will be suitable to combine any point of view with any other point of view. Consider an object occluded for each point of view but not occluded for your virtual view (the combination of the two actual views): there is no solution but to guess areas that can be very wide and for situation very frequent (if 2 objects are near, like your hand and your body for example). This would create unacceptable artifacts that would be far more annoying than the gaze effet.
And to answer your question that you are about to ask: why our brain can do it? It can't. Try it, honestly.
Moreover, cameras placed on both side of the screen would be too far away for a stereoscopic system.
What the paper does not say either is that no algorithm can segment any video (or say differently, any segmentation algorithm can be fooled by specific situations), and equivalent artifact (to the combination artifact) will be produced by this pseudo-3D: it is sometime un-decidable. Unacceptable for professionnal systems. It's a toys.

Re:2.5D, not 3D (3, Informative)

forkazoo (138186) | more than 5 years ago | (#26659895)

So what I would like is a multi-camera system that uses similar kinds of interpolation to rebuild the image of the person so that they are looking directly at the camera. So if I put one webcam on either side of my screen, they can combine their images to create a shifted image where I am looking directly at the viewer on the other end.

Geometric view interpolation is not unknown in the labs right now, and in some cases is being researched for exactly the reason you suggest. As another poster suggested, there are certainly some cases where the interpolation will break down. (Put a hand in front of each webcam at the side of your monitor, and it won't interpolate two palms to look like your face, for example.) Another one is that anything transparent makes it impossible to estimate the depth at a particular point because there are actually two depth values there. So, the smoke from your cigarette which is an amorphous volume of semitransparency through which you can see a window, the schmutz on the window, a reflection on the window, and something through the window will just ruin any chance of doing the interpolation properly. When you try to shift the pixel correctly to accomodate for the view shift, you get like seven different answers for what direction it is supposed to go.

Still, look up the Foundry's "Ocula" system for 3D cinematography. It's a shipping commercial product that does a lot of strong magic with stereoscopic imagery on a daily basis. (Which i would have assumed was currently impossible.)

It's too slow to be used for real time conferencing. You let it cook overnight for a single shot, or a handful of shots to compute disparity maps offline. It needs to be at least an order of magnitude faster to be practical for real time work. Thankfully, there are a lot of researches trying to figure out clever hacks to speed up these sorts of things, and a lot of engineers figuring out ways to build stonking GPU's to run OpenCL in a year or two. Expect stereo stuff to become mainstream somewhere around 2011-2012 would be my guess.

Re:2.5D, not 3D (0)

Anonymous Coward | more than 5 years ago | (#26664781)

You are right for transparency, amorphous volume, etc. In fact any complex scene would be impossible to combine if taken from 2 point of view: hairs, water, any jewelry,
What I tried to study was if any simple scene would have been difficult to combine. I found that very simple scene, likely to occur often, were impossible to combine: impossible because the algorithm would have to invent large part of the video.
I actually studied the state-of-the-art disparity map algorithm, and though a lot of progress were made, none will ever be able to invent missing area. Period.
The Ocula system will do a good job, not because of the CPU power available during a night, but because a human will correct its limitation, tweak its parameters for special cases, invent with an artistic point of view, like photoshop is used by professional nowadays.
Another poster said something insightful: videoconf should do "higher definition, less latency," etc... and not use tricks to correct problems that cannot be solved algorithmically.

Re:2.5D, not 3D (1)

amirulbahr (1216502) | more than 5 years ago | (#26660457)

My first idea in response to this was to put the camera somehow behind the display. Maybe by having a translucent display or perhaps there is some technology out there in which the display emitters could also be used as detectors.

So I jump on to Google and it turns out Apple has already patented [appleinsider.com] my idea. How did that pass the test of novelty and non-obviousness for a patent claim?

Re:2.5D, not 3D (1)

dangitman (862676) | more than 5 years ago | (#26661065)

So what I would like is a multi-camera system that uses similar kinds of interpolation to rebuild the image of the person so that they are looking directly at the camera. So if I put one webcam on either side of my screen, they can combine their images to create a shifted image where I am looking directly at the viewer on the other end.

Sounds overly complicated. Why not just put the camera behind the screen, so the user is actually looking directly at the camera, rather than faking it?

Re:2.5D, not 3D (1)

VanessaE (970834) | more than 5 years ago | (#26664059)

Not to rain on your parade, but there's just no way this would work with current full-sized monitor tech.

A CRT would need to have, at the very least, some optics embedded into the tube so that the camera itself could remain outside, and then you're interfering with the beam no matter where you put those optics. Besides, CRT's are pretty much obsolete except for a few corner cases.

An LCD is out because you'd have to poke a small hole in the backlight reflector and diffusing layers for the camera to see out of, which means you'll end up with a really big dead pixel.

Front projection is out - either it'll light-fog and wash out the image the camera picks up, or the user will end up blocking part of the projected image. Rear projection is out because even the best optics will still be big enough to obscure part of the projected image. In both cases, you still have to poke a hole in the screen.

Plasma is out because the pixels are simply too big to be useful on something like a computer monitor.

An argument can be made for LED or OLED displays, since the pixel density is enough to be useful, but I don't seem to recall there being any options to cheaply replace the average full-sized monitor.

Seems to me that it would just be easier to use some kind of teleprompter setup with a half-reflective mirror, a good, sensitive webcam, and a really *bright* run of the mill monitor.

If you really want a compact unit, here's how I would do it with current (if expensive) technology:

Make an OLED display of a useful size, e.g. 17-19 inches or better. Make a 1/8 to 1/4 inch square area in the center of the screen transparent, if that's possible (I'm not sure). Put the camera behind that area of course, but sandwich in an LCD shutter that can block the lens when opaque. Seal everything against ambient light. Have the monitor's firmware logically separate a 1x1 inch area around that transparent spot from the rest of the screen, and have said firmware flick the shutter and that 1x1 area on/off together at a fairly high speed (say, 150-200 Hz), with an on-time of no less than 95%. Obviously, the LCD would be opaque when the 1x1 area is "on". Have the monitor's firmware automatically balance the rest of the screen's brightness, contrast, etc. against the now dimmed 1x1 area, so that the differece isn't apparent. The setup should look like any ordinary monitor to both the user and the video card/monitor drivers. The camera can be any ordinary consumer-grade webcam, as long as it can take good images in very low light without excessively long exposure times. It can integrate however long it likes, thanks to the LCD shutter.

I figured a 1x1 inch area would be large enough to help keep the camera from picking up glare from the parts of the screen around it that are always on. If necessary (and assuming it is even possible), erect a microscopic opaque "wall" around the 1x1 area in the spaces between adjacent pixels, to block some of the incident light.

Re:2.5D, not 3D (1)

crmarvin42 (652893) | more than 5 years ago | (#26657887)

I agree. It's neat, but not really useful for anything.

I think getting the whole "eye contact" thing worked out would be much more useful as a way of making the experience feel more "natural". I am used to looking at peoples faces and having them look at mine when I chat with someone in person. Video chatting requires you to either get accustomed to people looking over your head/off to the side the whole time, or only watch the video with your periferal vision so that the person you are chatting with gets the impression you are looking them in the eye while they are talking.

Neither one is comfortable for me, so I tend to just call or do voice chats to avoid the weird factor.

Re:2.5D, not 3D (2, Insightful)

dinther (738910) | more than 5 years ago | (#26659369)

Very cool and I like the fact that they give the webcam a double function. The 2.5D effect against a static background is indeed novel only but I see there is a confusion between 3D and stereo vision.

I agree with most writers that stereo vision induces headaches which are simply due to the fact that the eyes each see a different image which giver your brain a depth cue, yet your eyes focal point (To your screen) conflicts with that depth cue thus resulting in a headache. It is unavoidable with normal screens.

However, if it was possible to determine for each pixel in the webcam image at what depth it occurs then each pixel could be placed in 3D space and a virtual camera could be used to rotate around the subject.

For this you would need two web cam's and some real clever software to figure out the depth of each pixel. So the result here is a 3D video model of a real life subject shown on a 2D screen like is done with every 3D game.

But I suppose such software is difficult to write as the Darpa urban challenge with autonomous cars proved. Those guys used extensive sensor arrays that were used to build a 3D model of the road in front of the car.

Not even 2.5D (0)

Anonymous Coward | more than 5 years ago | (#26665157)

This is not even 2.5D. 2.5D was the term used for games like Doom, where you had a flat floor plan, plus floor and ceiling height (but couldn't have things above each other).

This is more like the side scrolling games from the Amiga days, except they usually had four layers moving at different speeds, rather than the two layers presented here.

In the Amiga days, nobody called those games 3D or even 2.5D. It was simply the way 2D side scrollers were made back then.

Some cartoons use the same technique, and have done so since long ago.

The tech is cool, sure.. (5, Insightful)

Quarters (18322) | more than 5 years ago | (#26657531)

...but that sample conversation at the end of the video may have well been between two drunken epilepsy sufferers on boats in the North Atlantic. Who moves around like that while they are talking?

Re:The tech is cool, sure.. (1)

Jonah Hex (651948) | more than 5 years ago | (#26660865)

I kept thinking of Stevie Wonder, and after watching that I understand why only he can move like that, it's extremely visually disconcerting and headache inducing.

HEX

Re:The tech is cool, sure.. (1)

d0rp (888607) | more than 5 years ago | (#26666797)

Clearly this is meant for playing games of virtual dodgeball!

Did any body else read that as... (0)

Thelasko (1196535) | more than 5 years ago | (#26657571)

CGNU [homestarrunner.com] Video Conference System Gets 3D From Cheap Webcams

Brilliant (1)

sepelester (794828) | more than 5 years ago | (#26657653)

This does tons for immersion! It has to be implemented wherever there is a stationary camera (it obviously doesn't work with a camera phone). IIRC, Johnny Lee's work was free to use, so get to it and add that "Enable 3D" checkbox, developers! If only they'd cropped the resulting image to get rid of the black-ground, but that was probably just to show how it works.

IBM did it before (0, Offtopic)

Anonymous Coward | more than 5 years ago | (#26657699)

Back in 1988 at an IBM convention in Vegas a similar technology was demonstrated running on a OS/2 server. The premature termination of OS/2 caused of course the end of the project: I am happy to see that somebody revived again the idea.

Re:IBM did it before (1)

denis-The-menace (471988) | more than 5 years ago | (#26658823)

[citation needed] or else corps will try to patent it.

same as wii head tracking vr (0, Redundant)

t35t0r (751958) | more than 5 years ago | (#26657705)

Re:same as wii head tracking vr (1)

RiotingPacifist (1228016) | more than 5 years ago | (#26657869)

i thought that failed if you have two eyes?

Re:same as wii head tracking vr (0)

Anonymous Coward | more than 5 years ago | (#26658049)

Congratulations on reading to the end of TFS.

Re:same as wii head tracking vr (0)

LingNoi (1066278) | more than 5 years ago | (#26658939)

Johnny Lee didn't invent this, it's been done tons of times before he even started his PHD thesis on using the wii mote.

Look at the paper "3D display based on motion parallax using non-contact 3d measurement of head position"

All Dr Lee did was a much simpler demo with a 3d box and 2D sprites using a wii mote instead of a camera and now everyone worships him like he's so amazing for it.

If you did some research into it you'd realise that his demo sucks and if you read his paper he doesn't go into any detail at all about it. If I remember correctly it was one paragraph in his entire paper.

The only reason which makes Johnny's demo so popular is because he put it on youtube, that's it.

Re:same as wii head tracking vr (1)

Zerth (26112) | more than 5 years ago | (#26661257)

Duh, of course he didn't invent anything, he just hooked a wiimote up to a PC and used it to provide positioning for a camera in a virtual scene. Nothing special there.

The reason it wowed eveyone is A) because nobody at nintendo thought to demo it first and B)it let everyone at home do the same thing for way cheaper than before.

Game control? (0)

Anonymous Coward | more than 5 years ago | (#26657749)

This would be really cool for controlling a videogame, like looking around a corner or something. Just add the webcam to the controls, and use lean left/right in real life to do the similar things in the game...

Re:Game control? (3, Informative)

Wumpus (9548) | more than 5 years ago | (#26657825)

John Carmack prototyped this a few years back. His conclusion at the time was that there was too much lag in the system to make it really useful.

Re:Game control? (1)

Chabo (880571) | more than 5 years ago | (#26657909)

Yeah, FPSs that wish to implement leaning have established a convention by now of using Q and E for that purpose. Call of Duty was one of the first games I played to implement it.

Re:Game control? (2, Interesting)

nomorecwrd (1193329) | more than 5 years ago | (#26658191)

5 years of applying Moore's law should have overcome this by now.
Much better/clever implementation than for video conferencing.
Come on... be honest, everyone has done that unconsciously on Counterstrike... even without a webcam ;-)

Re:Game control? (1)

Wumpus (9548) | more than 5 years ago | (#26658307)

The lag wasn't due to CPU speed - it was due to cumulative delays in the webcam itself, the USB bus, and only a tiny bit of image processing. I think his analysis was done on his .plan proto-blog, way back. I have no idea where it might be archived these days.

I know that even today, when capturing video from a USB camera, I can see a noticeable delay between when I move an object and when I see it moving on the screen, so I don't think that much changed since then. The only video capture setup I'm aware of that doesn't suffer from this problem is when you capture video via a PCI capture card at a high frame rate, and most people don't bother to set something like this up.

Re:Game control? (0)

Anonymous Coward | more than 5 years ago | (#26658617)

Stop using such cheap setups?

Lag like you describe hasn't been a problem for at least two years. And sorry, Carmack isn't the be-all-end-all of truth when it comes to "what's possible".

Re:Game control? (1)

Wumpus (9548) | more than 5 years ago | (#26659527)

My setup isn't that cheap, I would guess that most people's setups would be cheaper, and I don't see how you can speculate on the amount of lag I'm experiencing when all I said was that it was noticeable, which it is.

Anyway - if you want to implement it, go right ahead. Don't let some guy on Slashdot stop you.

Re:Game control? (1)

autophile (640621) | more than 5 years ago | (#26659401)

John Carmack prototyped this a few years back. His conclusion at the time was that there was too much lag in the system to make it really useful.

1. Cite? 2. Because as everyone knows, as time goes on, CPU doesn't get faster and RAM doesn't get bigger.

Re:Game control? (1)

Wumpus (9548) | more than 5 years ago | (#26660419)

Sorry, can't find the place where I saw this. The closest I came was this:

http://doom-ed.com/blog/1999/11 [doom-ed.com]

This is an archive of his old .plan updates in blog form. I know that the actual .plan updates are archived somewhere on www.bluesnews.com, but I can't figure out where they are. That post just mentions that he started working on it, but there's no followup there. I do remember reading a followup somewhere else some time later, and he mentioned the latency issue.

The latency had nothing to do with the CPU speed, and everything to do with the camera buffering a couple of frames before sending them on the bus. Granted, cameras are better today, but I can still see the latency. At this point, it's probably acceptable for most applications, but people tend to notice latency in games more than they would in other applications.

Re:Game control? (1)

I3ooI3oo (1215428) | more than 5 years ago | (#26658033)

I think they already have something along those lines. http://www.naturalpoint.com/trackir/ [naturalpoint.com]

Re:Game control? (0)

Anonymous Coward | more than 5 years ago | (#26658645)

Police 911 was a light gun arcade game that used motion sensors to control your ducking behind objects in a 'time crisis' like timed shooter

http://en.wikipedia.org/wiki/Police_911

drunk dialing (0)

Anonymous Coward | more than 5 years ago | (#26658009)

Just watched the video. I'm sure this will go over real well for those drunken late night videochats.

Im waiting for the mod... (0)

Anonymous Coward | more than 5 years ago | (#26658083)

That makes a girls boobs pop out 3x larger.

Bandwidth reduction? (3, Interesting)

Anonymous Coward | more than 5 years ago | (#26658263)

I wonder if a more practical use would be to use the technique for video bandwidth reduction. If you know where the person is, you could concentrate video bandwidth on the face region, while keeping the rest of the "video" relatively static. No point in continuously compressing and sending boring background. Of course many codecs already do temporal compression that gives a similar effect, but this might increase the efficiency for video chat.

Max Headroom (0)

Anonymous Coward | more than 5 years ago | (#26658377)

W-w-wow! I'd n-n-never guess they'd thought of t-t-this in the 80's!

Great (1)

MrEricSir (398214) | more than 5 years ago | (#26658785)

Now I can see everyone's zits in 3d.

No good enough (0)

LingNoi (1066278) | more than 5 years ago | (#26659001)

The reason we'd be moving left to right would be to see something which isn't in the frame.

An idea I have been thinking about for awhile is to have the remote camera move when the user on the other side does. This would be much more convenient instead of having to ask the person to keep adjusting the camera angle to see outside the frame.

I worked on that too. Look at these vids... (3, Interesting)

dinther (738910) | more than 5 years ago | (#26659061)

Inspired by Johnny Lee's stuff, I pulled some old code out over a year ago and turned it into a decent engine that handles multiple screens and head tracking (TrackIR) to achieve the motion parallax effect. Like with all 3D effects, it needs to be seen but the following videos give you a good idea.

Have a look at these demo videos and you can even download a demo:

My first test
http://nz.youtube.com/watch?v=X8PevTuEWlg [youtube.com]

More accurate tracking
http://nz.youtube.com/watch?v=yf1hu6GLmf0 [youtube.com]

Multi screen study
http://nz.youtube.com/watch?v=ZBdtPz2V_vY [youtube.com]

Engine complete
http://nz.youtube.com/watch?v=ku76aHq3pps [youtube.com]
Download Demo
http://vandinther.googlepages.com/virtualwindow [googlepages.com]

Re:I worked on that too. Look at these vids... (1)

mugnyte (203225) | more than 5 years ago | (#26661251)

  I just ran your demo, quite nice although a click-drag on the head (instead of fly) would be more educational.

  Since head tracking has a common solution, there's no need for IR (although precision is better). You should open this and get it connected to standard head tracking. It'd be quite nauseating, even with the lag. But that's a compliment in this area.

DrunkChat (tm) (1)

dangitman (862676) | more than 5 years ago | (#26661033)

It looks like the application for this is chatting when you are drunk, standing up, and swaying about. I don't know anybody who constantly moves their head around when videochatting. They tend to look straight into the camera. And wouldn't you be rather concerned if the person on the other end of your chat did start moving around and looking at you from weird angles?

Ma Ma Ma Max (1)

nategoose (1004564) | more than 5 years ago | (#26661295)

The floating square of background with a floating talking bust reminds me of Max Headroom.

Better than Star Trek! (0)

Anonymous Coward | more than 5 years ago | (#26661435)

Wow...that looks better than what Star Trek people get when they say "on screen"!

Sorta backwards (0)

Anonymous Coward | more than 5 years ago | (#26663577)

Wouldn't it make more sense if the image moved as it tracked the other persons movement instead of yours? Otherwise you look sorta like a jagof swaying back and forth.

The Wonder cam arrives. (1)

The Real Tachyon (1332153) | more than 5 years ago | (#26668293)

While this is pretty neat, I'm not sure it 'enhances interpersonal communication' since everyone using it will be bobbing back and forth like a Stevie Wonder impersonator convention.

Not to mention some schmuck in the US will soon sue because it made them puke from motion sickness.

Why not just use two webcams? (1)

The Real Tachyon (1332153) | more than 5 years ago | (#26679589)

USB webcams are pretty cheap these days. Why not use two, one on each side of the monitor?
In fact I've seen web cam kits with 2 in the package.
The would let you have true parallax, AND would have the benefit of making it appear that you are looking at the viewer.
Solves the two main problems I see being discussed here for an extra $29.95 or so.
Plus, it would make cool things like 3D position tracking possible (think Minority Report).
Check for New Comments
Slashdot Account

Need an Account?

Forgot your password?

Don't worry, we never post anything without your permission.

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>
Create a Slashdot Account

Loading...