Slashdot is powered by your submissions, so send in your scoop

 


Forgot your password?
Close
typodupeerror
×
Graphics Software Technology

Colorizing Images and Video by Scribbling 272

Posted by timothy from the only-a-child-can-do-it dept.
Guspaz writes "Up until now, colorizing a video or image has been a painstaking and mostly manual task. However, researchers in Israel have come up with a new way of colorizing images just by making a few scribbles. The technique works on the premise that 'neighboring pixels in space-time that have similar intensities should have similar colors,' and also allows colorization of videos by 'marking' about one in ten frames."
This discussion has been archived. No new comments can be posted.

Colorizing Images and Video by Scribbling More

Colorizing Images and Video by Scribbling

Comments Filter:

  • Help! (Score:5, Funny)

    by Kimos ( 859729 ) <`moc.liamg' `ta' `todhsals.somik'> on Monday March 14, 2005 @03:03PM (#11935007) Homepage
    I'm colorblind! That whole site is very confusing! :P

  • A play on history (Score:4, Interesting)

    by Anonymous Coward on Monday March 14, 2005 @03:05PM (#11935028)
    Back when voyager(s) were flying by planets I recall reading how the cameras worked. From what I remember, the cameras actually capture images in black and white. The cameras can detect much more "color" depth than color cameras could (or can?). The scientist would process the pictures to colorize them, you identify one area of color you know and the algorithm would process the rest of the 1 billion shades of gray into a color mapping for people to view. Now why cant identify this gray shade as the color red; anytime you see it then that is red. Go on for each color spectrum or have the algorithm adjust what a little red hue is for a given little hue of gray. It appears that is what the scribbles are doing which is quite clever and the algorithm doesn't have to work (guess) so much.

    • Re:A play on history (Score:4, Informative)

      by srmalloy ( 263556 ) on Monday March 14, 2005 @03:44PM (#11935511) Homepage
      The problem with doing this is that, for any given camera, there will be a band of RGB color combinations that produce the same luminosity, so a single camera does not provide enough information to produce a full-color image. It requires several cameras, each filtered to a different spectral range, to be able to produce a full-color image, unless you know in advance that your image is monochrome.

  • Photoshop (Score:3, Insightful)

    by FoXDie ( 853291 ) on Monday March 14, 2005 @03:05PM (#11935039) Homepage
    Personally I can't wait until there is a Photoshop filter for this. :D

    • Re:Photoshop (Score:5, Funny)

      by Anonymous Coward on Monday March 14, 2005 @03:14PM (#11935174)
      > Personally I can't wait until there is a Photoshop filter for this. :D
      You have misspelled Gimp..

    • Re:Photoshop (Score:5, Interesting)

      by pavon ( 30274 ) on Monday March 14, 2005 @03:20PM (#11935242)
      No kidding. Even if you are not doing colorization, the boundry detection algorithm he is using kicks ass over the "magic wand" tools in both photoshop and gimp. Perhaps it is the fact that it is doing several "magic wands" at once and boundries are determined by what matches the best, rather than just "does this match good enough".

      • Re:Photoshop (Score:2, Interesting)

        by MankyD ( 567984 )
        Im not sure about that. I'm taking a wild guess, but I believe they simply start propogating out from all scribbles, so that, as the growth of two colors approach each other, the stronger one wins out. The wand tool in photoshop starts only from the point you click (rather than the 10+ scribbles in this algorithm). There's no competing areas of propagation.

        Create a magic wand tool that requires multiple clicks on the various regions of the image and you'd have pretty good results.
        • I _like_ that idea: have serveral magic wands active at the same time. To inhibit one wand, start another, in the region where you don't want the first to go.
      • This is an image processing technique known as segmentation [ttp], which is an active area of research. Combined with "texture classification", and you can easily break up a scene into regions of distinct visual appearance.
    • My first thought was "Gimp Plugin!", but a Photoshop plugin would be good too.

  • Seems simple but... (Score:5, Interesting)

    by suso ( 153703 ) on Monday March 14, 2005 @03:06PM (#11935051) Journal
    from looking at the before and after images, this technique looks pretty cool and will probably have applications for recoloring an image that is already color. For instance, the image where he recolors the fabric on the chair.

    • Re:Seems simple but... (Score:2, Interesting)

      by tehshen ( 794722 )
      I agree, this does look cool, but there are already colorization tools and Photoshop filters out there - how does this stack up against them? The article says these are "tedious, time-consuming, and expensive tasks", but does not mention the speed of their own scribble method. Using older colorization methods may be preferable if time is an issue.
    • I'm painting the upstairs of our house and can't get inspired for anything.

      Being younger we went with "different" color schemes in our house, orange, grey, shit brown (all really fits too) but the upstairs is still *blah* and I can't imagine any color there than what already is. I've got paint chips taped up here and there and something like this would be nice to have at home (far off I know).

      Taking a picture then scribbling on it would be a nice way of previewing paints. In fact I suggest here (in the op

  • Awesome! (Score:3, Insightful)

    by JesusCigarettes ( 838611 ) on Monday March 14, 2005 @03:06PM (#11935062) Homepage
    Now it's even easier for corporations re-releasing films to completely destroy the original beauty of a film by adding unnatural and unnecessary color!

    Coming soon, new dubbing techniques will allow easy substitution of the original actors' voices and dialogue with trite teen-angst to appeal to younger generations.
    • The obvious extension of this is to run software which can identify objects in film and track their motion. Then you apply colour to the 3D digital copy of the item, implant it back into the film and have the original shading information from the B&W film cast on top of the coloured, reimplanted, object.

    • Re:Awesome! (Score:3, Insightful)

      by mzwaterski ( 802371 )
      Then don't watch the re-released version. I mean come on, of all things to complain about. "Some company has the ability to change something I like into a form that other people will like."
      • This doesn't stand up though..as colorized films have failed big time. They were a fad for a while, but it just petered out.

        But I have a big problem with colorizing films anyway. Just because you CAN do something doesn't mean it should be done. The weak weak argument that "the kids won't watch it" fell on it's ass as the kids didn't watch the colorized crap either! Not to mention that at the time when colorizing was huge, music videos were the rage...and about half of them were in B&W!

        Also, why don't

    • Re:Awesome! (Score:5, Funny)

      by worst_name_ever ( 633374 ) on Monday March 14, 2005 @03:19PM (#11935223)
      Coming soon, new dubbing techniques will allow easy substitution of the original actors' voices and dialogue with trite teen-angst

      Are you listening, Mr. Lucas?

    • Re:Awesome! (Score:3, Interesting)

      by tgd ( 2822 )
      You mean like dubbing Mel Gibson in Mad Max?

      It happens already.
    • Now it's even easier for corporations re-releasing films to completely destroy the original beauty of a film by adding unnatural and unnecessary color!

      You mean as opposed to Technicolor? [wikipedia.org]

    • "Now it's even easier for corporations re-releasing films to completely destroy the original beauty of a film by adding unnatural and unnecessary color! Coming soon, new dubbing techniques will allow easy substitution of the original actors' voices and dialogue with trite teen-angst to appeal to younger generations."

      Oh come off it you load of flamebait. Nobody is threatening your precious black and white films. If people want to watch the originals, they will. If they want to watch a new colorized vers

    • Let us just hope someone will apply this to clerks and release it in color!

  • Unfuckingbelievable. (Score:2, Insightful)

    by syukton ( 256348 ) *
    Their site is going to get slashdotted real quick. Lots of content on there. It's beautiful. It's ... it's amazing what they've done with so little effort.

    I am so curious what this could do for so many old movies...

    • Re:Unfuckingbelievable. (Score:4, Interesting)

      by nacturation ( 646836 ) <nacturation AT gmail DOT com> on Monday March 14, 2005 @03:16PM (#11935193) Journal
      I am so curious what this could do for so many old movies...

      Ruin them? :) A lot of the appeal of older B&W movies is the fact that they aren't in color. You get a much broader range of contrast when it's filmed on B&W film than a color image which has been desaturated.

      If you meant older color movies which have degraded, then I agree. This seems like a very useful technique for restoring the original vibrancy of colors to films whose media hasn't stood the test of time.
      • I don't necessarily believe that old black and white movies are good BECAUSE they are black and white. Granted, a lot of "colorized" movies look like crap, and I'll also grant you that a lot of black and white movies are good. I think that the correlation between the two is probably imagined. Colorizing a good movie doesn't necessarily lessen the movie, and can add considerably to it I'd say. The act of adding color (if done well), by itself, is not going to ruin the movie, in my opinion. Adding color,
        • Colorizing a black and white movie is adding something that the director did not put there. And generally, he's dead, so you can't consult him about it.
          • The director didn't add colour because he probably couldn't! The question is would he have wanted to see the movie in colour if available?
            • Well, to be fair it's very possible the director might have made different cinematographic choices if he'd had color to work with.

              On the other hand, in cases where the director is DEAD... chances are he either doesn't care anymore or has bigger issues to worry about.

              From a historical perspective I do think it's a shame if the original versions are lost or suppressed, but otherwise the result of colorization or anything else deserves to be judged on its own merits. Not whether it might offend dead people.
        • they might not be good because they're bw, BUT: coloring them will do zero to improve the quality.

          and they don't do it properly ever. if you're goin gto do that you might as well make a full remake of it(when thinking in hollywood terms).
      • I meant black and white movies. I'm not saying we should colorize them and then burn the originals though, which is how you appear to be interpreting it. I'm saying that providing the option of watching an old movie in color may revitalize many old movies and television shows, allowing them to be rehashed and rerun for more corporate profit.

        You don't think hollywood would want to re-release older movies or television shows to a younger generation to milk the franchise one more time? The content has already
        • You don't think hollywood would want to re-release older movies or television shows to a younger generation to milk the franchise one more time? The content has already been made, it just needs to be brought up to speed to be appealing to the youth of today.

          I thought the usual formula of bringing old movies up to speed and *trying* to appeal to the youth of today was to add Ben Affleck and re-shoot. Of course, shoot Ben Affleck and re-add might be preferable to most.
        • I meant black and white movies. I'm not saying we should colorize them and then burn the originals though, which is how you appear to be interpreting it. I'm saying that providing the option of watching an old movie in color may revitalize many old movies and television shows, allowing them to be rehashed and rerun for more corporate profit.

          I wouldn't care to watch a classic old movie with added color. Doubtless, in many cases the director would probably have used color if he could have, but the movie he

  • Mirror of the site, with images (Score:3, Informative)

    by hankwang ( 413283 ) * on Monday March 14, 2005 @03:07PM (#11935071) Homepage
    This page is over 10 MB, so it's electable as the fasted slashdotted site ever.

    Here is the coralized mirror [nyud.net].

  • I sure hope they can patent this... (Score:5, Funny)

    by Deep Fried Geekboy ( 807607 ) on Monday March 14, 2005 @03:08PM (#11935080)
    ...and raise enought money to rebuild the smoking ruins of their server room.

  • That's nothing... (Score:3, Funny)

    by TheBrakShow ( 858570 ) on Monday March 14, 2005 @03:09PM (#11935099)
    The folks at slashdot can take down a webserver just by making a few scribbles on their website.
  • Looks like a very cool idea. The videos are pretty amazing, however; the recolor job in the birthday party clip makes it look like a bad acid trip.

  • Been there, Read that (Score:3, Interesting)

    by tonsofpcs ( 687961 ) <slashback@tonsofpc s . com> on Monday March 14, 2005 @03:16PM (#11935195) Homepage Journal
    Read about it earlier this week, it looks cool, colorization looks nice, but with so few samples its hard to tell if it will work with a wide range of inputs, or only ones with contrast ratios like those shown.
  • This work is very similar to some work that was presented at last years siggraph using graph cut optimization titled Interactive Digital Photomontage [washington.edu] by some researchers at the University of Washington. This stuff is really cool and has applications outside of just re-coloring black and white. For example, compositors in the film industry adjust the color composition of scenes that were filmed during the day to look like they were filmed at night. Sometimes they just need to tweak the color because the art director isnt happy with it. Other times it's because they introduced CG elements into live action scenes and they dont quite match. If they can tweak those colors interactively, without authoring masks, it is faster than re-rendering the scene and that saves money.

    Very cool stuff.

    Pete
    • This stuff is really cool and has applications outside of just re-coloring black and white.

      Industry applications are interesting, but nothing new -- the industry has been using this technology for a long time when it was more labor-intensive, because they can afford to.

      The REAL impact of this technology will come when you see it migrate into new versions of iPhoto and Photoshop Elements. In Photoshop, recoloring a part of a photo is relatively easy, but it still involves a mildly complicated process of

  • Not a new concept (Score:2, Interesting)

    by UMhydrogen ( 761047 )
    I'm not sure if this was intended to sound like some new ground breaking technique, but it really isn't. I am a masters Electrical Engineering student and am currently taking an Image Processing class. Using neighboring images to reconstruct an image is a VERY VERY common task - in fact, it's almost the only way to do it. How else are you supposed to guess the colors (or what pixel is *supposed* to be there) without knowing what's around it. It's obvious that the highest correlation will be between the near

    • Re:Not a new concept (Score:3, Insightful)

      by James McP ( 3700 )
      I think the "break through" in this process is that it works over a series of frames automatically rather than requiring each frame to be manipulated. It was my uneducated understanding that colorization tended to be a frame-by-frame process.

      If this can cut the work down to 1/10th normal it becomes plausible for the general public. While I'm no budding spielburg, I know a lot of people who might want to touch up the color quality of their wedding video.

  • application to motion video (Score:5, Interesting)

    by mzs ( 595629 ) on Monday March 14, 2005 @03:29PM (#11935337)
    The site is slashdotted so I cannot read it, but i wonder if something akin to this could be used for compressing motion video. For example the intensity is encoded with currrent techniques, but instead of the color being encoded at a lower resolution, instead only a very small amount of colored points are encoded. Then during the decoding, the decoder uses an error function, intensity, and the time domain of previous and future frames to 'fill' the colors out.
    • It could theoretically reduce the bandwidth, except that the CPU cost to apply the data and reconstruct the image would be prohibitive with anything we have now-a-days.

      They are NOT applying these scribbles in real-time - they scribble, the computer **grinds**, and quite some time later an image appears.

      With the kind of CPU power it would take to apply this sort of data in real time, you could probably get much greater compression using a very exhaustive wavelet search.

  • Take a grayscale image, and you already have the brightness and value for each pixel. All you do is add the hue component based on the color "scribbled" by the user. Stop filling with that color when you hit something that marks a definitive boundary.
    • Its not that simple. The problem is how do you define a "definite boundary"? And even if you do what you suggested, you are likely to get rather flat shaded images. What these folks do is really solve an optimization problem and try to make use of the fact that neighboring pixels of similar intensity would have similar color. It is kind of like a fancy flood-fill algorithm, but applied to a new area, and thats what makes this novel!
    • Stop filling with that color when you hit something that marks a definitive boundary.

      That's the "and then a miracle occurs" step.

  • Tech Geeks vs. Film Purists (Score:3, Insightful)

    by venomkid ( 624425 ) on Monday March 14, 2005 @03:32PM (#11935374)
    You b/w film purists. If all you can see is a threat to your bizarre, luddite idea of what film should be, you need to get your heads checked, or at least you need to listen to your inner geek. Stop using these folks' achievement as an opportunity for chest-thumping.

    The idea that one could color correct video with a few strokes from mspaint is staggering. Imagine if one could do this to color video, in real time... you could color-highlight an object and the computer could follow it without sensors or other pre-implanted devices, and that's not even a particularly original idea. This is awesome technology with applications probably well beyond what we see here.

    • Why is this marked +5 Insightful?

      You b/w film purists. If all you can see is a threat to your bizarre, luddite idea of what film should be, you need to get your heads checked, or at least you need to listen to your inner geek.

      I see NOTHING in the original post that mentions black & white films. It might be implied, but even then I don't see any mention about how black & white is better than color, or how colorizing an old film is bastardizing that work. Some people like to see films in their ori

      • In the first paragraph, I was referring to the multitude of irrelevant, huffy "colorizing b/w movies is a sin!" posts.

        But hey, since you missed the point of that, it doesn't bother me that you missed the overall point of my post, which was in the second paragraph.

        Cheers.

  • If you load up page [nyud.net] and watch the very last video, can see a slight artifact in the reflection of the hanging stuffed animal. There's little spots of orange color, like you would get out of the AirCan tool in MSPaint.

    I have a feeling this could be corrected with another scribble or two. Really a stunning piece of work. Very cool.
  • I dabble more in the area of 3d art, but at times I've lifted a pencil and come up with some decent b&w sketches. Pencil shading is easy but sometimes getting the colours just right is more difficult than you might think.

    When I looked at a lot of B&W webcomics, I can see that they'd look better in colour (especially the ones where the artist does occasionally do vibrant colour cells, but usually don't have time). This could change that though... want to see what your character would look like with

  • Colorizing examples. (Score:5, Interesting)

    by ChrisUK ( 92178 ) on Monday March 14, 2005 @03:45PM (#11935513) Homepage
    I tried out their matlab code and put a few example colourings on my web page, for the interested:

    http://www.inference.phy.cam.ac.uk/cjb/ [cam.ac.uk]
    • Very cool. For what it's worth, and you might already realize this, I believe you can scribble white on areas you don't want re-colored. So it'd be easier to change just your shirt color and the birds in the 2nd photo.

  • Realtime (Score:4, Insightful)

    by phorm ( 591458 ) on Monday March 14, 2005 @03:45PM (#11935522) Journal
    I could see this working best as a "realtime" colour filter, especially if you're using a pen or something similar. Scratch near a border and view the result... if it goes a bit beyond where you want scratch on the other side of the border. If it's not quite enough lengthen your scratch.

    I wonder how much CPU power is required, if you could do this realtime or close to it would be quite awesome, but having to make your scratches and click "apply filter" then wait for 30 seconds would not be nearly as useful/efficient.
  • When can I download the Photoshop plugin?
  • For example, on Farscape, given Virginia Hey's problems with makeup and contact lenses... heck, any of these humanoids-with-funny-skin-color shows would benefit from not having to put in the hours upon hours of makeup. Instead, we'd see hours upon hours of post-production...

  • Blurs? Fine background detail? Motion? (Score:3, Insightful)

    by dpbsmith ( 263124 ) on Monday March 14, 2005 @03:54PM (#11935668) Homepage
    In the days when colorized videos of black-and-white films were common, I watched a few. The so-called "colorization" had some very serious problems, and I wonder whether this new method addresses them.

    The problems tended to be in the background, and they probably thought people's attention would stay on the foreground, but I think like many things in film you notice them subconsciously. Either the background is out of focus, in which case there are no sharp edges for the colorization to work on, or it contains a basically infinite quantity of detail as the background gets farther and farther way. Either way, it was extremely common to see uncolored areas in the background.

    It was fairly common to see black-and-white paintings hanging on walls, for example. The walls would be some fairly uniform wash of plausible wall color, but nobody was going to take the time to handcolor the paintings hanging on them.

    A similar problem concerned scenes with machinery in them, or anything with lots of complex, detailed motion (so that successive frames weren't similar). Thus, you'd see black-and-white printing presses operating in a colorized newspaper building...

    In addition, the fact that the colorized faces, for example, were a uniformly colored wash, rather than varying in color as well as brightness, created a subtle kind of phoniness. To me, the result was the conveyance of a sort of emotional coldness. The colorized movies looked colored, but they didn't feel colored.

    The exact opposite of the kind of lift you couldn't help feeling in the fifties when you saw a Technicolor spectacular--in the days when "Technicolor" meant that by golly you were watching genuine dye-imbibation prints from real color separations. Sweet as candy, but irresistable. (The effect does come through in the best DVD restorations).
  • Is there any OSS applications out there that are similar? This would be a really cool Sourceforge project.

    Also, does anybody happen to know if a similar technique was used for the new Tom Cruise movie coming out? You know, the one that kinda looks like Waking Life in style?

  • could come in handy for re-drawing maps in the area. Save new layer, don't finalize changes.

  • nature (Score:5, Interesting)

    by Jodka ( 520060 ) on Monday March 14, 2005 @04:45PM (#11936290)

    " The technique works on the premise that 'neighboring pixels in space-time that have similar intensities should have similar colors"

    Interestingly, the retina exploits that same property of natural scenes to compress images. This correlation between luminance and color is an opportunity to throw out redundant information. The eye multiplexes color and luminance information over a single channel, transmitting luminance while discarding color at high spatial frequencies and transmitting color while discarding luminance at low spacial frequencies. First reported by C.R. Ingling, color/luminance multiplexing is an inherent property of the linear color-opponent center-surround receptive field. For a good explication of the subject, see:

    Vision Res. 1985;25(1):33-8.
    "The spatiotemporal properties of the r-g X-cell channel."
    Ingling CR Jr, Martinez-Uriegas E.

    Abstract: Analysis of the simple-opponent r-g receptive field of the X-channel shows that it is tuned to both high and low temporal frequencies, high and low spatial frequencies, and that its spectral sensitivity is both chromatic and achromatic.
  • that is quite possibly the coolest thing i've seen in 2005

  • Applications for Art (Score:3, Interesting)

    by leoboiko ( 462141 ) <leoboikoNO@SPAMgmail.com> on Monday March 14, 2005 @06:01PM (#11937220) Homepage
    Some folks seem to be excited (or angry) with the possibility of coloring B&W movies with this technique. Forget realistic coloring, this looks amazing for artistic recoloring.

    Go take a look at the "recoloring examples" in the coral cache. Also look at what a slashdotter did [slashdot.org] with the code. Photographers, designers and painters could do neat things with a filter like this in Gimp...
  • The first example picture is a bit strange. In the result picture the boy has green eyes, but there are no green markings on the eyes on the BW picture?
  • Funny thing is i was arguing with a photoshop wonk about this just last week. It seems that when it first surfaced, it was 'debunked' by end-users on worth1000.com.

    Levin et al were published by ACM more than a year ago.

    Some criticism: Their website only shows images that show the strength of their method. They don't show any examples of it's weaknesses, and it does have some.

    A much more interesting website would be this one:

    http://mountains.ece.umn.edu/~liron/colorization/ [umn.edu]

    A similar technique, a compa
  • Unfortunately a probably life cycle of this kind of tech will be:

    1) use in a carefully restored, prestigious film
    3) good reviews and ratings/audience
    2) creation of a mac based product
    4) profit

    but... oh yes

    5) a slew of third rate knock offs that use the effect to justify a 5th rate plot

  • Dammit (Score:3, Interesting)

    by Andy Dodd ( 701 ) <atd7NO@SPAMcornell.edu> on Monday March 14, 2005 @06:54PM (#11937876) Homepage
    Apparently, one of the critical differences between Octave and Matlab is that Octave apparently doesn't support multidimensional matrices/arrays with dimensions greater than 2.

    Which means that the current code is completely incompatible with Octave, as it depends on Matlab's implementation of imread() which returns image data as a three-dimensional matrix.

    Going to see if I can get it to work easily, but there's a good chance I won't be able to. :(

Slashdot Top Deals

Say "twenty-three-skiddoo" to logout.

Close