×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Image Processing By Example

timothy posted more than 12 years ago | from the draw-this-way dept.

Graphics 127

Aaron Hertzmann writes: "My collaborators and I will present a paper called Image Analogies at SIGGRAPH 2001 this summer, where we describe a machine learning method for 'learning' image filters for example. For example, given a Van Gogh painting, the algorithm can process other images to look somewhat as if they were painted by Van Gogh."

"It can also 'texturize' images based on a sample textured image, e.g. to create landscape photos. It can do many other types of filters, as long as you give appropriate 'before' and 'after' examples to learn from." I especially like the idea of inferring a high-resolution image from a low-res one. The software is available for Windows and Unix, and "the source code is freely distributed for educational, research and non-profit purposes."

cancel ×
This is a preview of your comment

No Comment Title Entered

Anonymous Coward 1 minute ago

No Comment Entered

127 comments

wow (1)

Anonymous Coward | more than 12 years ago | (#127260)

i just noticed that resizing link. Thats awesome - i cant think how many movies i've seen where they did the same thing and i was thinking "yeah right". I cant wait till someone makes filter that loads all the websites from coolhomepages.com and spits out composition to my empty canvas in photoshop ...

Since when did Slashdot become... (1)

Anonymous Coward | more than 12 years ago | (#127261)

... a vehicle for people to toot their own horns and advertise their OWN papers, etc?

A VanGogh Monet Original (1)

Falrick (528) | more than 12 years ago | (#127262)

I just wonder what it would look like if you passed Monet's Water Lillies through the VanGogh filter. An approximation of an impression of an impressionist's painting. Figure out the logistics of that one.

Re:Make a bad painting from a good photograph (1)

ananke (8417) | more than 12 years ago | (#127270)

i wouldn't even call it "painting". pc does nothing but manipulate the existing image. doh.

Using a computer now more like fine art? (2)

mattkime (8466) | more than 12 years ago | (#127271)

At first, I was going to say, "Well, I guess this is further proof that using computers doesn't count as fine art." (Only joking, I'm a BFA student and use computers plenty.)

Then I looked at the images this program produces. I guess I can't expect anything BETTER than what is stuck into it - Van Gogh would never have painted scenes like those. Hell, any beginning art student could "paint like Van Gogh" - just mix your paints a bit thick! Perhaps the program needs a content selection algorythm. (I'd like to see it do a self portrait.)

At any rate, the "holy grail" of this technology is to emulate air brushed velvet. Not a fan of elvis but love the look of painted velvet? Just use a pic of your favorite celeb! Finally, Natalie Portman recognised as she should be!

Somewhat like Van Gogh? (3)

geophile (16995) | more than 12 years ago | (#127273)

So, you're going to find all the pr0n and remove the ears? Whatever floats your boat.

Re:Text vs. Image (1)

dillon_rinker (17944) | more than 12 years ago | (#127274)

the real advantage both masters have is not in their actual prose but in the ideas they express - and no filter is going to be able to duplicate that.

Au contraire - see sci-fi book a minute at rinkworks.com. :)

Re:Hi-rez from Lo-rez (1)

cale (18062) | more than 12 years ago | (#127276)

I think the point is that you train it in how to make up resolution in a certain situation with a certain type of picture. Could be useful in some situations, not that I can really think of any, but if I could well, then I'd have tons of karma and stuff.

Re:Just make sure your training data is good... (2)

BWJones (18351) | more than 12 years ago | (#127277)

The problem of finding "tanks" or whatever is actually an old one that goes back to the late sixties and seventies when the CIA was planning on sending up real reconaissance satellites with multispectral capabilities and NASA was sending up their interplanetary probes. The math used for this problem is older than that, but some new methods are being implemented by both government agencies and private corporations. It used to be much easier when we were just taking visible light photos. For instance, the wife of a friend of mine used to specialize in examining runway lengths. All she would do all day is look at photos of runways and compare their lengths to previous images to determine if they were being upgraded for bigger/more powerful planes. Now you can imagine that given the same image of a runway, but now you have twenty to one hundred and twenty separate images at different spectral wavelengths why there is a need for automation. Unfortunately all of the methods to analyze this type of data require fairly significant amounts of computational time and analyst time to extract the information you are looking for. There are many ways of going about looking for these "tanks" or whatever you want to look at in images (SCUD missiles, concrete desguised to look like granite, gold bearing strata, oil bearing strata, alpha ganglion cells in the retina, etc etc etc....) Using traditional multispectral classification techniques, one does not neccessarily have to implement neural networks. (There are times they can be problematic, but this is probably due in many ways to immature code) You can simply use supervised classification techniques where you specify the spectral fingerprint (you have to know what you are looking for) and have the algorhithm extract or highlight the pixels it finds. Alternatively, if you do not know exactly the spectral fingerprint of what you are looking for, you can perform an unsupervised classification such as a k-means or ISODATA classification. These techniques can be fairly time intensive from both a computational and an analysts perspective thus the push for more automated methods. It is the automation that has proven to be difficult. Many things like co-registration of hyperdimensional images is easily implemented using embedded fiducial points, but the actual analysis and taking apart of hyperspectral data often is more of an art than science (I hate to say that as thats what people say when they don't understand all of the aspects of the problem but....)and as such it currently requires lots of "biological" as opposed to silicon supervision.

matching training data to subject matter (2)

cindy (19345) | more than 12 years ago | (#127279)

It looks like the training data needs to be similar to the target data if the results are to be visually similar. In the watercolor example the training data of the apples produces the best results on the photo of the tulips. The filtered landscape photos don't look anywhere near as good. I wonder if you could use several sets of training data (still life, landscape, portrait, etc.) to create a more general purpose filter.

I would think that approximating the unfiltered source part of the training data from a painting (like the Van Gogh example) would produce kind of twisted results from photographic data. I wonder if they've tried getting a "real" painter to mimic Van Gogh's style from a photograph and using both the photo and the painting for training data. I expect the results would be better. (Although that wasn't really the case with the pastel examples so maybe not.)

Maybe I'll get a chance ask at the show.

I think you all have it wrong... (3)

Darth Maul (19860) | more than 12 years ago | (#127280)

I'm seeing a lot of posts just saying "Big deal, it's a Photoshop filter". But that's not the point. The point of this is that given the source and the final image, the algorithm learned to reproduce the filter. In otherwords, it dynamically generated the "Photoshop filter" based on the input and output of the sample set. That's pretty cool!


It's not just using the filter; it is creating the filter.

There's more then the Van Gogh stuff (2)

delmoi (26744) | more than 12 years ago | (#127281)

Take a look at some of the other images. And they didn't just "go into xv and sharpen the image" they got the computer to figure out how to do it on its own, given only two inputs!

I doubt you could program something similar if your life depended on it, looser.

That is what they did (2)

delmoi (26744) | more than 12 years ago | (#127282)

That is what they did, although in the case of the paintings, they had the 'after' shot to start with and simply blured it to get the 'before' shot. In other words, there was no 'orgional' filter.

Re:Our Wow threshhold (3)

delmoi (26744) | more than 12 years ago | (#127283)

. In the case of Van Gogh, you just need to fuzz up the image a little. But there's a lot more to Van Gogh than fuzziness!

Wow, THEY DID Did you even read the site? or just look at the example? Here's one of some guy named Lucian Freud [nyu.edu]

And keep in mind they didn't 'just tell the computer to 'fuzz up' the image' they just gave the computer a copy of Starry night, and a blurry copy of Starry night and said 'figure out how to go from the blurry one to the original'. After that, the computer did all the work

Author identification (3)

harmonica (29841) | more than 12 years ago | (#127284)

You can use PPM compression for this.

Say you have samples of the works of N authors and a text T that has to be identified. Compress the text N times, each time the system is initialized with the samples from another author. T will usually compress best when the system was initialized with the samples from its own author.

See Bill Teahan's PhD thesis [rgu.ac.uk].

Re:That's too bloody cool (2)

Bob Uhl (30977) | more than 12 years ago | (#127285)

Well, while filters have existed in PhotoShop or the GIMP for ages, they tend to do a not altogether excellent job. Did you look at the one where they took an image and applied a watercolour filter? The result looked nothing like a watercolour, but had encoded in it what some folks might think a watercolour produced from an image might be. They then fed other photographs into the engine and produced what look to my untrained eye like real honest-to-goodness watercolours. I think that's pretty neat: they created good stuff by mimicing bad filters.

Re:Make a bad painting from a good photograph (2)

Bob Uhl (30977) | more than 12 years ago | (#127286)

The point is, this is an ecellent first step on the way to making software which can paint. It may not be here today, but we are progressing twoards that end. What I find amusing is that this software has no concept of aesthetics or beauty; it is by no means a thinking machine. And yet it creates what we find to be aesthetically attractive pieces. Indicates to me that perh. artists are not more deeply connected to the soul or heart than the rest of us after all.

Re:Close, but no severed ear... (1)

KillNateD (31007) | more than 12 years ago | (#127287)

Van Gogh was not an expressionist. He was a post-impressionist.

I only took the trouble to correct your mistake because I disagree with your fairly backwards view of art...this idea of the artwork as a vessel for "the true meaning of the artist" is really trite.

Another pedantic slashdot reply. I am an ass.

Re:Wow (1)

AntiFreeze (31247) | more than 12 years ago | (#127288)

Norman, I truly did not write that in the spirit of a troll. The word/pixel thing was something I pulled out of my ass to try and make my point, I do realize how stupid that actually is, I just thought it would help some people to visualize what I meant.

---

Wow (5)

AntiFreeze (31247) | more than 12 years ago | (#127289)

This is actually quite impressive software.

What I've started to wonder is where else it's underlying principles could be used, or where this sort of technology could lead in the future.

Could it be used to analyze text from certain authors (hey, text and art are no different to a computer - treat words as "pixels" and sentences and structures flow like colors) and mimic their style? Could this one day be used to turn my dull crud into something Fitzgerald or Hemingway or even Asimov or Heinlein might have written?

I also have the following few questions:

  • What happens when one feeds a Van Gogh through the Van Gogh filter? Does the resultant image change much?
  • Does the program apply the "filter" differently depending on what type of input it encounters, or is the same method applied to all input?
  • Conversely, can the program be used to recognize when a work is of a certain artist?
  • Or can it be used to see if an image has already been passed through a certain filter?
  • Are there cases which cause the method to fail or create an undecipherable image? And if so, are these cases unique or do they conincide with a certain type of artistic style? [e.g. Monet -> Van Gogh just won't work right?]

I think that sums up my feelings. This stuff is really impressive guys, I hope the conference goes well.

---

RAH (1)

Unknown Poltroon (31628) | more than 12 years ago | (#127290)

As soon as you get a computer to do the work and create the thoughts that Heinlein put out, I think its time for the human race to quietly disappear into the backround to be reused for carbon.

Re:Wow (1)

SpinyNorman (33776) | more than 12 years ago | (#127293)

Good grief. :-( I thought there for a second that the moderators were showing slightly better than usual troll detection, then I came back and this has been moderated up to +5!

Clue: The word/pixel gibberish. Geez.

Could it be used to analyze text from certain authors (hey, text and art are no different to a computer - treat words as "pixels" and sentences and structures flow like colors) and mimic their style? Could this one day be used to turn my dull crud into something Fitzgerald or Hemingway or even Asimov or Heinlein might have written?

No. A local filter is a local filter. It is not a painter, nor an author.

If you put your dull crud through a filter it'll turn it into a *SHOCK* filtered version of the same dull crud. Try the jive filter if that sort of thing impresses you.

What happens when one feeds a Van Gogh through the Van Gogh filter? Does the resultant image change much?

Yah, same as passing a finger painting thru it - you'll get a version with the same local distortions added. It's a filter, get it?

Does the program apply the "filter" differently depending on what type of input it encounters, or is the same method applied to all input?

Does it say it's data adaptive? No.

Conversely, can the program be used to recognize when a work is of a certain artist?

No, neither can it make toasted sandwiches or draw mandlebrot sets. It's a filter.

Or can it be used to see if an image has already been passed through a certain filter?

No, it's a filter.

Are there cases which cause the method to fail or create an undecipherable image?

No, it's a filter.

And if so, are these cases unique or do they conincide with a certain type of artistic style? [e.g. Monet -> Van Gogh just won't work right?]

If you put a Monet painting thru the filter it'll look no more like a Van Gogh than if you put anything else thru - it'll look like a filtered Monet.

Re:I think you all have it wrong... (2)

SpinyNorman (33776) | more than 12 years ago | (#127294)

So what? It's just optimizing the filter paramters to reduce the difference between the source and target. The only moderately interesting work they did was the manual step of choosing the parameterized filters. The program itself is just a best fit algorithm, and the fact that some impressionist styles can be approximated by local distortions that approximate brushstroke styles doesn't turn a best fit algorithm into Van Gogh.

Re:Since when did Slashdot become... (2)

prizog (42097) | more than 12 years ago | (#127295)

Since when did Slashdot become... ... a vehicle for people to toot their own horns and advertise their OWN papers, etc?

Yeah! A story is so much more interesting when it languishes unknown in the vastness of the web for six months, only to be discovered by the time it has officially become Old News!

Bring back the good old days of virus reports from even mistier pasts.

Re:Maybe those movies weren't wrong... (1)

keete (42938) | more than 12 years ago | (#127297)

Ever see "the Sentinel"...
Computer nerd, my ass. What about the ubermensch whose senses were sharpened in the Brazilian rainforest to the point where he can clearly see the limited edition Willie Mays watch visible for a split second on the two-pixel wrist of a bank robber caught on grainy security camera?
--

Re:Just make sure your training data is good... (1)

keete (42938) | more than 12 years ago | (#127298)

I seem to remember a PBS special (Nova?) that had a segment on experiments done with image recognition in pigeons. These pigeons were placed in a large, darkened box that displayed a projected image on one wall. They were then prompted to respond to the image in different ways and were given food feedback if they were "correct" (pecked when the right image was displayed). The pigeons were first introduced to one image and then shown related images as well as unrelated images to see if they could generalize... it emerged that pigeons can distinguish Monet paintings from Cezanne, though they have some trouble telling Cezanne from Picasso.
--

Re:Wow. Hi-res pics are cool. (1)

apnu (46219) | more than 12 years ago | (#127299)

well, not really, the brush strokes are too predictable and don't always work. take a look at some of Van Gogh art and you will see that he did more than side to side brush work.

they do look very intersting though...

Re:Text vs. Image (1)

vectro (54263) | more than 12 years ago | (#127300)

Except that the book-a-minutes are neither automated filters nor (IMHO) do in fact capture the essence of the book. From what I can see they are generally a poorly defended criticism of the book, or a synopsis of one or two points in the plot.

Text vs. Image (2)

vectro (54263) | more than 12 years ago | (#127302)

Actually, text and art _are_ substantially different to a computer. The former has rules, the latter does not.

If a single pixel is off by a bit, you won't notice - your brain subconsciously blends the whole thing together anyway. If, on the other hand, the wrong word is chosen, it will stick out like a sore thumb - even if it's only a preposition.

Finally, I would posit that even if a filter could make your prose sound like Asimov or Hienlien, the real advantage both masters have is not in their actual prose but in the ideas they express - and no filter is going to be able to duplicate that.

Actually you can to a point (3)

Jafa (75430) | more than 12 years ago | (#127304)

There are a few companies (iterated.com, lizardtech.com, some others) that have been doing fractal and wavelet scaling for a while now. Pretty impressive stuff. I don't know all the theroretical details, just the practical uses. Scaling up to about 1600% is possible with no noticable artifacts (to the human eye). We've been using some of this stuff in the prepress/graphic arts industry for a while.

Jason

Here, let me enhance the image... (2)

Spunk (83964) | more than 12 years ago | (#127305)

I especially like the idea of inferring a high-resolution image from a low-res one.

Wow! Does that mean that a certain staple of movies with spy-cameras crucial to the plot will finally have a basis in reality?

Every time I hear, "I can't see his face because the video's not clear enough - here, let me enhance the image," I just wanna scream...

--

Tulips with ripples? (1)

Florian Weimer (88405) | more than 12 years ago | (#127306)

Did van Gogh really paint tulips with ripples? It seems that the ripple effect used for the real ripples has a strong impact on the derived filter and dominates the result. In addition, the ripples do not follow the natural lines in the image.

I think manual filters are still much better, oops, I wanted to say: nothing can replace a real van Gogh.

The Obvious Question (2)

krmt (91422) | more than 12 years ago | (#127307)

So when's it going to show up in the Gimp? This is some cool stuff. People could make custom filters without programming anything, just train the filter to do what you want. Let the computer do the thinking in terms of imitating the effect you want. Very very impressive.

"I may not have morals, but I have standards."

Van gogh? (1)

Punto (100573) | more than 12 years ago | (#127308)

Van gogh is boring. What about a software that 'looks' at all my pr0n, and categorizes it?
Or maybe a program that looks at pictures of people, and generates pictures of those people having sex.. (like Xena and Scully; Taco will like [slashdot.org] that one)

--

Re:Close, but no severed ear... (3)

BerkeleyBull (101498) | more than 12 years ago | (#127309)

Sometimes, when I look at a Van Gogh, it breaks my heart,it is so beautiful. When I look at these pictures, I get the same nausea-induced feeling that any cheap knock-off imitation gives to me. It's not even interesting technology.

Re:Make a bad painting from a good photograph (1)

sommere (105088) | more than 12 years ago | (#127310)

I'm guessing you didn't read the article... Datamining may very well have the same effect as something a process a human could think of (like the one you explained) but in most cases is more general and that is what makes it usefull. It can make it VanGogh like, maybe you could too, but the same program given a bunch of paintings by, say me, could make it look kinda like I did it... Could you think of a procedure that given some of my art could make it look like mine? probably not, since you don't know what my art looks like (stick figures mainly.) But this could.....

---

Re:What about p0rn images (1)

pingflood (105369) | more than 12 years ago | (#127311)

Can it make a non-sexy image look sexy again?!

Sorry dude, you're destined to remain on fugly.net

Argument Closed? (1)

JojoLinkyBob (110971) | more than 12 years ago | (#127312)

This is really cool, but I don't think it helps those who argue that computer graphics is an art. When you can't tell the difference between human art and computer generated art, is it really an art? or does it pass some Turing-esque test?

What about p0rn images (1)

garoush (111257) | more than 12 years ago | (#127313)

Hmm. I wander how good a job does it do with p0rn images. Can it make a non-sexy image look sexy again?!
---------------
Sig
abbr.

A look and feel stealer (1)

atillathehun (113468) | more than 12 years ago | (#127314)

Maybe you could apply it to a well known computer system and give it the look and feel of another system. If this works call your filter a gate. You could bill for your gates...

Re:Since when did Slashdot become... (1)

snarkh (118018) | more than 12 years ago | (#127315)

Slashdot is certainly an unusual forum to discuss scientific publications.

I wonder if this can be applied to cryptography... (1)

malfunct (120790) | more than 12 years ago | (#127317)

It seems possible to apply this technology as a more general filter. If you take data encoded by a process and the result, make a great number of training sets, then it should be possible to take some result and go "backwards" through the encryption process to get the result. The problem is there isn't nearly as direct of a link between the decrypted and encrypted code as there is between the old visual object and the new visual object.

I guess I'm just curious of how much this is limited to the "visual" world and whether it could be applied to create abstract digital filters.

Re:Close, but no severed ear... (2)

garagekubrick (121058) | more than 12 years ago | (#127318)

Admittedly, yes, Van Gogh was a post impressionist, but expressionism derives directly from himself and Gaugin to a large degree (and perhaps even Japanese watercolor).

As for my interpretation of "the true meaning of the artist" - whether or not I agree with the concept of all art needing some intangible, metaphysical, or even spiritual explanation that is derived from the ego of the artist (which I don't necessarily) - Van Gogh has become a cause celebre for the artist whose life was as important as their work. It's part of the reason for the ridiculous prices associated with his work. His letters to his brother established him as a figure after his death, and a cursory read would reveal that there were specific personal and deterministic notions in his endeavours. It may be trite to yourself, but it drives the ridiculous art market today, and many would say that it began with Van Gogh. Van Gogh's work is part and parcel with his life, interminably, for better or for worse - it is impossible to seperate him from his time and its technology and image processing capabilities.

And I do stand corrected on what's different about this work and what it's for (imagine using texture by numbers to generate landscapes for computer games rather than storing the textures). I will stand by the fact that when it comes to mimicking an artist's strokes the material will only ever be as good as a) what's fed into it and b) a pale imitation, good for only some FX gag I expect to see in some music video and then vanish.

Close, but no severed ear... (5)

garagekubrick (121058) | more than 12 years ago | (#127319)

This is interesting and all well and good, but ultimately where it fails is that the produced image is entirely dependent on the original photograph's perception of the world. A reproduction of an image through halide crystal activation, which is enough for human memory and recognizance, but it lacks the true meaning of the artist. Van Gogh never used contrast or flat lighting as exhibited in the source pictures, and he often burst highlights with striking colors that may not have been actually present to his eye. It's what seperates him from a Turner - not just his brush stroke or how thick he worked in paint but how he saw the world. It's pretty churlish to adopt the first real expressionist painter (who deliberately attempted to paint their perception of the world rather than reproduce it) as an example of this algorithim, as the resultant images show that without an interpretation or perception this is pretty useless stuff. All I see here is a souped up photoshop filter.

Linus Torvalds Code Filters (2)

Jagasian (129329) | more than 12 years ago | (#127320)

Van Gogh filters are cool and all, but in this geeky day & age, I would be more impressed with a code filter that could be trained to make my C programs look like they were written by Linus Torvalds... you know, change the indention styles, white space usage styles, identifier naming styles, and how bout going as far to use the same algorithmic patterns too?

The question is, what would happen if you fed your average Visual Basic program written by your average VB coder, through a Linus Torvalds filter? Wouldn't that be like "crossing the beams" in Ghost Busters?

Wow. Hi-res pics are cool. (2)

Mustang Matt (133426) | more than 12 years ago | (#127323)

I kind of wish they would have added one more picture to compare which was an actual high res shot.

So you could compare low res, high res, and filtered high res.

They do look impressive though!

Re:Somewhat as if they were painted by Van Gogh! (1)

wfaulk (135736) | more than 12 years ago | (#127324)

Unfortunately, it appears that the sample they gave it to learn from contains a vast majority of horizontal brush strokes, as Van Gogh tried to emulate the rippling of the water and match that in the night sky. The only non-horizontal strokes are in the stars, very slightly in the buildings and in the boat at the bottom. I wonder what it would have looked like if they had taught it from Sunflowers [vangoghmuseum.nl] or Irises [vangoghmuseum.nl] or even The Bedroom [vangoghmuseum.nl].

It would be even more interesting to take The Bedroom and produce a before picture for it that straightens the odd angles and de-fisheyes it. What sort of schizophrenic things might it produce then?

already done (1)

asv108 (141455) | more than 12 years ago | (#127325)

Corel painter 6 formaly Fractal painter has an auto Van Gogh filter. Fractal Painter 5 introduced the auto VanGogh filter.

patterns-by-numbers (1)

teefal (142948) | more than 12 years ago | (#127326)

To me, the most interesting part of the project is their patters-by-numbers. I can see this being useful immediately. Anyone know of Photoshop filters that do similiar tasks?

Re:Call me crazy, but... (1)

jelson (144412) | more than 12 years ago | (#127328)

Because they generated the "before" shot of the training pair, not the "after" shot.

i.e.: They started with a painting by Van Gogh, then ran it through the "Smart Blur" filter of photoshop to remove the Van Gogh-esque-ness of the painting, leaving a textureless image. Then they ran the texture learning algorithm over that pair of images, and applied the learned texture to the target.

Make a bad painting from a good photograph (2)

zincks (145594) | more than 12 years ago | (#127330)

Hey this is pretty cool. You take a good photograph, and make it into a bad painting. Paining is more than just mixing oil paints to be the average color for part of an image... Van Gogh paintings typically emphasize the edges, the cheekbones, the jaw, the eyebrows. I'm bitter that someone can take an image, use xv to sharpen the image and then shift the colors and get a PhD out of it. I wonder if he was funded by the department of defense?

That's too bloody cool (2)

Echo|Fox (156022) | more than 12 years ago | (#127333)

As I was going through the page I was fairly unimpressed, largely. Much of what was there has been in Photoshop (and probably the Gimp) for years now, i.e. filters to make photo's look like paintings or apply textures or whatever.
However ... the texture by numbers section was exceedingly cool and I could see a myriad of uses were someone to port it to Photoshop's plugin format. Being able to take an image, mask it with various different colors, and then have an entirely new image generated based on the textures you extracted ... yeh, thats nifty =)
By far the coolest thing I've seen on /. in a while.

Our Wow threshhold (2)

fm6 (162816) | more than 12 years ago | (#127334)

This is actually quite impressive software.
Impressive how? Lots of paint programs have Van Gogh filters. It's easy to "reproduce" a given artist by superficially imitating some well-known aspect of his or her style. In the case of Van Gogh, you just need to fuzz up the image a little. But there's a lot more to Van Gogh than fuzziness!

I'd be more impressed if they had trained the software to imitate multiple artists, such as O'Keefe [byu.edu], Rembrandt [mystudios.com], Klee [postershop.co.uk], Gauguin [artprintcollection.com], Monet [cafeguerbois.com], Picasso [guggenheimcollection.org], etc., shown as a single image, as processed by all of the above, side by side.

__

Maybe those movies weren't wrong... (1)

WuTangClanner (181082) | more than 12 years ago | (#127337)

Wow, maybe those parts in movies (that we all criticize) where the computer nerd enhances a still, low-res image to show the bad guy's face in a blurry crowd of people might become reality sooner than anticipated. Then we'd all feel bad for laughing at those scenes...

...nah

:)

Re:Van Gogh (1)

Chundra (189402) | more than 12 years ago | (#127338)

There are already computers that behave like Salvador Dali. They run windows.

*bsod*
Windows user: Fucking computer, it's on drugs.
computer: "I don't use drugs. I am drugs."

Re:Van Gogh (1)

Chundra (189402) | more than 12 years ago | (#127339)

oh yeah, forgot a few:

windows dali box:
"So little of what could happen does happen."

"Have no fear of perfection -- you'll never reach it."

"It is either easy or impossible."

"It is good taste, and good taste alone, that possesses the power to sterilize and is always the first handicap to any creative functioning."

"To gaze is to think."

"Those who do not want to imitate anything, produce nothing."

"Wars have never hurt anybody except the people who die."

Re:Call me crazy, but... (2)

ichimunki (194887) | more than 12 years ago | (#127340)

Actually, what they haven't shown (that I saw) that would be most cool for this is to have a photo of a scene and a painting of that scene and have the filter learn that! Otherwise, what they have done that's still pretty neat is built a system that figures out what mathematically needs to be done to one set of numbers (i.e. the RGB bytes for each pixel) to get it to a second state. Not bad.

Additional applications? (1)

martyb (196687) | more than 12 years ago | (#127341)

What I've started to wonder is where else it's underlying principles could be used, or where this sort of technology could lead in the future.

I was thinking along the same lines! (umm, well, brushstrokes?) Granted the amount of computation likely precludes real-time generation today. But, I could well imagine that within (pulls number out of the air) 2-5 years, it should be quite possible. Applications? Here are some ideas (some are admittedly off-beat, but why not?)

  • Tivo: Watch Friends, Battlebots, etc. in a "whole new light".
  • Home videos: Show videos of your trip to the Grand Canyon, ala Rembrandt.
  • pr0n: Sometimes, the suggestion is more powerful than the details. Enter the impressionists!
  • Cartoons: Bugs Bunny, Wylie Coyote, and Marvin the Martian done YOUR way.
  • Video Games: Would make for a real challenge to try and play as simple a transform as a watercolor rendition of quake. For the truly adventurous, try a Picaso filter! :)
  • Any Videotape or DVD: You get the idea.

I have no illusions the results would be spectacular, but I'm quite sure they could be really interesting. Heck, even a commercial or a presidential speech could take on a whole new perspective.

Nitpicking (1)

KurdtX (207196) | more than 12 years ago | (#127342)

Did anyone else notice for the "Van Gogh" images, it only did horizontal brush strokes. I'm not any kind of art buff, but I don't think artists would only use horizontal brush strokes. Although maybe it's a problem with the training data.

Also, the "image enhancing" stuff wasn't too impressive, yet. Maybe with more examples it might do better. Of course, the thing to remember is it's not programmed for any particular filter, but it still had a problem ignoring small details, like the black borders for the rugs.

Kurdt

In other news ... (3)

BlowCat (216402) | more than 12 years ago | (#127343)

For example, given a Van Gogh painting, the algorithm can process other images to look somewhat as if they were painted by Van Gogh.
Another algorithm, given a post on Slashdot, can produce comments to look as if they were created by Slashdot readers.

Composition (1)

ManDude (231569) | more than 12 years ago | (#127345)

I think what is missing is the composition. Van Gogh didn't just paint with funny streaks and wobbly reflections. He picked subjects and views that he wanted to express. Computers can do it, but not in the same vein as an artist. We would need a thousand monkeys first. Oh and a some absinth (Absinth.com).

Call me crazy, but... (2)

BoarderPhreak (234086) | more than 12 years ago | (#127346)

If you have a "before" and "after" shot (therefore implying that you have a/the effect filter), then why don't you just apply the same filter to image B that you did with A to get to that "after" shot?

That is, why a "learning filter?"

It would be more useful if it could discover a technique by looking at one image.

Re:Wow. Hi-res pics are cool. (1)

nanoakron (234907) | more than 12 years ago | (#127347)

True, I have to agree with apnu.

I hope the sample images aren't the be-all-and-end-all extent of their program w. respect to Van Gogh-like rendering. Painting is to represent what you see in a permanent form - I believe Van Gogh would have turned a sky full of clouds into something other than a smudge of off-white patches - that he would have tried to create an accurate 'representation' of what he could see onto the page.

But nonetheless, it is an impressive learning algorithm and perhaps has a future in processing for image cleanup....you know, that crap they show in movies where a 'computer guy' presses a few buttons and a blurry satellite photo suddenly shows license plate numbers...

Let's just say I was impressed by the overall approach, but not by the examples given (brush strokes to uniform and lacking attention to the defining details of the images).

-Nano.

Somewhat as if they were painted by Van Gogh! (2)

ishrat (235467) | more than 12 years ago | (#127348)

This is what it is exactly and nowhere near the original style of Gogh. Those familiar with his style can straight away differentiate in the style, as the strokes are all horizontal and the artists srokes were in all directions giving a bold movement to the painting.

Looks like these guys have much to improve. Yet it is an effort in the right direction and we can congratulate them, for at least their program can copy lesser styles, if not the masters.

And above all no matter how a good a copy it is, the copy can never measure up to the original.

Re:Call me crazy, but... (1)

dlkf (261011) | more than 12 years ago | (#127352)

Your implication does not always hold. The power of this tool is that you dont have to have the filter. If you do, dont bother with the learner. If for some reason you dont have the filter, then this will be very useful. Look at some of the paint by numbers examples. They are a very good demonstration of the power of this tool.

DeCSS? (2)

kilgore_47 (262118) | more than 12 years ago | (#127353)

If someone can use this to encode DeCSS, so it cans till be decoded, maybe THAT will convince people that its an artistic piece.

Seriously, though, this is an amazing project. I'm impressed.
____
Blood, guts, guns, cuts;
Knives, lives, wives, nuns, sluts.

Re:That's too bloody cool (2)

chhamilton (264664) | more than 12 years ago | (#127354)

As I was going through the page I was fairly unimpressed, largely. Much of what was there has been in Photoshop (and probably the Gimp) for years now, i.e. filters to make photo's look like paintings or apply textures or whatever.

Yes, Photoshop and the Gimp have had similar filters for a long time, but that's not the point of the project. Filters that are used in the Gimp and Photoshop were hand created and are each specialized and unique. This algorithm provides a general framework for learning how to apply a given filter. Ie: it can be used to easily create filter types that didn't exist just based on sample inputs and outputs. It's broader than the relatively simple filter operations found in typical imaging tools...

I must admit though, the texturizing feature is damn cool.

Re:Exactly what I was talking about (1)

regexp (302904) | more than 12 years ago | (#127355)

I agree that computer graphics *can* be an effective tool or medium for creating great art. But you can't seriously call those "Van Gogh-like" filtered photographs art. It's just a little trick of image processing. No one is likely to mistake those ugly images for a real Van Gogh painting or even a passably good non-Van Gogh painting.

Hi-rez from Lo-rez (1)

GuyZero (303599) | more than 12 years ago | (#127356)

So, there's no need to explain that you can't make data up out of thin air and that it's impossible to really get more resolution out of an existing image. The same applies to out-of-focus images, under or over exposed images, etc. I mean, if you could do any of that you'd be able to do other cool things, like making an infinite-compression file compressor. Woah!

Anyway, while it's interesting that they trained the tool to "increase" resolution, I doubt it's a generally useful technique. I mean, look at the training pairs compared to the test data. I'd like to see what the results woul dbe if they switched the sets of training data - hi-rez-ify the carpet with the forest data and vice-versa. That would probably look... very artistic.

Re:photoshop (1)

GuyZero (303599) | more than 12 years ago | (#127357)

No, what they did is train a computer to replicate the 'brush-stroke' effect without the computer having any a priori or internal knowledge of how to make a brush stroke.

It probably uses a neural net or something similar. The paper itself hasn't been published yet, so who knows.

Applications (1)

pgpckt (312866) | more than 12 years ago | (#127360)

This seems to me to have a great deal of potential. There have been some comments thus far that this isn't the first time such a thing has been done before. Error correcting algorithms help satellites to transmit correct images, and there are some programs with filters on the market.

I see this as being a more sophisticated program. And its free...come on, you /. people love that kind of thing. Source code too.

The thing about this program is that it isn't a bunch of pre-collected filters, like any good imaging software, but that it can create new filters on the fly. So, if you want to recreate something to look like something else, you can. Not morphing, but in similar styles. That is very cool. Totally custom and unique filters, and that is new.

I really like the pictures where there is a painting that is converted to look like a river in a marsh. [nyu.edu] The computer makes it look real. So there is potential for artists. Scientists can use it to project data based on samples. I think there are some unique and useful uses to this program.

Uh-Oh (1)

Jin Wicked (317953) | more than 12 years ago | (#127362)

I guess I'll be obsolete soon. At least computers can't dress up and black and sit around in coffee shops, while making strange faces at the people that walk by. Yet. :(

Of course, on printers (which is what most people would have access to) the best this program can do is create the illusion of depth and paint texture. The printed surface would still be flat. Van Gogh's paintings are very thick and textured... he would sometimes even use his fingers to paint because the brush wouldn't hold enough to satisfy him. Or are they working on a solution for this problem too?

There is more to a painting than just the image. There's the warmth of the medium it was created on (wood, canvas, clay, etc.) and the media used to create it. Oil paints have a certain shine, texture, scent, and even behaviour as they crack and dry out over time.

Why don't we just build a robot to paint for us, so we don't have to waste our valuable brain power any more doing things like being creative or actually having ideas? Then we can build robots to appreciate the robotic art, because we'll all be sitting in front of the TV watching Friends every night. Why even bother being human to begin with? :(

Bah! (1)

cosmo7 (325616) | more than 12 years ago | (#127366)

How can people be so clever and yet so stupid?

We don't need computer's to produce Van Gogh paintings we had Van Gogh to do that, and the changes in the way we look at artwork are already made. This is at best a homage to Van Gogh, but I really don't see the point. If the algorithm has to learn the style of a painter then it is not producing anything new. Which is the whole point of art. Right?

Re:Maybe those movies weren't wrong... (1)

TheAwfulTruth (325623) | more than 12 years ago | (#127367)

Ok, but when are we going to have computer enhancement to 2d photographs that let you look around corners to see stuff behind walls like in Bladerunner? :)

Re:Wow (1)

Hays (409837) | more than 12 years ago | (#127368)

"Could it be used to analyze text from certain authors (hey, text and art are no different to a computer - treat words as "pixels" and sentences
and structures flow like colors) and mimic their style? Could this one day be used to turn my dull crud into something Fitzgerald or Hemingway
or even Asimov or Heinlein might have written? "
Hmm... well it would be tricky to do that conversion. Text is pretty linear and images are 2d. You could do it, though, I imagine you'd get garbage out of it. I recall some previous work done with Markov Random Chains in a similar way, and it produced some funny results.

"What happens when one feeds a Van Gogh through the Van Gogh filter? Does the resultant image change much?"
There's not a "Van Gogh" Filter, if you're trying to go from a photo to a Van Gogh painting, you need a training photo that has been converted to a training Van Gogh painting. Thing is, Van Gogh didn't photograph his subjects before painting them. So they use a filtered Van Gogh painting and it gets the job done. Now if they put another Van Gogh painting filtered in the same way through the system with the same training images as that web page shows, it should look like a Van Gogh again, though it would defintely change. (less if it were a work from the same period of his work, very dependent on local structure)

"Does the program apply the "filter" differently depending on what type of input it encounters, or is the same method applied to all input?"
I think it uses the same neighborhood kernal for all images.

"Conversely, can the program be used to recognize when a work is of a certain artist?"
not really, there's no higher order cognitive force driving the procedure. Just finding best neigborhoods and secondary neighborhoods.

"Or can it be used to see if an image has already been passed through a certain filter?"
Again, there's no metric for comparing that in this project that I know of. It converts, it doesn't evaluate.

"Are there cases which cause the method to fail or create an undecipherable image? And if so, are these cases unique or do they conincide with a certain type of artistic style? [e.g. Monet -> Van Gogh just won't work right?]"
Yes, things that are global effects do not work since Hertzmann is using only neighborhood comparisons. Things like watercolor could not be done correctly, as it is not entirely local in structure. You couldn't make a cubist painting either.

I'm doing work similar to some of Hertzmann's previous papers (Non-Photo-Realistic rendering for video). Image Analogies is going to be one of the more populas SIGGRAPH papers this year, and it's deserved. There are however limitations to this procedure. I think people here are getting the impression that it's some brilliant art evaluating and painting program when it's anything but. It's a clever (every research slaps self saying why didn't I think of this) method for duplicating local transformations characteristic of a certain image style. Few things in Vision are, however, intelligent or AI based.

Re:Applications (1)

Olinator (412652) | more than 12 years ago | (#127370)

I really like the pictures where there is a
painting that is converted to look like a river in a marsh [nyu.edu]. The computer makes it look real. So there is potential for artists. Scientists can use it to project data based on samples. I think there are some unique and useful uses to this program.

Scientists? Good grief. This is a military application -- you feed a computer lots of data on how to interpret infrared or other long-range recon data for a given type of climate, and then you can construct reasonable VR fly-throughs to train your pilots or image-recognition missiles or ground troops or whatever. The fact that it's apparently possible to do this with COTS components is probably going to make a lot of brass hats unhappy...

why NPR? (1)

Gary Yngve (416254) | more than 12 years ago | (#127371)

A lot of people seem to be confused on why we're interested in nonphotorealistic rendering. It is not because we want to reproduce exactly Van Goghs or Monets or Dalis. Surely one could argue convincingly that it would require solving the AI problem.

Instead NPR is about an alternative way to present content efficiently. Technical sketches are easier to understand than photographs of the same part. We might want an impressionistic style to an animation to capture a dreamy mood. We might want cartoon physics to do Roadrunner/Coyote stuff. A person can understand and feel a grossly simple cartoon but feel disturbed by a photorealistic rendering with imperfect shadows. There are other good reasons for NPR, but I'm off to an ultimate game...

Just make sure your training data is good... (4)

bacaloca (447079) | more than 12 years ago | (#127374)

This learning algorithm looks similar to a neural net. My CS professor told us a story about the US government making a neural net that would be used to recognize tanks in satellite photographs.
They spent many hours loading photos for training data. Some photos had tanks and others did not. Once they thought it was working, they tried it on some non-training data --it didn't work.
Instead of recognizing tanks, it learned to distinguish cloudy days from ones with sunshine. All of the pictures with tanks were taken on a cloudy day & all other pictures were taken on days that had sunshine.
He didn't know if they ever actually got it working or not...

missing the point? (2)

Johnny5000 (451029) | more than 12 years ago | (#127375)

I think the important point of this article is not that they can make the painting look sort of like a Van Gogh painting.

I think the new important discovery is that they can take an image and process it so it looks sort of like a Van Gogh, or a painting I did, or a painting your pet chimp did. Or maybe not even necesarily a painting at all.

Instead of saying "here's a photo, make it look like a painting" they are saying "here's one image, now make this other image look sort of like the first one."

-J5K

Van Gogh (5)

Greenrider (451799) | more than 12 years ago | (#127376)

Terrific...a computer that behaves like Van Gogh.

Next thing you know I'll come home from work to find that my PC has severed its own mouse cord in a fit of psychosis.

Scary (1)

SilentChris (452960) | more than 12 years ago | (#127377)

You know what's going to be real scary? When they create an algorhythm/machine that takes paintings by original artists (perhaps still wet) and transcribes them to "Van Gogh" pictures. With the brush strokes and everything. The technology is certainly there on the processor side (getting a good painting machine might be difficult).

But can you imagine all sorts of artists being able to "Van Gogh-ize" their paintings? You thought creativity was being diminished in society before...

nice application of existing techniques (2)

m08593 (455349) | more than 12 years ago | (#127378)

The underlying techniques are actually pretty widely used already in image processing and pattern recognition. And, yes, similar techniques are used in text analysis and many other areas. This is mainly a particularly neat demonstration and application of them.

How useful this particular application is remains to be seen; most people probably have a harder time giving an example of a van Gogh filter for the system to learn from than to use a canned filter.

Re:RAH (1)

Saeger (456549) | more than 12 years ago | (#127379)

Don't be afraid. :)

There's absolutely no doubt that one day us human's will shed our fragile shells; but the important part that makes us human will still live on through our "mind children." I've decided to name my 1,000th offspring, Babbage, my 2000th, Turing, and my 5000th, Moravec. No room for a Minsky in there. :-)

......That reminds me......for a laugh:

717 01:10:44,530 --> 01:10:46,521 I wish to merge with you.

718 01:10:46,599 --> 01:10:49,796 - Merge? - A complete joining.

719 01:10:49,869 --> 01:10:55,205 We will both be slightly changed, but neither will lose anything.

720 01:10:55,274 --> 01:10:59,574 Afterwards, it should be impossible to distinguish one from the other.

Re:Hi-rez from Lo-rez (4)

Keith Handy (456832) | more than 12 years ago | (#127380)

You're correct that it's impossible to increase *actual* resolution, but in the cases of art and music it's often quite desirable to *simulate* increased resolution. Much is done with lo-fi music to enhance the perceived high frequencies to make it more pleasing to the ear, though this doesn't restore actual high frequency information lost from the original performance. I would assume similar principles are applied to visual art in this case.

how to test it? (1)

thejustice (461538) | more than 12 years ago | (#127382)

if you seriously want to find out how good it is: TRAINING 1 -> TRAINING2 BEFORE-1 -> AFTER-1 and then TRAINING 2 -> TRAINING 1 and AFTER-1 -> after -2 and compare after 2 with before 1 :)

Re:Hi-rez from Lo-rez (1)

thejustice (461538) | more than 12 years ago | (#127383)

this reminds me of mp3-pro.. this image tool could prove to be the greatest compression algorithm since jpg then if it would be 100x faster :)

I'm too damn lazy to Paint (1)

ejracke (461642) | more than 12 years ago | (#127384)

Does this mean I don't have too watch BOB anymore? It's just that those trees are just too hard to render on my own!!

Exactly what I was talking about (1)

abitkin (462096) | more than 12 years ago | (#127385)

This is exactly what I ment when I said computer graphics could go the same way as photography. (look for my message in the ask slashdot: fine art?) Photographers trying to get the technique reconized as art blurred their pictures to make them look more like paintings.

Now it seems that computer graphics can take the same aproach. The intresting thing is photography did not become an artform as a a result of blurring the pictures. It became an artform by having people reconize being in the right place at the right time, and being able to see a message in the movements of the everyday takes as much work, skill and insight as painting.

(Apple thinks different, creates the iMac)
Yahoo Finance--
Company: Apple

Re:Exactly what I was talking about (1)

abitkin (462096) | more than 12 years ago | (#127386)

The point of my message was, making photos look more like paintings didn't make them fine art, and by the same token making computer graphics look more like paintings wont make them fine art either. The key is to understand: that computer graphics, (even if we to filter them to make them look more like a painting) is a media in which someone can express feelings. That expression of feeling is the very essence of art.

(Apple thinks different, creates the iMac)
Yahoo Finance--
Company: Apple
Load More Comments
Slashdot Account

Need an Account?

Forgot your password?

Don't worry, we never post anything without your permission.

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>
Sign up for Slashdot Newsletters
Create a Slashdot Account

Loading...