Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Content-Aware Image Resizing

kdawson posted more than 7 years ago | from the got-a-nice-gui-too dept.

Graphics 174

An anonymous reader writes "At the SIGGRAPH 2007 conference in San Diego, two Israeli professors, Shai Avidan and Ariel Shamir, have demonstrated a new method to shrink images. The method is called 'Seam Carving for Content-Aware Image Resizing' (PDF paper here) and it figures out which parts of an image are less significant. This makes it possible to change the aspect ratio of an image without making the content look skewed or stretched out. There is a video demonstration up on YouTube."

cancel ×

174 comments

Sorry! There are no comments related to the filter you selected.

I For One (-1, Offtopic)

iPaige (834088) | more than 7 years ago | (#20357427)

I For One Welcome Our New Content-Aware Image Resizing Overlords.

Re:I For One (0)

Shikaku (1129753) | more than 7 years ago | (#20357457)

I thought JPEG already did this.

Re:I For One (3, Insightful)

cnettel (836611) | more than 7 years ago | (#20357533)

It's not compression as we know it, Jim. It's more like scaling on totally overcool steroids. The basic idea seems rather simple. I would even imagine you could get a bit of enhanced picture quality by coding simplified vector info on seams, and then doing a normal JPEG of a downscaled picture. That would be a quite contrived way to get a kind of VBR-like behavior in normal JPEG. One issue with JPEG is, after all, that redundancy is detected and handled on the block level, while this algorithm works along arbitrary paths.

I'm really impressed. Again, maybe not too hard to implement at first, but probably damn hard to get working perfectly, and I might just be ignorant (and I'm entitled too, it's far from my field of work), but I've not seen anyone doing it before.

Let us be wholly thankful... (1, Funny)

Arthur Grumbine (1086397) | more than 7 years ago | (#20357809)

What is unmentioned in the PDF is that in order to test significance-measuring algorithms for parts of an image, they used goatse. After thousands of tests they were only satisfied once the method returned a blank image.

Those performing the evaluation of the test results before the final version were forced to be institutionalized.

I would like to take this opportunity to personally thank those brave soldiers for their unblinking fortitude in this ultimate self-sacrifice. It's times like this when I become truly aware of my own gaping inadequacies, and feel the deep, deep obligation to rectify my own short comings.

Re:Let us be wholly thankful... (3, Funny)

kennygraham (894697) | more than 7 years ago | (#20358267)

It's times like this when I become truly aware of my own gaping inadequacies, and feel the deep, deep obligation to rectify my own short comings.

hehe... gaping... deep deep... rectum... i mean rectify... hehe

i need to get some sleep

The paper via ACM (4, Informative)

xenocide2 (231786) | more than 7 years ago | (#20357465)

The author's website was pegged serving that 20MB PDF before slashdot got ahold of it, I doubt it'll survive now. The paper is also hosted by the ACM [acm.org] , if you're a subscriber.

Re:The paper via ACM (4, Informative)

spydir31 (312329) | more than 7 years ago | (#20357685)

The Coral Cache" [nyud.net] has it also.

Re:The paper via ACM (5, Informative)

Anonymous Coward | more than 7 years ago | (#20357763)

I used a lossy compression algorithm on their paper and got this...

Shrink image:
Step 1: Run an edge detection algorithm.
Step 2: Find minimal energy (least amount of edges crossed) path from top to bottom or left to right (graph-cut algorithm).
Step 3: Remove pixels along that path.
Step 4: Repeat steps 2 and 3 as necessary.

Extend image:
Step 1: Run an edge detection algorithm.
Step 2: Find minimal energy (least amount of edges crossed) path from top to bottom or left to right (graph-cut algorithm).
Step 3: Insert pixels along that path (interpolated from neighbors)
Step 4: Repeat steps 2 and 3 as necessary.

Remove objects:
Step 1: Run an edge detection algorithm.
Step 2: Mask object by giving its pixels low/negative energy values.
Step 3: Find minimal energy (least amount of edges crossed) path from top to bottom or left to right (graph-cut algorithm).
Step 4: Remove pixels along that path.
Step 5: Repeat steps 3 and 4 as necessary.

Re:The paper via ACM (5, Insightful)

Anonymous Coward | more than 7 years ago | (#20358119)

I think you've got it except for a small detail in the "Remove objects", which the narrator alludes to around timestamp 4:01 of the video. You might want to add:

Step 6: Extend image to match original size using the previous extend image algorithm

(Of course, I leave the obligatory Profit step as an exercise for the reader).

Re:The paper via ACM (2, Informative)

DotDotSlasher (675502) | more than 7 years ago | (#20358579)

This excellent website http://trowley.org/sig2007.html [trowley.org] has a host of links to almost all of the papers presented at SIGGRAPH 2007.

Re:The paper via ACM (0)

Anonymous Coward | more than 7 years ago | (#20358697)

And the full resolution demo video [idc.ac.il] .

Great (0)

Anonymous Coward | more than 7 years ago | (#20357469)

I really like the way where the people are erased and you can hardly notice it. But the when it comes to the faces, the algorithm seems to need more work.

Re:Great (1)

pclminion (145572) | more than 7 years ago | (#20358889)

I really like the way where the people are erased and you can hardly notice it. But the when it comes to the faces, the algorithm seems to need more work.

The authors have already demonstrated how automated face recognition could be applied to protect face-like areas on the images. I'm not sure what else you want.

Holy crap, this is awesome. (1)

samwh (921444) | more than 7 years ago | (#20357473)

I wonder when we can get their software that does this...? The "erase" function almost seems like a better then image inpainting.

The Commissar Vanishes (1, Funny)

Anonymous Coward | more than 7 years ago | (#20357577)

This technology could render very visually-convincing (but not computer/analytically convincing) image censorship or alteration. I am strongly reminded of this example of photo-editing from the 1940s:

http://www.newseum.org/berlinwall/commissar_vanish es/vanishes.htm [newseum.org]

I don't mean to instigate a knee-jerk, authoritarian censorship discussion. I think it's obvious that this technique is just plain cool and has great potential for beneficial use, even if it might be used for ill. That's just an intersting historical example that it might have made easier in execution (har-har; gallows humor and pun 2-for-1!).

Re:The Commissar Vanishes (1)

tomhudson (43916) | more than 7 years ago | (#20357731)

"This technology could render very visually-convincing (but not computer/analytically convincing) image censorship or alteration. I am strongly reminded of this example of photo-editing from the 1940s:

http://www.newseum.org/berlinwall/commissar_vanish [newseum.org] es/vanishes.htm "

Need I remind you, komerade, kommisar Nikolai Yezhov was originally ADDED to pikture, and that our Ministery of Truth only restored the photo to original kondition? Everyone knows that in Soviet Russia, photo alters YOU! Now, your papers, please. And remember to smile for the kamera, komerad.

Re:The Commissar Vanishes (1)

Lars T. (470328) | more than 7 years ago | (#20357881)

Yeah, Americans would never stoop so low - they gloriously reenact the photo op [iwojima.com] . Much more honest that way.

Re:The Commissar Vanishes (2, Interesting)

larry bagina (561269) | more than 7 years ago | (#20357941)

It could be worse.

In December 2001 The New York Fire Department unveiled plans for a statue based on the photograph [wikipedia.org] to be placed at the Brooklyn headquarters. In an effort to be politically correct, the statue was to include black, white, and Hispanic firefighters. However, it was cancelled in an outcry about rewriting history -- the depicted firefighters are white.

Re:The Commissar Vanishes (1, Insightful)

Anonymous Coward | more than 7 years ago | (#20358505)

PC.. The 'correctness' part of the phrase really irks me. It's censorship, propaganda and thought police all at the same time. At least with a bigot you know where you stand - with a PC freak nothing is sacred, noone is safe. Our society would be far better off with blatant racism over this politically correct crap any day - at least racism is in the realm of debatable idiocy, the kind of PC stupidity you presented just has no rational, logical or sane evaluation possible.

Re:The Commissar Vanishes (1, Informative)

ArcherB (796902) | more than 7 years ago | (#20358427)

Re:The Commissar Vanishes (0, Troll)

tomhudson (43916) | more than 7 years ago | (#20359227)

Citizen, you have resurrected the vanished komissar. This is prohibited by the DMCA and the PATRIOT Act. We kannot tell you exactly what you have infringed, that too is klassified as anti-capitalist behaviour.

You are obviously a either a kommunist or suffering some form of mental incapacity. Fortunately, in Soviet Amerika, we are generous, and give you the benefit of the doubt. We have reserved a room for you at the Gitmo Health Cpa and Vakation Resort.

Your bank account has been pre-billed so you kan take advantage of our generous discount rates.

We will take kare of informing your relatives, to spare you any embarrassment; Please sign this blank konfession^Wletter to them.

This [trolltalk.com] is a picture of the last person who failed to show up for their health treatments. These [trolltalk.com] are pictures of his family. Yes, it is most unfortunate, they were obviously deranged.

"- - My name is George Bush, and Dick Cheney Says I Approved This Message"

Re:The Commissar Vanishes (1)

Glytch (4881) | more than 7 years ago | (#20357789)

I agree. This is a very cool technology, but definitely underscores the trust problem in journalism. We're fast approaching (and some would say long past) the point were we need digital signatures for photos published on news sites. People need to know that what they're told is reality is actually reality. I think this software is fine for any sort of photography where reality isn't as important as aesthetics (artistic landscapes, portraits, abstracts, fine art, etc) but it has no place in ethical photojournalism.

Re:The Commissar Vanishes (1)

Original Replica (908688) | more than 7 years ago | (#20357955)

There is already so much "spin" in modern journalism that you have to learn about any event of importance from sources that you know are biased in opposite directions, just to be able have a good guess at the actual facts. This is why you get more truth from The Daily Show than you do from Fox News or Air America, The Daily Show's spin is obvious because it's funny and/or ridiculous.

Re:The Commissar Vanishes (0)

Anonymous Coward | more than 7 years ago | (#20358131)

Well, if the ease of manipulation makes people a bit more circumspect about imagery, that's at least one good effect. The realism of photos gives people a heady sense of omniscience. But even if the picture has not been changed, and if the scene has not been staged, it's still only one small slice of what happened, and can give people the dangerous idea that they know everything that happened, will happen, or should have happened.

"The general killed the Viet Cong; I killed the general with my camera. Still photographs are the most powerful weapon in the world. People believe them; but photographs do lie, even without manipulation. They are only half-truths.

What the photograph didn't say was, 'What would you do if you were the general at that time and place on that hot day, and you caught the so-called bad guy after he blew away one, two or three American people?'"


-Eddie Adams [wikipedia.org]

Re:Holy crap, this is awesome. (0)

Anonymous Coward | more than 7 years ago | (#20357639)

If the algorithm isn't patented, you'll quickly see implementations. Once you've seen it, the algorithm is almost trivial: Run an edge detection algorithm, find minimal energy (least amount of edges crossed) path from top to bottom or left to right (that's a graph-cut algorithm), remove pixels along that path. Repeat step 2 as necessary.

Impressive (1)

CastrTroy (595695) | more than 7 years ago | (#20357481)

Watching the video, this seems pretty impressive. It's also neat to see them use the same technology to remove items (people in this case) from an image with only about 10 seconds of work.

I Think You'll Find (3, Insightful)

JamesRose (1062530) | more than 7 years ago | (#20357613)

10 Seconds of work there, most probably a good deal longer finding a picture that is easy to do it to...

Impressive-Clip art. (0)

Anonymous Coward | more than 7 years ago | (#20357647)

"It's also neat to see them use the same technology to remove items (people in this case) from an image with only about 10 seconds of work."

Newspapers rejoice!

nice! (4, Interesting)

White Shade (57215) | more than 7 years ago | (#20357485)

It seems like a little bit of work is left to make it as completely automated as you would need to have it just "always work" on any platform or device, but it seems like they're already working on that...

Other than that though, that's pretty awesome... I'm sure there's more instances where it doesn't look right than what they showed, but it's definitely cool how well it works as it stands!

I can imagine it would be extremely useful for ex-boyfriends or ex-girlfriends; just load up all their photos of them and their ex, wave the magic eraser, and *boom* you don't have to delete all your old vacation shots ;)

I wonder how well it would work for the porn industry too; nice automatic resizing of breasts without ruining the picture! Fetishists will be SO happy! :)

Re:nice! (2, Interesting)

Allicorn (175921) | more than 7 years ago | (#20357681)

It seems like a little bit of work is left to make it as completely automated as you would need to have it just "always work"
Completely right, yes. The images in the video have been selected to show this technique in the best possible light. There's a great variety of images that'll really not work quite right with a completely automated treatment. Speaking from experience having implemented this last week.

That said, as pointed out in the paper there's plenty of room for a higher level of analysis over the top of the basic seam-carving procedure. The function used to calculate the energy of a given pixel is easily swapped out with any one of dozens of different approaches. A more user-friendly implementation could attempt seam-carving based on a number of different feature maps and work out which is likely to produce the least distortion for a given image.

Anyhow, cracking bit of work IMHO, with boatloads of potential applications.

Alli

Re:nice! (-1, Troll)

tomhudson (43916) | more than 7 years ago | (#20357683)

"I wonder how well it would work for the porn industry too; nice automatic resizing of breasts without ruining the picture! Fetishists will be SO happy! :)"

They obviously "cherry-picked" the photos they worked with. Even then, there are problems with how they "distort reality", so that the landscape isn't what its represented to be. Somehow, I think it won't have such an easy time being so "content-aware" with this picture [trolltalk.com] , or any of these "beauty contest" entries [trolltalk.com] . And who'd want to - the reality is more than enough without being "enhanced!"

Re:nice! (1)

PsyQo (1020321) | more than 7 years ago | (#20357691)

We didn't even have ONE girlfriend, let alone an ex, you insensitive clod!

Re:nice! (1)

ceoyoyo (59147) | more than 7 years ago | (#20357725)

Problem is you have to remember to keep some nice featureless space between you and your boy/girlfriend in case you want to erase them afterwards. Unless you don't mind becoming an amputee in your photos, that is.

Re:nice! (5, Funny)

aliquis (678370) | more than 7 years ago | (#20357757)

I'd never understod this hate-your-ex-thing? The person where part of your life for some time but you have decided to hate it and want to erase it from it?
Better never get a partner then at all if you are going to hate the person once it doesn't work longer.

But then I'm a regular slashdot visitor and don't have any exs so what do I know.

Re:nice! (1)

Thalagyrt (851883) | more than 7 years ago | (#20357995)

Sometimes keeping a friendship after severing those types of ties can be hard as hell just because of all of the emotions involved -- it can be painful to see or hear about them. I only really understand the hate-the-ex thing in certain circumstances (cheating, being a total bitch/asshole/whatever) but in general if it's possible to keep a friend then keep a friend. It's just tough, which overall leads most people to just shut out their exes entirely... I'm currently trying to keep my friendship with my ex from completely falling apart, it's simply not an easy thing to do.

Re:nice! (1)

Original Replica (908688) | more than 7 years ago | (#20358017)

I'd never understod this hate-your-ex-thing?

Pay alimony for awhile and I'm sure you'll understand it much better.
Most of the bitterness arises because you didn't get a good return on your investment of time/love/money/etc. You expected lifelong compatibility, but got a partner that has a significant personality change in the first few years of you relationship. I take more of a "high price of a lesson learned" view of relationships that ended badly, but then I just toss the photos and old cutesy stuff out, no need to try to edit it.

Re:nice! (0)

Anonymous Coward | more than 7 years ago | (#20358415)

The hate thing is a natural self-defense mechanism that helps you move on instead of getting attached to the we-were-happy-why-did-we-brake-up feeling. Also, whether you dumped your partner or you were dumped, brake-ups usually involve a big deal of pain that you will most likely associate with your ex.

So in summary, hating them is a good way of not missing them, especially if you are not really over them yet.

Re:nice! (0)

Anonymous Coward | more than 7 years ago | (#20358645)

it's about betrayal

Re:nice! (1)

pclminion (145572) | more than 7 years ago | (#20358915)

I'd never understod this hate-your-ex-thing? The person where part of your life for some time but you have decided to hate it and want to erase it from it? Better never get a partner then at all if you are going to hate the person once it doesn't work longer.

Sometimes the reason it doesn't work any longer is because you've grown to hate the person.

I don't think any reasonably adjusted person would culture hatred for a past parter just because the relationship didn't work out. People who do otherwise are just being childish.

Re:nice! (1)

Feanturi (99866) | more than 7 years ago | (#20359455)

It doesn't have to be hate that makes you throw out all the stuff that reminds you of them. The pictures, letters, knick-knacks, whatever, conjure memories of when you were happy with them. This brings much pain because it is a strong reminder that they are not there now and not coming back. I think a reasonably adjusted person would not wish to live in a shrine of lost memories. Get rid of it and you can get on with life all the sooner.

She can just DIAF (1)

Scrameustache (459504) | more than 7 years ago | (#20359133)

Better never get a partner then at all if you are going to hate the person once it doesn't work longer.
It is better to have loved and lost than never to have loved at all... even if the bitch dumped you for some guy she met in a bar.

Re:nice! (1)

Chris Pimlott (16212) | more than 7 years ago | (#20359283)

You'll notice most of their example images have generous amounts of low-detail "empty space" - sky, water, sand, etc. As they point out, it falls down wrt to higher level semantics, or in high detailed backgrounds. I doubt you would ever get this to a point where you could trust it to "just work" at any size or aspect ratio (at least not without a lot of prep working hinting the important parts of the image, as they show). I see it rather as a useful artist's tool in editing and manipulating images. You need a higher level system monitoring the process that understands when an unacceptable choice has been made, e.g. cutting text in half or squashing someone's face.

Practical uses (2, Funny)

themushroom (197365) | more than 7 years ago | (#20357513)

Finally, a way to reduce the space between surgically augmented breasts and lengthen wangs on Flickr!

Re:Practical uses (1)

AchiIIe (974900) | more than 7 years ago | (#20357723)

> Finally, a way to reduce the space between surgically augmented breasts and lengthen wangs on Flickr!

I see your reduced breasts and raise you a 'Seam Carvied Content-Aware Resized Image' midget porn. Guess who Elizabeth Hurley looks like now.

Re:Practical uses (0)

Anonymous Coward | more than 7 years ago | (#20357737)

Ohh I love trannies

Re:Practical uses (0)

Anonymous Coward | more than 7 years ago | (#20357873)

Too bad this is just automated cropping and scaling. It doesn't actually manipulate the image.

Likely, from the description, it would zoom in on areas with lots of edge changes, like a nipple. Me, personally, would rather see the whole rack than a nipple.

Slightly Strange (3, Interesting)

JamesRose (1062530) | more than 7 years ago | (#20357531)

Okay, I get that they remove the pixels with least energy, so the unimportant information is lost when shrinking, it kinda works, looks a bit strange, but it's okay. however, when they make an image larger they also add the least information so you end up with a large image- but the useful information is the same size and the extra/useless low energy or background gets duplicated- to me, I think thats kinda pointless, I mean, you're adding stuff you've analysed and found NOT to be the focus of the picture. This may work for pictures with no obvious background, but lanscapes like one of the examples, have such an obvious background that only that gets enlarged and just gives you more background. You may aswell just add a nice blue frame round the edge of the picture to make it fit.

Re:Slightly Strange (2, Interesting)

Anonymous Coward | more than 7 years ago | (#20357597)

when they make an image larger they also add the least information so you end up with a large image- but the useful information is the same size and the extra/useless low energy or background gets duplicated- to me,

According to the video, the added background information is actually the averaging of the extra "low energy" information around it. So it's not quite duplicated.

Re:Slightly Strange (1)

Raenex (947668) | more than 7 years ago | (#20357751)

You may aswell just add a nice blue frame round the edge of the picture to make it fit.
The whole point is to avoid artificial techniques that stand out, like frames. A trivial application would be expanding an image to fit as your desktop background. If you're trying to fit a picture into a rectangle with particular dimensions, you may want to both squeeze it one dimension *and* expand it in another to give you the most natural looking effect.

For example, think how a 4:3 TV screen often displays wide-screen movies: They scale (shrink) the image to fit and put bars on the top and bottom. Instead of doing that, they could both push it in on the sides, and if they could only push it so far before losing too much information, fit the rest by scaling and then expanding the top and bottom. That would result in less scaling and no bars.

Or consider the opposite: Some widescreen TVs expand 4:3 media to avoid bars, but they do so by stretching the image horizontally.

Re:Slightly Strange (1)

CastrTroy (595695) | more than 7 years ago | (#20357819)

I just wish windows would support resizing an image for your desktop without changing the aspect ratio. I think they had this in Linux about 6 years ago. And I'm pretty sure it's still not in Windows Vista. Come on, it isn't that hard.

Re:Slightly Strange (1)

Korveck (1145695) | more than 7 years ago | (#20357771)

Isn't this exactly what the method intends to happen? The part with high energy (importance) should remain undistorted when you expand or shrink the image. If you expand the high energy parts, they either become less clear or appear stretched.

Re:Slightly Strange (4, Insightful)

Nutty_Irishman (729030) | more than 7 years ago | (#20357821)

I think you're missing the point of their method, which is to provide realistic images during rescaling that aren't corrupted by blind interpolation (equal averaging). In downscaling the images, it preserves parts of the images that would lose their information through downscaling (e.g. complex textures, people), while at the same time removing textures that would not lose information through downscaling (sky, water, sand). The sky, water and sand will still look like sky water and sand whether it's at 1/4 or 10x resolution, people however look much different if you try and downscale them or upscale them(they would appear blurry and hard to distinguish). The same works in reverse. The sky is still going to look like the sky whether you scale it to 10x or 5x-- it would still look natural. Tree's on the other hand, would not. Once you start to scale up the trees you would expect to be seeing different characteristics-- leaves, branches, etc. Any type of scaling up of a tree would make it seem very blurry and unnatural (lacking leaves, branches, etc.)-- you cannot create an additional information that isn't present in the original image. Therefore, the most natural looking image would be to increase the sky.

It's not perfect of course. I'm guessing that if you had a picture of two people next to each other, one with a solid colored shirt, and the other with a striped colored shirt, that the solid colored shirt guy would get skinner than the striped when shrinking, and the reverse when enlarging. However, it's a neat idea, and I look forward to reading the paper.

Re:Slightly Strange (1)

maxume (22995) | more than 7 years ago | (#20358349)

It's probably worth noting that this technique is exactly not rescaling the image. It alters the content of the image(or produces new similar content!) in order to change the aspect ratio while preserving unique elements of the image. 'Scaling' very much implies that the transformation will be linear.

Re:Slightly Strange (0)

Anonymous Coward | more than 7 years ago | (#20358341)

Seems this could be used to re-process old videos for modern wide format screens.

A picture speaks a thousand words... (3, Insightful)

Aphrika (756248) | more than 7 years ago | (#20357547)

So does this mean you're taking some of those words away?

There are probably a few situations where the 'unimportant' bits of an image are still as relevant as the rest. Sports photos for instance - especially those played on grass - would not give you a true picture (literally) of what's going on in the scene.

This'd be good for reference photos - like the animals at the start of the YouTube video, but applications where precision and distance are required wouldn't benefit. Nice bit of work though and I reckon with some smart scaling embedded too (rather than its 'folding effect'), it'd cater for most image retargetting requirements.

Re:A picture speaks a thousand words... (1)

itzdandy (183397) | more than 7 years ago | (#20357823)

just to clarify, this is good for fitting content into spaces where it would not fit otherwise, not just a method to reduct image size. on a pda phone for instance, one might prefer to see the modified image of a football scene rather than not be able to see it at all for lack of screen real-estate.

Re:A picture speaks a thousand words... (1)

Mal-2 (675116) | more than 7 years ago | (#20357837)

It's the Reader's Digest Abridged Version (Photo)!

There are circumstances where it makes sense to abridge (or retarget) and others where it makes more sense to simply rescale. Since this appears to allow the content provider to choose the method that will be used, the overall effect should be fairly acceptable. For existing content, and future unmarked content, some guesswork will be necessary, and I would imagine this is going to be the hardest part. Computers still can't "see", so getting them to assign the same priorities to pictured objects as we would is going to be a difficult task, though one that is likely to have applications far beyond the mere manipulation of images.

Mal-2

Re:A picture speaks a thousand words... (5, Informative)

Fred Ferrigno (122319) | more than 7 years ago | (#20357923)

It's not removing any more pixels than normal resizing or cropping would, it's just doing it such that the least important ones are removed first. Instead of:

he uic bownfoxjumed verthelaz yelowdog

You get:

Th qik brwn fx jmpd ovr th lzy ylo dog

Which reduces the total size by the same amount, but retains more information than treating every bit of information the same.

Re:A picture speaks a thousand words... (1)

flappinbooger (574405) | more than 7 years ago | (#20358151)

Wow, that's a great analogy. Which, for /., is nice!

lol.

Re:A picture speaks a thousand words... (1)

GraZZ (9716) | more than 7 years ago | (#20358263)

/me wants a car analogy. This is /. afterall.

Re:A picture speaks a thousand words... (1)

Feanturi (99866) | more than 7 years ago | (#20359495)

Ok, so it's like you've got a car, and you find a line through it that doesn't have many intersections. So you take a cutting torch to that spot and slice through the car. And then you, uh... attempt to claim to your insurance company that this somehow happened in a car wash. Some new laser-operated one, yeah that's the ticket. And then you promise yourself to never make a bet while drunk ever ever again.

Re:A picture speaks a thousand words... (5, Interesting)

random735 (102808) | more than 7 years ago | (#20358559)

while this is technically true, you're also rearranging the relative positioning of those pixels. cropping something out doesn't change the relationship of what is left in the photo (though it may remove critical details).

if you have 3 people in a picture and you crop it down to 2, you've erased a person, but you haven't changed who is seated next to whom. if you use this method and the middle person is erased, you make it appear as though the outer two people were in fact seated next to each other when they weren't.

we are used to the idea that a picture can be cropped (mentally considering what might be just outside the frame). We aren't yet used to the concept that the photo has effectively been cut and pasted together to create new relationships between the objects in the photo (though of course photoshop is getting us there).

to continue your analogy, if we take:
the quick brown fox jumped over the lazy dog

and drop letters, we can create:
the cow jumped over the dog

whereas "cropping" might let us say:
the quick brown fox jumped

I think it's clear that one of these is more misleading than the other, though in both cases you're just removing information. (in one case, some of that information happens to be spaces between letters/words)

Re:A picture speaks a thousand words... (2, Insightful)

zippthorne (748122) | more than 7 years ago | (#20358859)

I don't know whether I'm "used to it" or not, but after watching the video, I'm totally ready for more intelligent image resizing that isn't quite scaling. Most of the applications I see this being used in don't really require that the exact photographic position (which really isn't the same as what you'd see if you were there) relationships be maintained anyway.

Hopefully someone will write a GIMP plugin and we can all experiment with it. Also a firefox plugin. Obviously some metadata will eventually need to be included in the the images to delineate faces and whatnot, but web designers can easily handle sloppy painting-over in photoshop type tasks.

Re:A picture speaks a thousand words... (1)

random735 (102808) | more than 7 years ago | (#20359073)

don't get me wrong, this is a very cool technology with plenty of legitimate uses. i just wanted to call out the difference between cropping and what this is doing, in terms of losing image information/context.

given the choice of an image which is cropped, or an image which had this done to it, if i'm trying to use the image to reconstruct the "truth" of a scene, i want the cropped image. it's at least slightly less artificial.

but for practical applications, such as browsing the web on a 320x320 screen, yeah, this is definitely good stuff.

A picture shrinks a thousand words... (0)

Anonymous Coward | more than 7 years ago | (#20359319)

"Which reduces the total size by the same amount, but retains more information than treating every bit of information the same."

So what MPEG does for video, this does for still images?

Re:A picture speaks a thousand words... (2, Insightful)

pclminion (145572) | more than 7 years ago | (#20358999)

There are probably a few situations where the 'unimportant' bits of an image are still as relevant as the rest. Sports photos for instance - especially those played on grass - would not give you a true picture (literally) of what's going on in the scene.

Sorry -- "true picture?" That assumes such a thing can exist in the first place. Take a color-blind viewer for instance. Can he (and I say he because statistically, most color-blind people are male) look at ANY image and say that he is seeing the "true image?" How is his experience any more or less true than the experience YOU have when you look at the image?

Any scaling of an image, by definition, must remove (or insert, if up-scaling) information in an image. Usual scaling techniques insert or remove a constant information density across the image. This means that areas with low information lose just as much fidelity as regions with high information. A better method would have removed more information from the area that is already low in information to begin with, leaving more information in the area where it matters. This is exactly what this new algorithm does.

So it is fairly obvious that this method is superior, from a purely information-theoretic standpoint, to typical scaling algorithms. Are there images where its application might be inappropriate? Yes. Compressing an image of an abstract piece of art might do unforgivable damage to it. There is a simple solution -- do not use this algorithm on such images.

DP Approach (3, Interesting)

xquark (649804) | more than 7 years ago | (#20357589)

This method is quiet interesting, though it falls over in situations where the detail level
or entropy of the background is as great as the foreground. Also the paper doesn't go into
too much details about the dynamic programming approach they used to find the path of least
energy, I guess that aspect of it is patentable. Another thing they could investigate is the
use of diagonal seams instead of just staggered vertical and horizontal seams.

All in all a very interesting read.

Re:DP Approach (0)

Anonymous Coward | more than 7 years ago | (#20357799)

Whats hard about it?
1. Take the top row of pixels, sort columns in ascending order of energy
2. Take first column from the list (could weigh by image position, starting at center)
3. Test the next row, same column, as well as the column left and column right, whichever is lowest, add it to the path
4. repeat step 3 with next row for each row left

This is a brilliant concept, but the basic unoptimized implementation should take all of 45 minutes

Re:DP Approach (1)

imasu (1008081) | more than 7 years ago | (#20358431)

Go for it! I just started my stopwatch.

Re:DP Approach (0)

Anonymous Coward | more than 7 years ago | (#20357853)

May be they used a cost-based path finding algorithm, like A*.
http://en.wikipedia.org/wiki/A* [wikipedia.org]

Re:DP Approach (2, Insightful)

The New Andy (873493) | more than 7 years ago | (#20358403)

I certainly hope it isn't patented, since by just watching the video once (without sound) I was able to to make my own implementation in C in under two hours. I completely agree that it is a cool idea, but I think the reason it is so cool is that the parts they used to build it are all so simple/well known - it is just a really novel combination of ideas that people have already come up with. The idea of a patent (I believe) is so that an inventor won't keep their invention to themselves, so that people can see how it all works and it benefits the public. There aren't any hidden tricks here - the (image-processing) public can easily work out how it is working just by looking at it.

Just in case I haven't been clear - I think that the idea is awesome, novel and brilliant. And I believe that it is possible for something to be awesome, novel and brilliant but also 'obvious'. Just like in maths when they showed you complex numbers, and how they bring some sanity into the system. Once they give you the hint that the square root of a negative number can be defined, then you can go away and easily derive all the cool things like Euler's form and whatnot. Now replace 'the square root of a negative number can be defined' with 'you can crop a jagged column from an image' and you have a pretty good parallel.

Re:DP Approach (2, Informative)

pclminion (145572) | more than 7 years ago | (#20359033)

Also the paper doesn't go into too much details about the dynamic programming approach they used to find the path of least energy, I guess that aspect of it is patentable.

Not so much patentable, as "Easy enough for the reader to implement that it deserves little mention."

Prior art (2, Informative)

SamP2 (1097897) | more than 7 years ago | (#20357625)

The technique was already invented by the Soviets in the '30s:

Before [wikipedia.org]

After [wikipedia.org]

Insignificant person removed.

In Soviet Russia... (0)

Anonymous Coward | more than 7 years ago | (#20358227)

In Soviet Russia,
eraser erases you.

Whao (4, Funny)

Arthur B. (806360) | more than 7 years ago | (#20357629)

Ths s rly gret !

This is pure EVIL! The Devil's Work! (0)

Anonymous Coward | more than 7 years ago | (#20357679)

Verily, no good will come from this!

Begone, Ye image distorters from Hell!

Thou hast spoke the awful truth (0)

Anonymous Coward | more than 7 years ago | (#20357827)

Yea, even the Fat Unemployed Internet Artiste has taken exception [youtube.com] to this devious work of The Prince of Darkness!

Re:Thou hast spoke the awful truth (0, Troll)

Ticklemonster (736987) | more than 7 years ago | (#20358399)

Whoa, that guy is totally wrapped up with self importance. I can't wait until this program is out so I can get all his pictures and mess them up. Out of pure spite, mind you. I really don't see why he's taking it so hard. What a blow hard.

Mod me as a troll, go ahead. I know you want to.

Re:Thou hast spoke the awful truth (0)

Anonymous Coward | more than 7 years ago | (#20358749)

Holly crap. That guy is the worst idiot I've heard in ages. Fucking retarted.

Gimp! (5, Interesting)

larry bagina (561269) | more than 7 years ago | (#20357709)

Although they demonstrated on Windows, a friend of mine is one of their graduate students and was peripherally involved. He said it was originally developed as a GIMP plug in, but moved to a separate Windows app to show off the realtime resizing, etc. Hopefully they'll release the GIMP plugin? More likely Adobe will write them a check and license it to make sure that never happens.

Paranoia! It's not just for Gimps (2, Insightful)

Anonymous Coward | more than 7 years ago | (#20357861)

"More likely Adobe will write them a check and license it to make sure that never happens."

Is that check going to cover the removal of their paper from above and the ACM archives, let alone OUR archives?

I can see the spam now (3, Funny)

MarkovianChained (1143957) | more than 7 years ago | (#20357735)

Shrink the rest of your body, and increase you penis size by up to 20 pixels!

Re:I can see the spam now (1)

tomhudson (43916) | more than 7 years ago | (#20357817)

"Shrink the rest of your body, and increase you penis size by up to 20 pixels!"

Open source alternative via the GIMP:

  1. use the "magic wand" tool to select your "magic wand tool"
  2. "convert selection to path"
  3. "stroke path"
Feel free to experiment by repeatedly stroking with different values ...

Does Anyone Find It Ironic (4, Funny)

szyzyg (7313) | more than 7 years ago | (#20357777)

I find a small irony in the fact that the video is posted on youtube, a site which stretches and squeezes video to fit into a 4:3 aspect ratio

Finally! (1)

Opportunist (166417) | more than 7 years ago | (#20357887)

I TOLD you it's 10 inches! See? SEE???

(Yes, I know, this thread is worthless without pictures)

Not ready for Prime Time (1)

bikerider7 (1085357) | more than 7 years ago | (#20357905)

The scheme relies heavily on edge detection. For the simple images shown in the demo (beach sccenes, open sky, plain color backgrounds etc) edge detection is easy. For more realistic photos with complex backgrounds, the approach breaks down.

Re:Not ready for Prime Time (4, Insightful)

pclminion (145572) | more than 7 years ago | (#20359059)

It has nothing to do with edge detection. The algorithm simply detects paths of minimal gradient which lead from one side of the image to the opposite side. This can be used to produce a "pretty picture" which shows the edges -- but this is merely fallout.

They showed what I thought were several realistic photos with complex backgrounds, and the algorithm did well overall, except on structures where people are closely attuned to exact detail -- such as human faces. If we weren't innately wired to process faces in incredible detail, we wouldn't even notice the distortion.

So it's not perfect. Can you show me something in this world that is? And I don't think there has been any mention of "prime time" application, whatever that means.

Very ugly (1)

Mprx (82435) | more than 7 years ago | (#20358037)

This introduces very obvious artifacts, and looks worse than simply distorting the aspect ratio. At least in that case you can undo the distortion in your head. Here you are throwing away information at a much higher level than just the details of textures. I'm sure it would look even worse on images that weren't carefully chosen for the demo.

My Implementation (5, Interesting)

The New Andy (873493) | more than 7 years ago | (#20358081)

I thought it was pretty cool, so I made my own version after seeing the video. It obviously won't be as awesome as their one, but if you want to play around with it, you can get my C source [ultra-premium.com] and have a play around. It is GPL3.

Forget about resizing! (1)

Vexorian (959249) | more than 7 years ago | (#20358181)

Take a look to the end of the video, this one is an awesome method to remove objects from pics. Very interesting.

removing the intended layout (2, Insightful)

SKiRgE (411560) | more than 7 years ago | (#20358321)

What about artistic photographs? Most photos in that sense are planned to have a certain layout, composition, empty spaces, etc. Say I make a nice panorama shot with a 6:1 aspect ratio. Now my photo that took careful planning is reduced to a 4:3 with all the 'unimportant' spaces removed? Maybe it's just me, but there seem to be lots more instances where this would hurt than help. Journalistic images? Sports photos? Oh, the image can't fit, let's get rid of everything between the 50 and 20 yard lines. There aren't any players standing there. I really only see this being beneficial for web ads. Instead of creating square, vertical, and horizontal versions of the same ad, just make one and let the image be 'resized' accordingly. </rant>

Re:removing the intended layout (0)

Anonymous Coward | more than 7 years ago | (#20358495)

Are you afraid that your images you upload to an online gallery get treated like that?
That's 100% not going to happen. Ever. It's simply not the target field for such an algorithm.

Just because there's an algorithm that turns a color image into a greyscale one doesn't mean any image on the web gets converted to greyscale. Why should it be any different with this algorithm? Sometimes i think people rant just for the heck of it

Better Quality Video (1)

Hennell (1005107) | more than 7 years ago | (#20358355)

Higher quality mov video here [nyud.net] (direct link) if you're interested. (Or you know,if you just want to completely hose their server...)
---
Contronyms: for people who are chuffed by antonyms
---

Ouch: (1)

Bairradino (1142321) | more than 7 years ago | (#20358369)

That eraser... Start using "Lossy" tecnology for sum good...

Youtube is definately the best medium.... (1)

r_jensen11 (598210) | more than 7 years ago | (#20358511)

because Youtube has outstanding video quality. I mean, Youtube does to video what the Playstation does to audio.

Re:Youtube is definately the best medium.... (0)

Anonymous Coward | more than 7 years ago | (#20358935)

Actually interestingly enough the original Playstation is considered by many audiophiles to be one of the best CD players out there and has garnered much attention as a high-end audio device. Not quite sure of the specifics or science but a quick Google search of "Playstation+Audiophiles" should point you in the right direction if you're curious.

image removal (1)

v1 (525388) | more than 7 years ago | (#20358651)

By far the most intersting part of the youtube video was the removal of the two out of five runners on the beach. I realize the removal would probably be a lot more noticeable if the image was higher detail, (what we get to look at on youtube is very low res compared to many practical applications) but it's still pretty stunning how easily and quickly the images could be altered. Very little post-removal editing would be required to selectively and convincingly remove content.

There's no reason why they could not use this same method to insert content. I'd like to see them add a few people on that beach for example. Simply stretching an area by cloning the region could do so much more if you could define what to put into the region besides similar content.

Ariel Shamir (2, Informative)

Schraegstrichpunkt (931443) | more than 7 years ago | (#20359121)

... not to be confused with Adi Shamir [wikipedia.org] (the cryptographer).

some code (3, Interesting)

Arthur B. (806360) | more than 7 years ago | (#20359241)

Too much caffeine in the blog, couldn't sleep... I can't get my hand on the paper but the youtube presentation was extremely clear and I just wrote this C code based on libgd2. Basically it lowers the height of an image by 1 pixel, you can run it multiple time to remove more line.

http://rafb.net/p/jinioy45.html [rafb.net]

(yeah my coding sucks but it produces awesome results and I reversed engineered the algorithm from youtube so please grovel...)

I'll improve it soon to remove an arbitrary number of line, horizontally or vertically
  - no recalculation of gradient, only the gradient near the line needs to be recomputed
  - precomputes a file that store the order of the pixel needing to be removed

I need help with something though, I understand how the algorithm can precompute a file which says in which order pixel should be removed, but I don't see how this can work in *both* direction. Suppose you want to reduce vertically and horizontally at the same time, the horizontal change should completely break the precomputed vertical changes. How would you handle that?

Re:some code (2, Interesting)

Arthur B. (806360) | more than 7 years ago | (#20359317)

here's an example produced by that code (this is Maine)

original
http://img96.imageshack.us/my.php?image=testxq4.jp g [imageshack.us]

somewhat reduced
http://img361.imageshack.us/my.php?image=outew8.pn g [imageshack.us]

very reduced
http://img484.imageshack.us/my.php?image=outas2.pn g [imageshack.us]
Load More Comments
Slashdot Login

Need an Account?

Forgot your password?