Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Researchers Claim "Effectively Perfect" Spam Blocking Discovery

ScuttleMonkey posted more than 4 years ago | from the sounds-evolutionary-not-revolutionary dept.

Spam 353

A team of computer scientists from the International Computer Science Institute in Berkeley, CA are claiming to have found an "effectively perfect" method for blocking spam. The new system deciphers the templates a botnet is using to create spam and then teaches filters what to look for. "The system ... works by exploiting a trick that spammers use to defeat email filters. As spam is churned out, subtle changes are typically incorporated into the messages to confound spam filters. Each message is generated from a template that specifies the message content and how it should be varied. The team reasoned that analyzing such messages could reveal the template that created them. And since the spam template describes the entire range of the emails a bot will send, possessing it might provide a watertight method of blocking spam from that bot."

cancel ×

353 comments

Sorry! There are no comments related to the filter you selected.

Is there the checklist for why this won't succeed? (1, Insightful)

Anonymous Coward | more than 4 years ago | (#30895416)

Is it coming?

Re:Is there the checklist for why this won't succe (-1, Redundant)

Locke2005 (849178) | more than 4 years ago | (#30895496)

Your post advocates a (x) technical ( ) legislative ( ) market-based ( ) vigilante approach to fighting spam. Your idea will not work. Here is why it won't work. (One or more of the following may apply to your particular idea, and it may have other flaws which used to vary from state to state before a bad federal law was passed.) ( ) Spammers can easily use it to harvest email addresses ( ) Mailing lists and other legitimate email uses would be affected ( ) No one will be able to find the guy or collect the money ( ) It is defenseless against brute force attacks (x) It will stop spam for two weeks and then we'll be stuck with it ( ) Users of email will not put up with it ( ) Microsoft will not put up with it ( ) The police will not put up with it ( ) Requires too much cooperation from spammers ( ) Requires immediate total cooperation from everybody at once ( ) Many email users cannot afford to lose business or alienate potential employers ( ) Spammers don't care about invalid addresses in their lists ( ) Anyone could anonymously destroy anyone else's career or business Specifically, your plan fails to account for ( ) Laws expressly prohibiting it ( ) Lack of centrally controlling authority for email ( ) Open relays in foreign countries ( ) Ease of searching tiny alphanumeric address space of all email addresses (x) Asshats ( ) Jurisdictional problems ( ) Unpopularity of weird new taxes ( ) Public reluctance to accept weird new forms of money ( ) Huge existing software investment in SMTP ( ) Susceptibility of protocols other than SMTP to attack ( ) Willingness of users to install OS patches received by email ( ) Armies of worm riddled broadband-connected Windows boxes ( ) Eternal arms race involved in all filtering approaches (x) Extreme profitability of spam ( ) Joe jobs and/or identity theft ( ) Technically illiterate politicians ( ) Extreme stupidity on the part of people who do business with spammers ( ) Dishonesty on the part of spammers themselves ( ) Bandwidth costs that are unaffected by client filtering ( ) Outlook and the following philosophical objections may also apply: ( ) Ideas similar to yours are easy to come up with, yet none have ever been shown practical ( ) Any scheme based on opt-out is unacceptable ( ) SMTP headers should not be the subject of legislation ( ) Blacklists suck ( ) Whitelists suck ( ) We should be able to talk about Viagra without being censored ( ) Countermeasures should not involve wire fraud or credit card fraud ( ) Countermeasures should not involve sabotage of public networks (x) Countermeasures must work if phased in gradually ( ) Sending email should be free ( ) Why should we have to trust you and your servers? ( ) Incompatiblity with open source or open source licenses ( ) Feel-good measures do nothing to solve the problem ( ) Temporary/one-time email addresses are cumbersome ( ) I don't want the government reading my email ( ) Killing them that way is not slow and painful enough Furthermore, this is what I think about you: (x) Sorry dude, but I don't think it would work. ( ) This is a stupid idea, and you're a stupid person for suggesting it. ( ) Nice try, assh0le! I'm going to find out where you live and burn your house down!

Re:Is there the checklist for why this won't succe (2, Funny)

odin84gk (1162545) | more than 4 years ago | (#30895524)

Formatting! Please use some proper formatting! my eyes are bleeding from your wall of text!

Re:Is there the checklist for why this won't succe (1)

amn108 (1231606) | more than 4 years ago | (#30896058)

Can't lay bricks made from shit, you know.

Re:Is there the checklist for why this won't succe (1)

MichaelSmith (789609) | more than 4 years ago | (#30895528)

Spoiled by html fail.

And BTW the spammers are just going to change the way their templates work. Make them more... evolutionary.

Re:Is there the checklist for why this won't succe (1)

sopssa (1498795) | more than 4 years ago | (#30895868)

Exactly. They just make the subtle changes in templates less subtle. They have a reason (money) to get around the blocking, like they already do. This isn't going to be some effectively perfect solution.

Case closed.

Re:Is there the checklist for why this won't succe (4, Insightful)

fuzzyfuzzyfungus (1223518) | more than 4 years ago | (#30895958)

Effectively perfect, no. If nothing else, for certain classes of spam(especially phishing) the money or perception of money can be good enough to keep actual humans at the keyboard.

However, the reason you use templates, rather than word salad or the first 100kb of /dev/urandom, is that you both need to peddle whatever it is you are peddling and look vaguely like a human constructed message. If the researchers can, in fact, target messages that bear signs of being generated from a given template, the spammers will be forced to be looser in generating messages from templates(which increases the risk of garbling beyond comprehension, or being flagged by filters looking for highly non-human output) or step up their game in terms of natural language synthesis.

Re:Is there the checklist for why this won't succe (5, Funny)

dkleinsc (563838) | more than 4 years ago | (#30895516)

Sure, I'll bite:

This group advocates a:
(X) technical ( ) legislative ( ) market-based ( ) vigilante

approach to fighting spam. The idea will not work. Here is why it won't work. (One or more of the following may apply to the particular idea, and it may have other flaws which used to vary from state to state before a bad federal law was passed.)

( ) Spammers can easily use it to harvest email addresses
(X) Mailing lists and other legitimate email uses would be affected
( ) No one will be able to find the guy or collect the money
( ) It is defenseless against brute force attacks
( ) It will stop spam for two weeks and then we'll be stuck with it
(X) Users of email will not put up with it
( ) Microsoft will not put up with it
( ) The police will not put up with it
( ) Requires too much cooperation from spammers
( ) Requires immediate total cooperation from everybody at once
(X) Many email users cannot afford to lose business or alienate potential employers
( ) Spammers don't care about invalid addresses in their lists
( ) Anyone could anonymously destroy anyone else's career or business

Specifically, your plan fails to account for

( ) Laws expressly prohibiting it
( ) Lack of centrally controlling authority for email
( ) Open relays in foreign countries
( ) Ease of searching tiny alphanumeric address space of all email addresses
(X) Asshats
( ) Jurisdictional problems
( ) Unpopularity of weird new taxes
( ) Public reluctance to accept weird new forms of money
( ) Huge existing software investment in SMTP
( ) Susceptibility of protocols other than SMTP to attack
( ) Willingness of users to install OS patches received by email
( ) Armies of worm riddled broadband-connected Windows boxes
(X) Eternal arms race involved in all filtering approaches
(X) Extreme profitability of spam
( ) Joe jobs and/or identity theft
( ) Technically illiterate politicians
( ) Extreme stupidity on the part of people who do business with spammers
( ) Dishonesty on the part of spammers themselves
(X) Bandwidth costs that are unaffected by client filtering
( ) Outlook

and the following philosophical objections may also apply:

(X) Ideas similar to this are easy to come up with, yet none have ever been shown practical
( ) Any scheme based on opt-out is unacceptable
( ) SMTP headers should not be the subject of legislation
( ) Blacklists suck
( ) Whitelists suck
( ) We should be able to talk about Viagra without being censored
( ) Countermeasures should not involve wire fraud or credit card fraud
( ) Countermeasures should not involve sabotage of public networks
( ) Countermeasures must work if phased in gradually
( ) Sending email should be free
(X) Why should we have to trust you and your servers?
( ) Incompatiblity with open source or open source licenses
(X) Feel-good measures do nothing to solve the problem
( ) Temporary/one-time email addresses are cumbersome
( ) I don't want the government reading my email
(X) Killing them that way is not slow and painful enough

Furthermore, this is what I think about them:

(X) Sorry dudes, but I don't think it would work.
( ) This is a stupid idea, and they're a stupid people for suggesting it.
( ) Nice try, assh0les! I'm going to find out where you live and burn your house down!

Re:Is there the checklist for why this won't succe (4, Insightful)

darkvizier (703808) | more than 4 years ago | (#30895726)

Furthermore, bad will always win because good is dumb.

Note that the "good guys" revealed their methods immediately after discovery, which means the "bad guys" can start looking for a workaround. The "bad guys" won't make the same slip.

Re:Is there the checklist for why this won't succe (1)

thhamm (764787) | more than 4 years ago | (#30895880)

because good is dumb.

nah, just lazy. think of the other stuff good is doing. like beer and women. and uhm yeah. stuff like that.

Re:Is there the checklist for why this won't succe (1)

rednip (186217) | more than 4 years ago | (#30895968)

Furthermore, bad will always win because good is dumb.

Nice saying, but if you need an excuse for being bad, you're not doing it right.

Re:Is there the checklist for why this won't succe (4, Interesting)

fuzzyfuzzyfungus (1223518) | more than 4 years ago | (#30896014)

Not in the same level of detail; but, when your business model is spamming, you inevitably end up sending thousands of samples to loads of ill-vetted email addresses, some fraction of which are either being operated as spamtraps, or are in the possession of users annoyed enough to forward samples on.

Your algorithms can, and often do, remain secret(unless one of your black-hat buddies cracks one of your cracked machines); but you'd be a lousy spammer indeed if the results of your technique weren't widely available.

Re:Is there the checklist for why this won't succe (2, Funny)

jeffmeden (135043) | more than 4 years ago | (#30895814)

It seems like "fails to account for (X) Asshats" is *always* the case.

Is it true, that perhaps "no one expects the asshats!"

Re:Is there the checklist for why this won't succe (3, Insightful)

lorenlal (164133) | more than 4 years ago | (#30896022)

Asshatitude always applies because you can never anticipate the next step in asshatitude evolution. They will always find new and innovative ways to be asshats.

Yeah, I don't see the point (2, Interesting)

Anonymous Coward | more than 4 years ago | (#30895824)

I RTFA and they tested it by giving it 1000 spam e-mails by the same bot and after that it recognized the spam sent by that bot with 100% accuracy. This means NOTHING. I could bet a nice sum of money that if you give a traditional, learning spam filter 1000 e-mails sent by the same bot and flag those all as spam, it can then recognize the bot's further e-mails as spam. Real enviroment doesn't work like that, however. You have a large amount of very different spam bots and their templates which is what makes it so difficult. In addition, you have loads of regular mail, some of which might somewhat resemble the spam e-mails but still be completely legitimate. And in real enviroment, some people eventually flag legitimate e-mail as spam but some spam isn't flagged as such.

The fact that their test was so limited implies that this was simply a test. A proof of concept for this kind of approach, one could say. I doubt they actually intended to this be a solution that ends spam.

A never ending battle (1)

mcgrew (92797) | more than 4 years ago | (#30895424)

Hooray for the good guys! Now if they could find something similar to fight viruses.

Re:A never ending battle (1, Funny)

eegad (588763) | more than 4 years ago | (#30895492)

Hooray for the good guys! Now if they could find something similar to fight viruses.

My pattern analysis indicates that if the Windows kernel tried to load it, it's a virus.

Re:A never ending battle (1)

Daniel_Staal (609844) | more than 4 years ago | (#30895684)

Your analysis is faulty: You miss-identified the virus.

Re:A never ending battle (0)

Anonymous Coward | more than 4 years ago | (#30895682)

Easy! Just upload your anti-virus to The Gibson and you'll be fine (I thought everyone knew this?)

Re:A never ending battle (0)

Anonymous Coward | more than 4 years ago | (#30895982)

I'm sure their solution will be just as perfect as this one.

But while contemplating the spam solution, I'm reminded of religious zealots who champion, "an eye for an eye." Perhaps we should force caught spammers to keep their eyes open, not letting them sleep until they have read out loud every spam email they ever sent...the number of times they sent them.

Then we should make them purchase every item they tried to sell, and donate the items to Haiti...or wherever the disaster of the month is. Now, that reminds me of the Nigerians...they should be sent to Haiti to share their ill-gotten gains, and live in cardboard villas.

"Perfect"??? (4, Insightful)

Locke2005 (849178) | more than 4 years ago | (#30895436)

Sure, it will work "perfectly" for about 2 days, until the spammers change their methods to work around it. This is an arms race; there is no "final solution" (although modifying the email protocol to allow authentication of the sender's address would be a big help.)

Re:"Perfect"??? (5, Funny)

NeoSkandranon (515696) | more than 4 years ago | (#30895452)

Oh, there's a final solution alright.

Re:"Perfect"??? (5, Funny)

Anonymous Coward | more than 4 years ago | (#30895488)

I don't know how killing all the Jews will help worldwide spam. Everyone knows all spammers are Nigerian Princes.

Re:"Perfect"??? (5, Funny)

Thelasko (1196535) | more than 4 years ago | (#30895506)

Oh, there's a final solution alright.

Hitler, is that you?

I'm all for stopping Spam, but genocide crosses the line.

Re:"Perfect"??? (1, Funny)

Anonymous Coward | more than 4 years ago | (#30895780)

Yeah, and I don't think all the spammers are Jews anyway.

Re:"Perfect"??? (0)

Anonymous Coward | more than 4 years ago | (#30895822)

I'm all for stopping Spam, but genocide crosses the line.

You'll have to genocide the goys. Spam isn't kosher.

Re:"Perfect"??? (1, Funny)

hoggoth (414195) | more than 4 years ago | (#30896068)

> Hitler, is that you?

Godwin, is that you?

Re:"Perfect"??? (4, Insightful)

Afforess (1310263) | more than 4 years ago | (#30895554)

The final solution is to nuke spammers from orbit. It's the only way to be sure.

Re:"Perfect"??? (0)

Anonymous Coward | more than 4 years ago | (#30895606)

GAME OVER MAN, GAME OVER!

Re:"Perfect"??? (2, Funny)

sconeu (64226) | more than 4 years ago | (#30895980)

Even that isn't guaranteed to work [userfriendly.org] .

Re:"Perfect"??? (0)

Anonymous Coward | more than 4 years ago | (#30895864)

Oh, there's a final solution alright.

So the solution is 42?

Re:"Perfect"??? (5, Funny)

Thelasko (1196535) | more than 4 years ago | (#30895870)

Oh, there's a final solution alright.

Your post advocates a
( ) technical ( ) legislative ( ) market-based (X) vigilante

approach to fighting spam. Your idea will not work. Here is why it won't work. (One or more of the following may apply to your particular idea, and it may have other flaws which used to vary from state to state before a bad federal law was passed.)

( ) Spammers can easily use it to harvest email addresses
( ) Mailing lists and other legitimate email uses would be affected
( ) No one will be able to find the guy or collect the money
( ) It is defenseless against brute force attacks
( ) It will stop spam for two weeks and then we'll be stuck with it
( ) Users of email will not put up with it
( ) Microsoft will not put up with it
(X) The police will not put up with it
( ) Requires too much cooperation from spammers
( ) Requires immediate total cooperation from everybody at once
( ) Many email users cannot afford to lose business or alienate potential employers
( ) Spammers don't care about invalid addresses in their lists
( ) Anyone could anonymously destroy anyone else's career or business

Specifically, your plan fails to account for

(X) Laws expressly prohibiting it
( ) Lack of centrally controlling authority for email
( ) Open relays in foreign countries
( ) Ease of searching tiny alphanumeric address space of all email addresses
( ) Asshats
( ) Jurisdictional problems
( ) Unpopularity of weird new taxes
( ) Public reluctance to accept weird new forms of money
( ) Huge existing software investment in SMTP
( ) Susceptibility of protocols other than SMTP to attack
( ) Willingness of users to install OS patches received by email
( ) Armies of worm riddled broadband-connected Windows boxes
( ) Eternal arms race involved in all filtering approaches
( ) Extreme profitability of spam
( ) Joe jobs and/or identity theft
( ) Technically illiterate politicians
( ) Extreme stupidity on the part of people who do business with spammers
( ) Dishonesty on the part of spammers themselves
( ) Bandwidth costs that are unaffected by client filtering
( ) Outlook

and the following philosophical objections may also apply:

( ) Ideas similar to yours are easy to come up with, yet none have ever been shown practical
( ) Any scheme based on opt-out is unacceptable
( ) SMTP headers should not be the subject of legislation
( ) Blacklists suck
( ) Whitelists suck
( ) We should be able to talk about Viagra without being censored
( ) Countermeasures should not involve wire fraud or credit card fraud
( ) Countermeasures should not involve sabotage of public networks
( ) Countermeasures must work if phased in gradually
( ) Sending email should be free
( ) Why should we have to trust you and your servers?
( ) Incompatiblity with open source or open source licenses
( ) Feel-good measures do nothing to solve the problem
( ) Temporary/one-time email addresses are cumbersome
( ) I don't want the government reading my email
(X) Killing them that way is not slow and painful enough

Furthermore, this is what I think about you:

(X) Sorry dude, but I don't think it would work.
( ) This is a stupid idea, and you're a stupid person for suggesting it.
( ) Nice try, assh0le! I'm going to find out where you live and burn your house down!

Re:"Perfect"??? (1)

StripedCow (776465) | more than 4 years ago | (#30895592)

Spammers will just incorporate this technique into their botnets to test whether sending certain spam will succeed.

Re:"Perfect"??? (4, Insightful)

MBCook (132727) | more than 4 years ago | (#30895662)

Fine with me. Most spam I get is obviously a template, since I get the same one for weeks. This would stop those additional sent copies. The false positive rate on this kind of thing is effectively 0%, so I'm willing to have it be an additional check on my email.

If it can stop a lot of this kind of spam, that's fine with me. Let it be an arms race. If the spammers have to make up new templates every 4 hours, that's going to make things a lot harder.

This isn't a cure for all spam, it's a fantastic filter for one (of the biggest) kinds of spam. Only headline makes it sound like it will solve all spam.

t's turtles all the way down (0)

Anonymous Coward | more than 4 years ago | (#30895736)

I dunno. Are we sure they won't simply realize that if using a template to procedurally generate unique email for spam is ineffective after a short time, they can use a template to procedurally generate templates to generate unique messages.

Re:t's turtles all the way down (2, Interesting)

MBCook (132727) | more than 4 years ago | (#30895770)

The more annoying it is to spam, the fewer people will do it. If writing software to get past this (or buying the software) costs a fortune, good.

Re:"Perfect"??? (3, Insightful)

xZgf6xHx2uhoAj9D (1160707) | more than 4 years ago | (#30895728)

There is a final solution: make sending spam more expensive. Spammers will only spam so long as it's mind-blowingly wealthy. If you can raise their operating costs and bump them down from "mind-blowingly wealthy" to only "obscenely wealthy", they might switch to other lucrative immoral industries like manufacturing printer ink.

What this does is increase the computational power required to generate a spam email. The method they described sounds like it's self-learning (just hook it up to a spambot "oracle" and it'll figure out the new template), so spammers will likely have to abandon the use of templates altogether. If you increase the amount of computational time required to generate spam, you decrease the amount of spam sent and really decrease the profitability of it.

We keep pushing the requirements for spam further and further up the computational totem pole (or Chomsky hierarchy, if you will) and you get closer and closer to a point where spammers are going to have to create strong AI to write spam. If they fail, we don't have spammers anymore and if they win, well we have spam, but we also have strong AI! Win-win, I say.

Re:"Perfect"??? (2, Informative)

Simon (S2) (600188) | more than 4 years ago | (#30896086)

There is a final solution: ...

Your post advocates a
(x) technical ( ) legislative (x) market-based ( ) vigilante

approach to fighting spam. Your idea will not work. Here is why it won't work. (One or more of the following may apply to your particular idea, and it may have other flaws which used to vary from state to state before a bad federal law was passed.)

( ) Spammers can easily use it to harvest email addresses
(x) Mailing lists and other legitimate email uses would be affected
( ) No one will be able to find the guy or collect the money
( ) It is defenseless against brute force attacks
( ) It will stop spam for two weeks and then we'll be stuck with it
(x) Users of email will not put up with it
( ) Microsoft will not put up with it
( ) The police will not put up with it
( ) Requires too much cooperation from spammers
(x) Requires immediate total cooperation from everybody at once
(x) Many email users cannot afford to lose business or alienate potential employers
( ) Spammers don't care about invalid addresses in their lists
(x) Anyone could anonymously destroy anyone else's career or business

Specifically, your plan fails to account for

( ) Laws expressly prohibiting it
(x) Lack of centrally controlling authority for email
( ) Open relays in foreign countries
( ) Ease of searching tiny alphanumeric address space of all email addresses
( ) Asshats
(x) Jurisdictional problems
(x) Unpopularity of weird new taxes
( ) Public reluctance to accept weird new forms of money
( ) Huge existing software investment in SMTP
( ) Susceptibility of protocols other than SMTP to attack
( ) Willingness of users to install OS patches received by email
( ) Armies of worm riddled broadband-connected Windows boxes
( ) Eternal arms race involved in all filtering approaches
(x) Extreme profitability of spam
( ) Joe jobs and/or identity theft
( ) Technically illiterate politicians
( ) Extreme stupidity on the part of people who do business with spammers
( ) Dishonesty on the part of spammers themselves
( ) Bandwidth costs that are unaffected by client filtering
( ) Outlook

and the following philosophical objections may also apply:

(x) Ideas similar to yours are easy to come up with, yet none have ever been shown practical
( ) Any scheme based on opt-out is unacceptable
( ) SMTP headers should not be the subject of legislation
( ) Blacklists suck
( ) Whitelists suck
( ) We should be able to talk about Viagra without being censored
( ) Countermeasures should not involve wire fraud or credit card fraud
( ) Countermeasures should not involve sabotage of public networks
(x) Countermeasures must work if phased in gradually
(x) Sending email should be free
( ) Why should we have to trust you and your servers?
( ) Incompatiblity with open source or open source licenses
( ) Feel-good measures do nothing to solve the problem
( ) Temporary/one-time email addresses are cumbersome
( ) I don't want the government reading my email
( ) Killing them that way is not slow and painful enough

Furthermore, this is what I think about you:

(X) Sorry dude, but I don't think it would work.
( ) This is a stupid idea, and you're a stupid person for suggesting it.
( ) Nice try, assh0le! I'm going to find out where you live and burn your house down!

How many times do I have to tell you (1)

Monkeedude1212 (1560403) | more than 4 years ago | (#30895438)

Unplugging the ethernet cable DOESN'T COUNT.

Re:How many times do I have to tell you (2, Funny)

Dunbal (464142) | more than 4 years ago | (#30896076)

Unplugging the ethernet cable DOESN'T COUNT.

      I'm using my neighbor's WiFi you insensitive clod!

Re:How many times do I have to tell you (1)

jd2112 (1535857) | more than 4 years ago | (#30896082)

You have to disable wireless as well.

effectively perfect? (1)

JNSL (1472357) | more than 4 years ago | (#30895444)

"Effectively perfect" overstates this claim in a big way.

What about changing the templates (1)

jimbolauski (882977) | more than 4 years ago | (#30895448)

So what happens when botnets start adjusting the templates?

Re:What about changing the templates (1)

ddxexex (1664191) | more than 4 years ago | (#30895638)

The filter notices a new template and blocks it. There are only so many ways you can say you're selling cheap viagra. I highly doubt a spammer can find enough ways to say the same thing so that an email in a certain format is less than the blocking threshold (apparently 1000 emails FTA).

Re:What about changing the templates (2, Insightful)

Tablizer (95088) | more than 4 years ago | (#30895692)

Man, building spamming systems and finding ways to vary the content but not the message seems like a fun cat-and-mouse game. Too bad it's so evil. Can I cut off my Guilt Lobe?

Re:What about changing the templates (1)

jeffmeden (135043) | more than 4 years ago | (#30895878)

Why do the spammers have to be on one particular side? It's an arms race, which is more like a game of cat and cat; we both (the good guys and the bad guys) want end users to get just the messages we send. Each will do whatever it takes to get in the others' way. In my experience, it's just as fun (and a lot more gratifying) to stay on the good side.

Re:What about changing the templates (1)

shabtai87 (1715592) | more than 4 years ago | (#30895836)

When botnets start adjusting the templates, they upgrade to skynet. Then we're all screwed!

effectively (2, Insightful)

characterZer0 (138196) | more than 4 years ago | (#30895450)

"effectively" = "not quite good enough to actually work"

Re:effectively (1)

Chris Burke (6130) | more than 4 years ago | (#30895796)

No no...

"Effectively" = "'Perfect' is a very effective word to use in marketing campaigns".

obvious (0)

Anonymous Coward | more than 4 years ago | (#30895456)

If you have the botnet's source, then it's results are obviously predictable.

spam template (5, Funny)

rhainman (952694) | more than 4 years ago | (#30895468)

1. Mash up dubious quality meat. 2. Insert into can.

Seems to make sense (4, Insightful)

Thyamine (531612) | more than 4 years ago | (#30895480)

And since most devices will download updates and things automatically, new templates could be discovered and pushed out as well. I'm sure there will be some work around that the spammers will figure out, but hey, I'm up for most anything that will cut down/stop/prevent spam. I am also still a fan of the 'kill them until they die from it' club when it comes to spammers.

Re:Seems to make sense (1)

Kell Bengal (711123) | more than 4 years ago | (#30895994)

Where can I buy/join one of these clubs? I'll happily do either.

Reactive only (5, Insightful)

oheso (898435) | more than 4 years ago | (#30895482)

So it still needs to see a certain volume of spams in order to figure out the template. Then it reacts to the template. Then when the spammers figure out it's uncovered the template, they change the template. Spam will exist until the fundamental nature of e-mail operation changes.

Re:Reactive only (1)

jimicus (737525) | more than 4 years ago | (#30895934)

In which case the spamming process will change to make it practical to update the template hundreds of times a day.

"Perfect" (4, Insightful)

VorpalRodent (964940) | more than 4 years ago | (#30895490)

You keep using that word. I do not think it means what you think it means.

So... (1)

magsol (1406749) | more than 4 years ago | (#30895498)

The researchers are seeking to infer the hidden distribution of spammers' find-and-replace tactics, rather than simply trashing emails with "pen1s" in the subject.

Correct me if I'm wrong, but haven't hidden markov models been around for decades?

Headline tomorrow (4, Insightful)

Korbeau (913903) | more than 4 years ago | (#30895504)

A team of hackers from Russia are claiming to have found an "effectively perfect" method for countering spam blocking technology. The new system deciphers the templates Spam Blocker is using to filter spam and then teaches spam generators what to write.

Calling BS (3, Insightful)

imunfair (877689) | more than 4 years ago | (#30895512)

I don't believe any spam filter that advertises 100% accuracy, especially one claiming to do it by figuring out the spam email 'templates'

Re:Calling BS (2, Insightful)

pz (113803) | more than 4 years ago | (#30896012)

I don't believe any spam filter that advertises 100% accuracy, especially one claiming to do it by figuring out the spam email 'templates'

Yeah, and calling this a discovery stretches credulity. Who here thinks that Google, Yahoo, Hotmail, and your favorite big mail service provider, don't already do some version of this?
 

Questions (I know, I know...) (4, Interesting)

Penguinisto (415985) | more than 4 years ago | (#30895522)

Err, what if I, as a corporation, blew out a spam that effectively incorporated a template unique to that which my largest competitor uses in their newsletters or customer communiques (or at least close enough to get my competitor blacklisted far and wide)?

(it would take a shedload of doing, but certainly not impossible, and if it could be done, would make for one hell of a cheap and easy DoS).

Heuristics is great and all, but go too deeply, and I can see it opening up a small but pretty scary can of worms.

Spam vaccination (1)

oldhack (1037484) | more than 4 years ago | (#30895534)

Hah, leaky disciplines.

Halting problem (1, Insightful)

Jessta (666101) | more than 4 years ago | (#30895538)

and then the researchers discovered the Halting problem and pretended it didn't exist.

Re:Halting problem (2, Informative)

Kijori (897770) | more than 4 years ago | (#30896036)

and then the researchers discovered the Halting problem and pretended it didn't exist.

I don't quite see your point - the halting problem proves that you cannot create an algorithm that will tell whether an arbitrary program will ever halt. It has no significance for this particular program, since it would be trivial to ensure that it does halt.

Worthless. Completely Worthless (4, Insightful)

damn_registrars (1103043) | more than 4 years ago | (#30895566)

As long as there is money to be made in spam, spammers will continue to send spam. This "discovery" does nothing for that. Indeed it just dedicates more CPU time to trying to identify spam, which is just another way that internet users shoulder the cost of the profitability of spamming.

I've said it before, and I'll continue to say it - spam is an economic problem. Until something is done to address the money that spammers make, they will continue to find ways around these "effectively perfect" "discoveries".

Re:Worthless. Completely Worthless (2, Interesting)

Anonymous Coward | more than 4 years ago | (#30895786)

Spammers send spam because it makes them money. It makes them money because people are stupid. The question is: why are people stupid, and how can we make them smarter? I would argue that spam is an educational problem.

Re:Worthless. Completely Worthless (1)

MobileTatsu-NJG (946591) | more than 4 years ago | (#30895912)

I've said it before, and I'll continue to say it - spam is an economic problem. Until something is done to address the money that spammers make, they will continue to find ways around these "effectively perfect" "discoveries".

There is always a demand to get a message out to n% of x hundred thousand people for cheap. You can't realistically stop that. What you can realistically do is increase the cost of getting those messages out. Treating spam as simply an economic problem won't work.

Re:Worthless. Completely Worthless (1)

damn_registrars (1103043) | more than 4 years ago | (#30896084)

What you can realistically do is increase the cost of getting those messages out.

The proposed "Spam Blocking Discovery" doesn't do jack shit to accomplish that goal. The people who install the spam filters aren't going to buy anything that was spamvertised, anyways. Meanwhile the spammers will continue to adjust their methods to get around the filters that are installed at the ISP level so that they can get their messages out to more people who would be interested.

This craptacular "Discovery" is just another round of whack-a-mole. Hopefully at some point people will finally get tired of this (and realize that they are getting nowhere by doing it) and actually work on the root economic problem, instead of just addressing the symptoms.

Re:Worthless. Completely Worthless (0)

Anonymous Coward | more than 4 years ago | (#30896026)

Bill Gates proposed (though I'm not sure where the idea originated) an escrow service for e-mail. You get to set the amount you're willing to pay me to read your e-mail. I can, at my option, take that money.

If I know you then you do a token $0.01 amount and I don't take it. If you're spam, then I take it and you lose money.

The only real problem I would anticipate is that spammers are in the same camp as those with fraudulent credit cards and the like. They would probably just fund their spam with fraudulent sources. Since it would be an escrow system, people would still receive that money, but it wouldn't actually be costing the spammers anything.

One side benefit of such an escrow service would be to finally open the way for micropayments on the web.

I did this first (4, Funny)

Ambiguous Coward (205751) | more than 4 years ago | (#30895588)

I, too, have designed a flawless spam filter. It works under similar principles, will filter 100% of incoming spam, will generate 0 false positives, and it's super easy to use:

if(is_spam(message)) { delete_message(message); }

Re:I did this first (1)

Jerome H (990344) | more than 4 years ago | (#30895774)

Just wrote one function... One last to go !

Re:I did this first (2, Funny)

Stavr0 (35032) | more than 4 years ago | (#30895986)

Just wrote one function... One last to go !

Me too! I'll send you the delete_message() I just wrote, you send me the is_spam() you wrote and I'll link them and publish the solution.

Again with the stupidity (1)

holophrastic (221104) | more than 4 years ago | (#30895622)

Had there been no spam filters, we'd all receive about the same amount of e-mail spam as we receive in the postal mail world. Instead, the spam industry spends it's time trying to break through spam filters -- and they do so with volume. Upping the ante further just doesn't help. So now you'll encourage spam without templates. My grandmother's just never going to have a chance.

Re:Again with the stupidity (3, Insightful)

jfengel (409917) | more than 4 years ago | (#30895748)

Had there been no spam filters, we'd all receive about the same amount of e-mail spam as we receive in the postal mail world.

I can't imagine what you base that statement on. Real-world junk mail is limited by the fact that it costs money to print and mail junk mail. Neither applies to spam.

Spammers aren't just competing with spam filters. They're also competing with each other for attention. Even in the absence of spam filters, the spammers would continually seek new ways to get more of their spam into your inbox than their competitors.

In fact, they might well invent the spam filter, with a deliberate back door so that their spam sails through while their competitors are dropped.

Re:Again with the stupidity (1)

holophrastic (221104) | more than 4 years ago | (#30896070)

My point was that spam fitlers can't solve spam. All they can do is make spam more sophisticated, and then lose again at an even bigger game.

And spam does cost money to send -- mail servers, developers to get around spam filters, and some actual sending thing, and managing lists, and making things more efficient, and dodging laws.

But mont importantly, spammers get paid, anti-spam doesn't get paid. Therefore, budget vs. no budget, budget wins every time.

Re:Again with the stupidity (1)

Culture20 (968837) | more than 4 years ago | (#30895768)

Had there been no spam filters, we'd all receive about the same amount of e-mail spam as we receive in the postal mail world.

...which I asked my postman to block (most intelligent spam filter ever). Before I asked him to do this, two or three days worth of "bulk rate mail" would be enough to fill my mailbox.

Information Security Puffery (3, Insightful)

dachshund (300733) | more than 4 years ago | (#30895664)

As a researcher in the academic side of the Information Security field, I can't help but notice a significant increase in the level of puffery and misleading promotion of research results. Self-promotion obviously isn't new, it's just that as the amount of newspaper-assisted promotion increases, the level of accuracy has dropped significantly. And more importantly, researchers seem much less apologetic about it. It's generating some real blowback.

The best recent example I can think of is Vanish, a cryptographic system for "destroying" data that was proposed out of University of Washington. It's not just that the system was broken [utexas.edu] a few days after it was presented, it's that this relatively minor result got more press than all of the perfectly legitimate crypto-systems research that was going on at the time. In fact, during the same time period a guy named Craig Gentry solved [techtarget.com] a major open crypto problem --- namely, how to compute on encrypted data --- and it got a fraction of the press coverage.

Not that I'm saying these researchers specifically asked to have their invention described as an "effectively perfect" solution to preventing spam --- which I guarantee you 100% it is not --- but that by going out on a University-encouraged PR junket, they've more or less encouraged this kind of coverage. This kind of stuff is damaging; people should describe their work as what it is. They've developed a technique that is highly effective at filtering /current-gen/ spam generators, in the lab. It won't stop all spam, and it's not effectively perfect, since spamfiltering is by nature an arms race. But of course that's not how it's going to be presented. In the long run this'll just make people more jaded with our field.

Re:Information Security Puffery (3, Funny)

istartedi (132515) | more than 4 years ago | (#30895858)

Don't worry. I'm working on a filter for security puffery. Just wait for my press release. It'll blow you away. Promise.

Uh huh. (2, Funny)

Snarkalicious (1589343) | more than 4 years ago | (#30895694)

Creators recieve chance to increase wang size in 3...2...1...

No conditional modifier for "Perfect" (1)

MetalliQaZ (539913) | more than 4 years ago | (#30895706)

The word "Perfect" neither requires nor allows a conditional modifier. "Effectively Perfect" makes no literal sense.

This makes it unsurprising that their approach seems uninspired. For example, who says the template cannot change? What if their template matches real email notes? What about image spam?

Email and Spam are like global thermonuclear war: the only winning move is not to play. (Spam will only go away when email does)

-d

Re:No conditional modifier for "Perfect" (1)

vlm (69642) | more than 4 years ago | (#30895962)

(Spam will only go away when email does)

Email is going away, but the spam will remain.

Socially, "everyone" uses social networking sites or instant messaging instead of email.

Corporations prefer you log in to their website to look at order status, to better track and market to you.

Email is for .... old people? Services that haven't migrated to something newer?

Usenet still gets spammed, its just very few people use usenet anymore. My email address will get spam for decades after I stop reading email.

Research Finally Catching Up With Spammers? (1)

dawilcox (1409483) | more than 4 years ago | (#30895708)

There's this race between spammers and researchers. It seems from the article that spammers had been ahead of researchers for awhile by figuring out how to modify their emails in such a way that the spam filters wouldn't catch them. The article claims that research has caught up and figured a way to detect this. This spam filter greedily exploits attributes of today's spam, not tomorrow's spam. It seems a bit early to start saying, "Our program that's trained on today's Spam will catch tomorrow's spam!" Doesn't it seem intuitive that the spammers are going to find another way to get their email through this spam filter?

Re:Research Finally Catching Up With Spammers? (0)

Anonymous Coward | more than 4 years ago | (#30895862)

language synthesis is harder than templates, but i'm guessing it would be alot less easier to filter.

The best spam block... (1)

jgreco (1542031) | more than 4 years ago | (#30895712)

The best spam block ... still comes in a can. And thanks to Top Gear and their arctic special for permanently engraving in my memory the image of a shotgun blasting a can of spam.

Um, an economics problem with this "solution"... (2, Interesting)

Primitive Pete (1703346) | more than 4 years ago | (#30895714)

As a former manager and an "email direct-marketing" firm, I should point out that the spammers can increase the amount of complexity/variation in the templates by a wide variety of techniques, including rearranging paragraphs instead of just letters, making parts of the message optional, performing syntactic modifications of the included text,... Each new minor modification starts a research effort on the detecting side. The cost of detecting spam will rise much faster than the cost of generating spam.

If you try to outsmart the spammers with this, you will lose. Complexity favors the spammers.

The real annoyance.. (1)

Roogna (9643) | more than 4 years ago | (#30895732)

Honestly, I have to say between all the various filters I have or have written, I don't get a whole lot of spam. What I -want- though, is a way to identify it more reliably before my mail server even has to accept the message. With the current protocols, you can simply only block so much based on IP ranges or whatnot. There's a point where you have to accept the message to analyze. Sadly the only way we're likely to increase the chance of dropping the connection before receiving the message now is for the protocols themselves to change from the ground up. And as everyone here knows, that's highly unlikely to ever happen.

Ahh well...

100% of x (1)

foldingstock (945985) | more than 4 years ago | (#30895792)

From reading the article, it would appear that this method can successfully block 100% of spam emails that are identified as spam. How many emails will get through while the data is gathered to determine which emails are spam...is inconvenient to this statistic so you should just ignore that for now.

Re:100% of x (1)

mwvdlee (775178) | more than 4 years ago | (#30895966)

It's easy to block 100% of spam emails.
It's harder to let the non-spam emails through.

small task still left to admins .. (1)

HollyMolly-1122 (1480249) | more than 4 years ago | (#30895804)

- To find right spamming botnet, study internals, find "templates" and voila - use it to prevent current spam messages! Or even simplier - to collect several millins of spam messages, analyze them all and find template. Than apply template and voila - problem is solved! Seems like thinking model of Homer Simpson.

Not our claim... :-) (5, Informative)

StefanSavage (454543) | more than 4 years ago | (#30895828)

As a co-author of this work, I should be clear that we never suggested that we have a perfect spam filter per se, simply a new tool that has the benefit of being orthogonal to existing techniques. For _existing_ botnets, our filters are extremely good, but the paper is also quite clear about the variety of ways that spammers might try to evade the approach.

I'll believe it when... (2, Funny)

strangeintp (892348) | more than 4 years ago | (#30895850)

..I don't see it.

Real world operation? Feed of templates? (2, Informative)

renger (1607815) | more than 4 years ago | (#30895876)

How would this work operationally?
  • Some anti-spam operators set up a network of honeypots to collect the spam,
  • analyze it using their new mechanism to divine the templates that are being used, then
  • create a subscription feed to distribute the templates to mail administrators to be used in filtering their incoming mail flow?

Divining the template seems to depend on analyzing numerous messages. Presumably, only very large mail servers (or an aggregated network of smaller servers) would be able to collect enough messages to rapidly divine the various templates. It sounds like a small or medium site could not benefit from operating the analysis software themselves; they would not have sufficient spam volume (from each template) to rapidly divine the template.

Re:Real world operation? Feed of templates? (1)

vlm (69642) | more than 4 years ago | (#30896066)

Presumably, only very large mail servers (or an aggregated network of smaller servers) would be able to collect enough messages to rapidly divine the various templates.

If they don't graylist, and if they insist on putting the spam filtering in between accepting and placing in the mbox/maildir.

If they wait for enough other small sites to aggregate the info, and then spamfilter mbox/maildir instead of spamfiltering the inputs to mbox/maildir...

Counseling (1)

twmcneil (942300) | more than 4 years ago | (#30895932)

I think it would be much more effective as well as cheaper to give free counseling to any one who ashamed of the size of their penis.

No thanks, I'm good.

Solution to spam? Scrap SMTP! (1)

sageres (561626) | more than 4 years ago | (#30895938)

Typically there were many solutions to SPAM that includes:
1. Filtering by keywords and phrases
2. Bayesian (or more complicated AI filtering)
3. Trusted Domains and its flipside: blacklist database
4. Domain keys

All of these are defeated by various means:

1. There are many ways one can say "Viagra", "ViAgRa", "V1@gra" or just "that blue pill women talk about". So word and phrase obfuscation is used to fight the spam.
2. Bayesian filtering is defeated by seeding the database with valid text (For example, how many of you saw The Lord Of the Rings passage on the bottom of your email? This effectively defeats the probability counters, but given enough email training, as long as the size of the given group is magnitudes higher then the size of the sample tested, poisoning technique would be less likely to work. There are some AI developments in filtering technologies to enable to overcome this problem.
3. Blacklists are easily defeated by "botnets" and all other distributed spamming "nets".
4. Domain keys are implemented by "Yahoo!" and "Gmail" and others, creates a net of "trusted" domains and smtp relays, but still defeated when a spammer infiltrates a given domain (as evidenced by recent influx of spam from gmail) and uses its trusted status to send out spam.

So, I am surprised noone has ever talked about scrapping SMTP protocol completely and replace it with something a lot more secure. AMTP (http://amtp.bw.org/) is a good start. My personal opinion is that there is supposed to be several important features present in the new protocols.

I have a 95% perfect solution... (5, Funny)

dccase (56453) | more than 4 years ago | (#30895942)

Since 95% of email is spam, just block it all.

No one will notice the statistically-insignificant 5% false positives.

Chicken - egg problem in action. (0, Redundant)

HollyMolly-1122 (1480249) | more than 4 years ago | (#30896000)

How dummy machine could defay creative thinking ? It's simply impossible. What prevents spammers from time to time sending legimitate looking emails without any advertising inside ? This pioneered "system" than will create UNIVERSAL "template": to kill any emails with wildcard equivalent of: "*" Somebody must learn this system anyway: what is spam and what not. That's dummy. Who was created first - chicken or egg ?

Worst slashdot article ever? (0, Flamebait)

psymastr (684406) | more than 4 years ago | (#30896010)

Probably the worst slashdot article I've ever read.

Recognizing spam is easy, if you see enough (4, Informative)

Animats (122034) | more than 4 years ago | (#30896072)

Spam filtering isn't very hard, if you see the email for a large number of accounts, as Gmail does. The one characteristic that spam must have is that it's sent in bulk. The commonality across receiving email accounts gives it away. The only hard part is recognizing the commonality, which is already working rather well. This is just a new technique for recognizing commonality.

Recognizing spam for a single account is tougher, because you don't get to see the "bulk" property.

Police wont put up with it (0)

Anonymous Coward | more than 4 years ago | (#30896080)

Police wont put up with it??

Despite what many may think, police are human too,
and they get spammed too,
and they are sick to death of it too

Load More Comments
Slashdot Login

Need an Account?

Forgot your password?