Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Researchers Build TCP-Based Spam Detection

samzenpus posted more than 2 years ago | from the beans-without-spam dept.

Security 81

itwbennett writes "In a presentation at the Usenix LISA conference in Boston, researchers from the Naval Academy showed that signal analysis of factors such as timing, packet reordering, congestion and flow control can reveal the work of a spam-spewing botnet. The work 'advanced both the science of spam fighting and ... worked through all the engineering challenges of getting these techniques built into the most popular open-source spam filter,' said MIT computer science research affiliate Steve Bauer, who was not involved with the work. 'So this is both a clever bit of research and genuinely practical contribution to the persistent problem of fighting spam.'"

cancel ×

81 comments

Sorry! There are no comments related to the filter you selected.

first post! (-1)

Anonymous Coward | more than 2 years ago | (#38495420)

omfg first post w00t w00t!!

Why do we keep doing this? (5, Insightful)

damn_registrars (1103043) | more than 2 years ago | (#38495432)

People are looking at the wrong end of the problem with much of their efforts - and this is just another example of that. You cannot solve spam with filtering, detection, or legislative actions. We've seen time and time again that those are just time and money-sucking stopgap measures that ignore the reality of the situation.

We won't see a real solution to the spam epidemic until people acknowledge the simple truth that spam is an economic problem. There is still a lot of money to be made by sending out spam, with very little expense for the spammer. The profit margin is high enough that it is well worth their while to find various ways around filters and any other silly mechanisms we throw at them.

If you want to make an actual difference in the fight against spam, you need to approach the economic motivations behind it. If you stop of the flow of money to the spammers, you will stop the spam as well. Because no matter how much some people may want to believe otherwise, spam isn't sent just to piss you off and ruin your day. Spam is sent out because spammers are paid to do so. If they don't get paid, they won't send spam, it is as simple as that. Any other kind of countermeasure only prolongs the fight and throws more money in the wrong direction.

Re:Why do we keep doing this? (1)

CSMoran (1577071) | more than 2 years ago | (#38495452)

So what exactly do you propose?

Re:Why do we keep doing this? (0)

Anonymous Coward | more than 2 years ago | (#38495496)

I would say educating is a good start. Tell grandma and your 10 year old to stop clicking the spam.

Won't work (2)

sakdoctor (1087155) | more than 2 years ago | (#38495534)

Even if the spam click-though rate is 0.0%, there are still enough suckers born every minute to buy the service of spammers.

Re:Why do we keep doing this? (1)

bhtooefr (649901) | more than 2 years ago | (#38495520)

Here's an idea - recipient's SMTP server refuses e-mails unless they get 0.01 cents with it.

Re:Why do we keep doing this? (1)

robot256 (1635039) | more than 2 years ago | (#38495650)

Then every email comes with a traceable credit account and anonymity goes out the window...It is a good idea, but it will take some interesting creativity to resolve that problem.

Re:Why do we keep doing this? (2)

kelemvor4 (1980226) | more than 2 years ago | (#38495728)

Anonymity is not a feature inherent in e-mail.

Re:Why do we keep doing this? (1)

Anonymous Coward | more than 2 years ago | (#38495874)

Anonymity is not a feature inherent in e-mail.

Yes it is. Anybody who can telnet to port 25 can send anonymous e-mails.

Re:Why do we keep doing this? (1)

kelemvor4 (1980226) | more than 2 years ago | (#38517202)

Just because you know enough about a system to exploit it's weaknesses does not mean that the exploit is a "feature".

Re:Why do we keep doing this? (1)

robot256 (1635039) | more than 2 years ago | (#38496276)

Anonymity, maybe not. But pseudonymity darn well is. Just think what the lawyers would do if every email address on every forum post could be subpoenaed unequivocally back to credit account and a person. It would be a hay-day for not only every vain celebrity crying "slander" but also the Sonys and RIAAs of the world looking for "violation-inciting behavior".

Re:Why do we keep doing this? (0)

Anonymous Coward | more than 2 years ago | (#38495844)

A problem, you claim. But please explain why it is a problem!
I think many mail receivers would not have a problem with disallowing any mail from those that wish to remain anonymous.

Re:Why do we keep doing this? (0)

Anonymous Coward | more than 2 years ago | (#38495692)

That is actually an old idea.
It fails because there is no universally accepted method of making such micropayments.
(there are also those that claim it would kill mailinglists, but I don't see why that would be a problem)

Re:Why do we keep doing this? (1)

realityimpaired (1668397) | more than 2 years ago | (#38495718)

There's also nothing to stop the spammers from forging the credentials of some other organization. Then we'd be hearing about Anonymous sending billions of spam messages, pretending to be BoA...

Re:Why do we keep doing this? (1)

glenn.ramsey (1668759) | more than 2 years ago | (#38495702)

I doubt that would work. I still get spam through the post office. If spammers are willing to spend the money on postage, they'll spend 0.01 cents.

Re:Why do we keep doing this? (1)

mcavic (2007672) | more than 2 years ago | (#38495832)

Yes, but how much spam do you get each day, versus how much paper junk mail? Filtering spam is basically an all-day task, but sorting paper mail takes me 30 seconds. Implementing a per-email charge would be almost impossible to implement well, even if people are willing to pay it. But it would eliminate the problem that sending 1 email costs the same amount of money as sending 1 million emails.

Re:Why do we keep doing this? (0)

Anonymous Coward | more than 2 years ago | (#38496172)

Interesting question you pose. At home, snail mail "spam" runs about 8 to 10 pieces a day for our house. Spam I get in my email runs about 1 a week. Now, at work I have mail admins with filters and whatnot and at home I have Gmail. But the bottom line is that what I get in my inbox is about 1 a week. Contrast that with paper mail running 40 to 50 per week. The real difference is that, in order to be sending paper mail, the companies are at least partially legitimate. Yes, it may be some stupid offer for a free room at a hotel in Reno (redeemable Sunday through Wednesday) - but it would actually get me a hotel room if I took them up on it instead of just throwing it away. They may be lying about their products just like TV ads do - but at least they have a product. Mail fraud is a serious crime, so the junk mail may be junk - but usually isn't a straight up scam like spam email tends to be.

Re:Why do we keep doing this? (1)

shentino (1139071) | more than 2 years ago | (#38498498)

At least with postal spam they have to print and mail it at their own expense.

Electronic spam is even worse since often the sending is slaved to botnets full of hijacked computers and the costs are diverted to unwilling participants.

Re:Why do we keep doing this? (2)

Mister Whirly (964219) | more than 2 years ago | (#38495740)

Yes, paying for all email. I can't see any drawbacks to that solution.

How about instead of elecrtronic mail, we devise a system where people write letters on physical paper and then we deliver those letters to the recipients. We could charge a nominal fee for the delivery, and that should end all "junk mail", right?

Re:Why do we keep doing this? (2)

wkcole (644783) | more than 2 years ago | (#38495842)

Here's an idea - recipient's SMTP server refuses e-mails unless they get 0.01 cents with it.

Don't bother trying to patent that idea. It has been proposed and even tried many times.

One problem with it is simply that there is no reliable mechanism in place to identify the responsible sender of every piece of email. Internet email is not a single system, but rather a loosely confederated mob of independently operated systems that mostly use a common set of protocols. Most email these days is spam, sent mostly by hijacked machines, of which most is rejected easily by most receiving systems. The bulk of spam that makes it to user inboxes is either being sent in ways that are intentionally deceptive and often using stolen resources or is arguably not really spam because it is pursuant to some formally (if ignorantly) accepted agreement to be sent mail. Neither of those is easily addressed by making rules for people to follow. The first set are not going to follow any new rules and the latter are working within the letter of the existing rules.

Re:Why do we keep doing this? (0)

Anonymous Coward | more than 2 years ago | (#38496292)

again the wrong end of the stick. the "cost" you want to incur on a spammer is opportunity cost. add something to the system that slows the spammer down so they can't send as many messages. I'm a fan of proof of work systems combined with reputation information. lets you put very high costs on a spammer (3 min load per message) and very low cost on legit user (3 min cost on initial messages until reply from recipient then 0 cost)..

one of the bigger wins is that it allows you to more easily extract good messages from the email stream. any technique that makes email more useful is a win

Re:Why do we keep doing this? (1)

Mister Liberty (769145) | more than 2 years ago | (#38496016)

they get 0.01 cents

Did you really mean only 1/100 of a cent?

Re:Why do we keep doing this? (1)

bhtooefr (649901) | more than 2 years ago | (#38503626)

Yes, I did, I'm not using Verizon math.

The idea is to keep things cheap for legitimate e-mail senders (e-mail providers could even soak up that cost), but it becomes a noticeable cost once you're sending tens of thousands of e-mails.

Re:Why do we keep doing this? (-1)

Anonymous Coward | more than 2 years ago | (#38495472)

That's assuming of course that the spammers need financial incentive to keep sending spam. Naturally this is an easy mistake to make but have no doubt that there exist internet trolls that can, will, and do send spam simply for the "lulz."

Re:Why do we keep doing this? (1)

Anonymous Coward | more than 2 years ago | (#38495480)

great. so what do you propose? banning advertisements and referral programs? because I think most of us would be 100% behind that

Re:Why do we keep doing this? (1)

marcosdumay (620877) | more than 2 years ago | (#38495556)

Not everybody is rational, even less so with their marketing expenses. Really, most companies don't even know the return of their marketing expenses, thus they can't act rationaly.

Also, filtering is great for reducing the results of spam, including spammer revenue. There is no reason not to do both, educate users and filter spam.

Re:Why do we keep doing this? (1)

damn_registrars (1103043) | more than 2 years ago | (#38495714)

Also, filtering is great for reducing the results of spam, including spammer revenue

Actually, it isn't, for at least two reasons:

  • The people who are willing to invest time and money in filtering aren't likely to click through and buy something based on spam any ways.
  • Total spam volume continues to increase in spite of filtering, which indicates it has not had any meaningful effect on the rewards for the spammer

There is no reason not to do both, educate users and filter spam.

Those are the two least useful tactics you can pursue. You would be better off praying to the flying spaghetti monster for a solution. My proposal is to actually get involved in the financial transactions that keep the spammer in operation; the people who are paying the spammer, the people the spammer is paying, and the other associates who are also getting a cut in on the action.

Unlike filtering, this has already been shown to be effective.

Re:Why do we keep doing this? (0)

Anonymous Coward | more than 2 years ago | (#38495960)

Total spam volume continues to increase in spite of filtering, which indicates it has not had any meaningful effect on the rewards for the spammer

[citation needed]

Counter data: McAfee Quarterly Report 2011Q3 [mcafee.com]

Re:Why do we keep doing this? (1)

MrLizardo (264289) | more than 2 years ago | (#38499338)

Also, filtering is great for reducing the results of spam, including spammer revenue

Actually, it isn't, for at least two reasons:

  • The people who are willing to invest time and money in filtering aren't likely to click through and buy something based on spam any ways.

The people who operate the filters != the end users of the mail system.
End users pay for the cost of operating the filters by seeing advertisements in their webmail or paying for the email service. And yes, this has been working well to prevent the vast majority of spam (something like 99.9% according to my GMail account) from landing in inboxes for 15 years or so at this point.

Re:Why do we keep doing this? (5, Insightful)

Tom (822) | more than 2 years ago | (#38495572)

The economic side has been tackled as well, and it turns out that it is not easier than the technological side. More importantly: It involves politics, and politics move slowly on all problems of the commons (i.e. low impact on many people).

Re:Why do we keep doing this? (1)

damn_registrars (1103043) | more than 2 years ago | (#38495756)

The economic side has been tackled as well, and it turns out that it is not easier than the technological side.

In a way, though, it is. There are actually fewer actions that need to be taken from the economic side than from the technological side; indeed economic actions can have very measurable and lasting effects in a short amount of time while technological actions are generally worthless.

More importantly: It involves politics, and politics move slowly on all problems of the commons

You may have misread me on that matter. Economic solutions are not inherently political, even though politics is inherently tied to economics. However, the companies who are on the financial take in the matter can be influenced without the necessity of legislative action.

Re:Why do we keep doing this? (2)

Tom (822) | more than 2 years ago | (#38501112)

In a way, though, it is. There are actually fewer actions that need to be taken from the economic side than from the technological side; indeed economic actions can have very measurable and lasting effects in a short amount of time while technological actions are generally worthless.

Do you say that as an economist or as a technician? Because I would take a bet that the other side would say the same thing, only in reverse.

You may have misread me on that matter. Economic solutions are not inherently political, even though politics is inherently tied to economics. However, the companies who are on the financial take in the matter can be influenced without the necessity of legislative action.

If it were that simple, someone would have done it by now, don't you think? If it is just that nobody has done it, then why don't you?

Re:Why do we keep doing this? (1)

damn_registrars (1103043) | more than 2 years ago | (#38501572)

If it were that simple, someone would have done it by now, don't you think?

It has been done, it's even been discussed on slashdot before [slashdot.org] . And it is far more effective than filters can ever hope to be.

Re:Why do we keep doing this? (1)

Tom (822) | more than 2 years ago | (#38513472)

It has been done, it's even been discussed on slashdot before. And it is far more effective than filters can ever hope to be.

Then why do I keep getting spam?

Many anti-spam solutions were extremely effective the first time around - until the spammers adapted. I remember when greylisting cut your spam to almost nothing. It seems to have almost no effect these days.

Re:Why do we keep doing this? (1)

shentino (1139071) | more than 2 years ago | (#38498522)

It's a problem that refuses to be solved since cutting off the flow of cash to spammers requires pissing off special interests that have the government in their pockets.

Re:Why do we keep doing this? (1)

Tom (822) | more than 2 years ago | (#38501102)

Spammers don't have a lobby. There is no special interest working for them, it's simply that the problem is so distributed that few people really care about it all that much.

Re:Why do we keep doing this? (1)

shentino (1139071) | more than 2 years ago | (#38502296)

No, but the credit card companies people use to pay for V14GR4 do...

Re:Why do we keep doing this? (2)

wbr1 (2538558) | more than 2 years ago | (#38495600)

For the same reason we have security theater.
For the same reason we have a 'War on Drugs'.

We seem to be blind to the fact (as a society or a government), that you cannot legislate or regulate a cure to a problem. People will always do what seems in their best interest, be it recreationally, economically, or otherwise.

Very little our government does actually address the core issue, it just places band-aids on top of it. This, I think at least partly because a democracy is a system of compromise and once you have compromised the strength of many solutions is sapped by that compromise. This is not to say that it never works, but it is degrading quickly. Creating stop-gap solutions and band-aids helps those in power feel like they have made a difference, and for the most part the willfully uninformed public follows and agrees.

As a case in point take the drug war. It would be unfeasible to say shoot all smugglers and dealers. It is also impossible (in our current society) to say legalize all drugs. Instead we have a multibillion boondoggle of a system to try to stop and regulate illegal drugs, and it has never worked. In addtion, even though it is obvious that it doesn't work, there are those who benefit from it's existence and will push to keep it even if it is a failure.

There will always be those who find it better to game the system than to stay within it.
There will always be a disenfranchised group who feels they have to act differently than the norms for the interest.
There will always be those who feel they have the tight to abuse or prey on others.

Re:Why do we keep doing this? (5, Insightful)

Halo1 (136547) | more than 2 years ago | (#38495634)

The same can be said about pickpocketing, burglary and almost any other kind of crime. As long as technical measures can help with partially or temporarily alleviating the problems without causing disproportional side effects or requiring disproportionately large investments (i.e., not TSA nonsense vs terrorism, but more like door locks vs breaking and entering), I don't see what the problem is with developing and deploying them.

Re:Why do we keep doing this? (1)

t00le (136364) | more than 2 years ago | (#38495662)

All we need is a global white list that allows trusted communication between peers. In the event spam is being sent from a member of the white list all of the email from that party would be flagged as suspect for 24 hours, then change to spam until the issue is rectified.

The problem is the lack of response from certain parts of the world, where I block tcp/udp connections from already. I have no issues with allowing people to communicate freely, but I have no issues with my libido and no need to buy Xanax.

Re:Why do we keep doing this? (1)

Stormthirst (66538) | more than 2 years ago | (#38496168)

And who would run the white list? The government? Which government?

Re:Why do we keep doing this? (0)

Anonymous Coward | more than 2 years ago | (#38499270)

Please reference the authoritative guide to why your anti-spam solution will not work [craphound.com] . Actually, anyone reading this story or any of the comments posted herein should probably read it. HTH.

Re:Why do we keep doing this? (1)

Belial6 (794905) | more than 2 years ago | (#38500360)

Yeah, that list if funny, but it is not something to be taken seriously.

Re:Why do we keep doing this? (0)

Anonymous Coward | more than 2 years ago | (#38496096)

Agreed. Take this a step further and go after the companies funding the spam.

Re:Why do we keep doing this? (0)

Anonymous Coward | more than 2 years ago | (#38496766)

While I love your post, I have to be a critic.

"Spam is sent out because spammers are paid to do so. If they don't get paid, they won't send spam, it is as simple as that."

My Network Security teacher told me something like this(paraphrased, including the numbers): SPAM has a statistical average of less than one percent of one percent of a percent, but with large enough volume, they can still make over $200k/month.

When you have something that is 0.000001% effective and you can still make millions, there is no free market way of stopping it.

Re:Why do we keep doing this? (1)

damn_registrars (1103043) | more than 2 years ago | (#38498532)

When you have something that is 0.000001% effective and you can still make millions, there is no free market way of stopping it.

You're wrong on that. There is a free market way to do it. You can stop spam on the market by getting the businesses who currently do business with spammers to stop. Some of them aren't even aware they are working with spammers because they are working in large volumes and the spammers are small fry. Some of them are two or more degrees away from the spammer and might never make direct contact with them. Nonetheless, if you can interrupt the money flow, you can stop spam.

Re:Why do we keep doing this? (1)

shentino (1139071) | more than 2 years ago | (#38498546)

The biggest problem with spam is that spammers are cheating on expenses and getting away with it.

I guarantee you spam would drop in a hurry if there was some way to make a spammer eat his own IT bills. At present the ones who REALLY pay for spam are gullible boobs who let their computers get hijacked.

Re:Why do we keep doing this? (1)

damn_registrars (1103043) | more than 2 years ago | (#38498658)

At present the ones who REALLY pay for spam are gullible boobs who let their computers get hijacked.

Correction - the boobs pay for about half the costs of spam. You are correct that spammers themselves pay a negligible portion.

However, the rest is paid by every person who accesses the internet, in any way, shape or form. Spam is consuming bandwidth, which costs users money even if their own machines are not propagating it. Spam is also consuming storage space on email servers, even if users never read it. Spam is consuming CPU time when filters are running, and spam is consuming human time to adjust those filters.

Spam is, in a very real way, driving up the cost of using the internet. And those costs are faced by everyone. To make matters worse, no amount of filtering will ever end the problem, or even reduce those costs that I just mentioned because the filters do nothing whatsoever to prevent spam from being sent out.

The only way to stop spam from being sent out is to remove the incentive for sending it out from the spammers. As we all know, spammers send our spam because it is profitable, so interrupting the flow of money to the spammer will remove that incentive, which will result in lower spam volumes. This has already been demonstrated as an effective technique.

Re:Why do we keep doing this? (1, Insightful)

Arrogant-Bastard (141720) | more than 2 years ago | (#38497476)

Actually, you're wrong. The problem is NOT economic. It'd be nice if it was -- because some obvious interdiction paths could be used. But it's not.

The spam problem is behavioral: spammers are sociopaths. That's why there are no ex-spammers: they can no more stop spamming than a pedophile can stop molesting children. They're (pick your terminology) mentally ill, sick, etc.

How do we know this? Because we can observe (and we have observed) that they continue spamming even when there's obviously no profit in it, nor any realistic hope of any profit in the future. They're not all/always doing it for the money.

Now...it's certainly true that some spammers do make a profit; certainly the spammers-for-hire that have adopted the guise of "responsible companies" do very well, well enough to hire skilled propagandists who paint them as professional email service providers -- even though they're just spammers with better suits. But that doesn't change their underlying motivation: doing what spammers do requires someone who's devoid of basic human compassion, remorse, responsibility, empathy -- all the qualities that enable people to relate to one another. And there's no easy/obvious fix for that.

Re:Why do we keep doing this? (2)

damn_registrars (1103043) | more than 2 years ago | (#38498776)

How do we know this? Because we can observe (and we have observed) that they continue spamming even when there's obviously no profit in it, nor any realistic hope of any profit in the future.

That is simply not true. There is plenty of money to be made in spam, and it is the motivating force behind it. The spammers that make the news when they get caught (almost always on other offenses) are especially wealthy relative to their home countries. Furthermore, the total investment for a spammer is minimal; they really just need to be able to talk a good game and get some time on a botnet to be able to make money fast. As we've seen, each time a spammer is thrown in jail or murdered , the spam volume at best remains the same (more often, it increases) because it is profitable.

Your very notion of spammers being inherent sociopaths simply makes no sense. If they just want to aggravate people electronically, they could do it by trolling discussion forums and not have to worry about what side of the law they are on. They are not all mentally ill, they are all just looking to make a buck. And many of them - have you ever looked at the lists on spamhaus? I'm guessing no - are from former second-world or current first-world countries where economic opportunities are scarce.

Re:Why do we keep doing this? (0)

Anonymous Coward | more than 2 years ago | (#38507258)

If that is the case, can we just bore them to death by allowing them to spam to infinity.

Re:Why do we keep doing this? (0)

Anonymous Coward | more than 2 years ago | (#38524056)

As a sociopath who usually associates with blackhats and pedos - I can give you a really quick fix for our issues. Whether it's implementing sexual control over a innocent and helpless creature, creating chaos and headaches all around, or just plain cheating people out of their money, it's a matter of control. The secure feeling that you get when you have some real world power, no matter how little or inconsequential it is - we desperately want to feel secure - and for one reason or another, normal human compassion and love does not provide it, due to emotional scars, or just plain lack thereof. Some love and respect would go a long way.

Re:Why do we keep doing this? (1)

shentino (1139071) | more than 2 years ago | (#38498460)

First, a side note:

Spam is profitable only if you ignore the costs absorbed by people whose computers get hijacked into botnets that send the stuff.

In much the same way that grow ops are cheap when you jump the meter and rip off the electric company.

In both cases the perpetrators get away with securing a windfall because they dump their cost burdens on unwilling participants.

Now for the main point:

How is most spammed product paid for?

Re:Why do we keep doing this? (1)

tlhIngan (30335) | more than 2 years ago | (#38500068)

We won't see a real solution to the spam epidemic until people acknowledge the simple truth that spam is an economic problem. There is still a lot of money to be made by sending out spam, with very little expense for the spammer. The profit margin is high enough that it is well worth their while to find various ways around filters and any other silly mechanisms we throw at them.

If you want to make an actual difference in the fight against spam, you need to approach the economic motivations behind it. If you stop of the flow of money to the spammers, you will stop the spam as well. Because no matter how much some people may want to believe otherwise, spam isn't sent just to piss you off and ruin your day. Spam is sent out because spammers are paid to do so. If they don't get paid, they won't send spam, it is as simple as that. Any other kind of countermeasure only prolongs the fight and throws more money in the wrong direction.

The problem is, you assume the one making the profit is the spammer, which is incorrect.

In the traditional spam model (emails sent for marketing, not distributing viruses to either control your machine or to keylog/proxy for your financial info), there are three entities.

First, you have the victim, you and me, who get their inboxes flooded. Enough said.

Second, you have the spammer. They advertise their "marketing services" - something like $100 for 10,000,000 emails.

Third, is the spammer's customer - the business buying spamming services.

The customer buys the services of the spammer - let's say $100. Spammer makes $100 profit, the customer is out $100 in marketing budget. Spammer then sends the emails. At which point, most of the victims don't even see it as a filter sends it to the bitbucket. Of the few remaining, most just delete it.

And there's zero feedback. The customer may or may not make that $100 marketing back in sales from that marketing campaign.

But to the spammer, it doesn't matter - they got paid ahead of time with no guarantee of results. And if the customer doesn't come back, no big deal - there's a lineup of other businesses needing "marketing services".

And most businesses don't have the ability to see that they spent $100 marketing to get orders that raked in only $10 in profit.

It's just like groupon - you get people to come, but the business never realizes that it's a really an expensive way to get the few new repeat customers.

Re:Why do we keep doing this? (1)

damn_registrars (1103043) | more than 2 years ago | (#38501606)

But to the spammer, it doesn't matter - they got paid ahead of time with no guarantee of results. And if the customer doesn't come back, no big deal - there's a lineup of other businesses needing "marketing services".

You made an error yourself in that statement. The vast majority of spam is not for existing domains, but rather for new ones. You can verify this yourself by looking through old spam; if you look at a spam message from a month ago and look at the spamvertised domain you will find it is not the same spamvertised domain that was listed in today's spam, even though they are selling the same products and using all the same web graphics, code, and template.

Furthermore if you run a WHOIS on domain that was spamvertised this morning you will likely find that domain was registered in the last week or so. While indeed the two domains are likely owned by the same group, moving their operation from one web host to another as they get discovered and shut down, the old spam is of no value, nor are returning customers important.

Which is why the spammers services are important to the owner of the spamvertised website, and why the spammer is getting paid nicely in the process. The only way that people find their way to these sites is via spam, because the sites aren't around long enough to be found in a search engine.

Re:Why do we keep doing this? (0)

Anonymous Coward | more than 2 years ago | (#38507754)

I think spammers are stupid enough to pay for the tools and technology to be able to spam, with the hope/promise that they will make money. I don't believe they actually make any money in the end.

The problem with using this technique (-1)

Anonymous Coward | more than 2 years ago | (#38495576)

I have been blocking spam at the SMTP protocol level for many years, and with good success.
I can usually tell that a spam will be coming long before the sender has said DATA.

However, I have not published the details or the spamblocking smtpd I am using, because I think that the spammers can easily work around this when they know what I am looking for. My spamfilter now works well, but it would cease to do so when the spammers know how it works. (e.g. when I publish the code)

I fear the same thing happens when these researchers publish their details. The botnet programmers can avoid the detection and continue spamming.

Works Great Until... (0)

Anonymous Coward | more than 2 years ago | (#38495592)

It works great, until the next bot net or spam cannon iteration.

This seems to be a losing battle. The amount of processing power used to detect or prevent spam is already very high and these increasingly complex detection schemes are just increasing the required processing cycles at an exponential rate. For the spammer, when one system becomes ineffective, they abandon it and move to the next, staying clean and lean. But, the detection system must continue to hold on to the old detection scheme for a very long time or forever because there's always a few spammer that continue to try the old ways.

Fix email to work like IM... (1)

DraconPern (521756) | more than 2 years ago | (#38495616)

The best way to fight spam is do what IM systems has been doing, by whitelisting. So, 1st email triggers a white list query, and the rest wil be invisible... May be do this on a per ip or per domain basis...

Re:Fix email to work like IM... (1)

mcavic (2007672) | more than 2 years ago | (#38495906)

Accepting email only from addresses or domains on your white list is a decent idea, as long as you check your spam folder every day for legitimate mail from new people. Otherwise, it's cumbersome to have to add someone to your list before they email you. You would need the ability to whitelist a whole domain, though, such as amazon.com, etc... something that address books usually won't let you do.

Skip the ITWorld article (4, Informative)

wkcole (644783) | more than 2 years ago | (#38495632)

I'm sure 'itwbennett' would rather everyone go to his employer's website to read that article, but it is clearly not written (or edited) by anyone who has any basic clues about spam-fighting. Just reading the subtitle makes me cringe for the unfortunate "journalists" lassoed into writing it, as it was clearly done by spam neophytes in a desperate scramble for click-scrounging content. The article is vaguely about a paper presented almost a year ago at LISA '11. There are links to an abstract and the original paper at the LISA '11 site: http://www.usenix.org/events/lisa11/tech/ [usenix.org]

The general space of sniffing out spam by looking at TCP characteristics has been mined for years usefully with Symantec and MailChannels both offering proprietary tools that use such techniques and some open DNSBL's using TCP sniffing to identify sources, but it would be incorrect to believe that any one methodology will ever be a magical silver bullet against spam.

Re:Skip the ITWorld article (0)

Anonymous Coward | more than 2 years ago | (#38496218)

What is "Informative" about this post? If you can't even get basic facts like the fact that the conference occurred two weeks ago (NOT a year ago) how are we to believe the remainder of your post?

Looks like a copy of someone else's work... (1)

DaveGillam (880499) | more than 2 years ago | (#38495786)

This REALLY sounds like a copy of Sendmail Inc.'s Rate Control component, which has been deployed to many sites for the last several years. Rate Control allows the admin to throttle or otherwise block email that breaks various TCP-related thresholds (messages/second, bad recipients/second, connections/second, etc.). Further, recent real world indications show that spammers are sending fewer spams per second from individual IP addresses--they make up the volume by increasing the size of the botnet, and coordinating activity so that not too many bots hit the same relay at the same time. This is why Rate Control added an IP Reputation subcomponent a couple of years ago.

It appears these Navy guys have simply come up with a tool that has already existed for years.

As far as being a solution to spam, I agree that spam is 99% a financial problem. The problem with attacking it as such, is that one tends to also hurt legitimate endeavors. If all the advertising were removed from the Internet, there would not be much of a non-commercial Internet--the advertising tends to keep many things free or very cheap. Also, education is great, but as soon as you teach one person to not fall prey to spam, there's another person born who will fall prey. Thus you need to do many things in concert to fight spam--educate, identify, legislate, prosecute. The closer to the front end we can identify spam, the more cheaply we can block or redirect it. Why redirect it? For prosecution and legislation reasons. If you can identify where the money goes, this evidence become important to justify cutting off the ability of that spammer to get funds--credit bureaus, etc.

Re:Looks like a copy of someone else's work... (2)

MightyMartian (840721) | more than 2 years ago | (#38495814)

Postfix has had throttling for several years now, based on the same basic concepts. I use Postfix with greylisting and to be honest, my Spamassassin and ClamAV filters rarely get hit. Since at least big spam attacks are by bots, and bots are primarily designed to just shove as much through as possible, greylisting alone does a spectacular job of killing them, though sometimes people get pissed when messages take a while to get to them from a recipient the first time.

Re:Looks like a copy of someone else's work... (0)

Anonymous Coward | more than 2 years ago | (#38496088)

Just want to second a vote for greylisting (we also use postfix). A few years ago we started greylisting and we kill thousands to 10s of thousands of spam messages per day before they are even queued (we graph our mail statistics, and I remember how surprised/impressed co-workers were immediately after we set this up.

Unfortunately, recently we have seen a huge rise in spam being sent from yahoo's assets (where greylisting doesn't help). Last year it was hotmail. Seems M$ finally got a handle on their infrastructure but yahoo has a huge way to go.

Re:Looks like a copy of someone else's work... (1)

Smask (665604) | more than 2 years ago | (#38501120)

The flood of spam from Yahoo accounts is because of "porn" or "warez" sites that loads an hidden iframe on Firefox. This iframe opens the user's Yahoo account and spam everyone in the address book. Only Firefox have that problem with hidden iframes and Yahoo mail.

Re:Looks like a copy of someone else's work... (0)

Anonymous Coward | more than 2 years ago | (#38496126)

I use a similar setup and when I first implemented Postgrey a few years ago, it was HIGHLY effective. However, as time has past, it has become increasingly ineffective. Whereas Postgrey alone once blocked ~80% of my spam, Postgrey today only blocks ~10%. The spammers seem to have updated their botnets and what have you so that they now come back and reattempt delivery after the greylisting period. :(

Re:Looks like a copy of someone else's work... (1)

kwark (512736) | more than 2 years ago | (#38497106)

So the first defense of greylisting has been defeated (I'm not seeing this in my logs though). But that still leaves the second advantage gained by it: by the time they get back to your smtpd they hopefully will be blacklisted.

Re:Looks like a copy of someone else's work... (1)

allo (1728082) | more than 2 years ago | (#38503686)

maybe you need to increase the greylist-period. Most bots run at dsl accounts, which means they will get a new ip approximate every 24h. When you require a period of 24h for unknown senders, they will not be able to resend it.

Of course only a possible solution, if you do not need to get your e-mails as soon as possible. But when you need to, you do not want to use greylisting at all.

Please stop (2)

WaffleMonster (969671) | more than 2 years ago | (#38495816)

I've always wondered how seemingly smart people can act so stupidly totally oblivious to the repercussions of their actions.

What happens when a busy computer that would cause it to naturally act in a similiar matter as a botnet zombie sends an email and that message is then flagged as spam?

Spammers are no fools or dinosaurs. They will simply adjust their spamming rate in zombie client below the threshold needed to induce effects needed to trigger the detection scheme.

End result as always is the same:

It won't stop anyone from spamming

It WILL make SMTP based Email even more unreliable than it currently is.

Re:Please stop (3, Interesting)

DamonHD (794830) | more than 2 years ago | (#38495916)

This rather assumes that every MTA will have the same threshold. It is not necessary (or helpful) to have a security monoculture.

A very simple first defence against such rate tuning is to randomly vary thresholds substantially between systems and from time to time.

Rgds

Damon

Time for the copypasta: (0)

Anonymous Coward | more than 2 years ago | (#38495944)

Your post advocates a

(x) technical ( ) legislative ( ) market-based ( ) vigilante

approach to fighting spam. Your idea will not work. Here is why it won't work. (One or more of the following may apply to your particular idea, and it may have other flaws which used to vary from state to state before a bad federal law was passed.)

( ) Spammers can easily use it to harvest email addresses
(x) Mailing lists and other legitimate email uses would be affected
( ) No one will be able to find the guy or collect the money
( ) It is defenseless against brute force attacks
(x) It will stop spam for two weeks and then we'll be stuck with it
( ) Users of email will not put up with it
( ) Microsoft will not put up with it
( ) The police will not put up with it
( ) Requires too much cooperation from spammers
( ) Requires immediate total cooperation from everybody at once
( ) Many email users cannot afford to lose business or alienate potential employers
( ) Spammers don't care about invalid addresses in their lists
( ) Anyone could anonymously destroy anyone else's career or business

Specifically, your plan fails to account for

( ) Laws expressly prohibiting it
( ) Lack of centrally controlling authority for email
( ) Open relays in foreign countries
( ) Ease of searching tiny alphanumeric address space of all email addresses
( ) Asshats
( ) Jurisdictional problems
( ) Unpopularity of weird new taxes
( ) Public reluctance to accept weird new forms of money
( ) Huge existing software investment in SMTP
(x) Susceptibility of protocols other than SMTP to attack
( ) Willingness of users to install OS patches received by email
( ) Armies of worm riddled broadband-connected Windows boxes
(x) Eternal arms race involved in all filtering approaches
(x) Extreme profitability of spam
( ) Joe jobs and/or identity theft
( ) Technically illiterate politicians
( ) Extreme stupidity on the part of people who do business with spammers
( ) Dishonesty on the part of spammers themselves
( ) Bandwidth costs that are unaffected by client filtering
( ) Outlook

and the following philosophical objections may also apply:

(x) Ideas similar to yours are easy to come up with, yet none have ever
been shown practical
( ) Any scheme based on opt-out is unacceptable
( ) SMTP headers should not be the subject of legislation
( ) Blacklists suck
( ) Whitelists suck
( ) We should be able to talk about Viagra without being censored
( ) Countermeasures should not involve wire fraud or credit card fraud
( ) Countermeasures should not involve sabotage of public networks
( ) Countermeasures must work if phased in gradually
( ) Sending email should be free
( ) Why should we have to trust you and your servers?
( ) Incompatiblity with open source or open source licenses
( ) Feel-good measures do nothing to solve the problem
( ) Temporary/one-time email addresses are cumbersome
( ) I don't want the government reading my email
( ) Killing them that way is not slow and painful enough

Furthermore, this is what I think about you:

(x) Sorry dude, but I don't think it would work.
( ) This is a stupid idea, and you're a stupid person for suggesting it.
( ) Nice try, assh0le! I'm going to find out where you live and burn your
house down!

95% accuracy (1)

Nikademus (631739) | more than 2 years ago | (#38496152)

While 95% accuracy at detecting spam may sound like "wow", it's a very low rate. Simply using correctly configured greylisting gives an accuracy in the 99% range. So I doubt this technique really improves anything but it will allow to say 'we did it another way'. Given than more and more spam comes from official mail relays, accuracy will only increase when analysing the body of the mail.

Re:95% accuracy (0)

Anonymous Coward | more than 2 years ago | (#38496674)

That depends on if you are getting a few hundred a week or few million a week...

Grey listing only works to a point you have to have things in the list first...

Re:95% accuracy (0)

Anonymous Coward | more than 2 years ago | (#38501236)

I think you need to read up on what greylisting actually is.

Re:95% accuracy (0)

Anonymous Coward | more than 2 years ago | (#38501632)

While 95% accuracy at detecting spam may sound like "wow"

Giving accuracy as a single number is actually the wrong way to measure it. You need to look at both the rate of false positives and the rate of false negatives. It is often implied that there are 0% false positives, meaning no legitimate mail is ever flagged as spam. If they truly had 0% false positives and 5% false negatives, then it would have been impressive. In reality any nontrivial method will have a nonzero error in both directions. This means you end up having to make a compromise between the rate of false positives and the rate of false negatives. At that point you can no longer describe it with just one number. And it requires a huge corpus of spam and legitimate email to produce a realistic measure of the error rates in each direction. Not everybody wants the same compromise, so even claiming the rate of false positives and rate of false negatives for an optimum threshold is inaccurate, as the optimum threshold is not the same for everybody.

95% accuracy could also be taken to mean 5% false positives and 5% false negatives. That is truly unimpressive. It is also the wrong threshold. False positives are much more expensive than false negatives. 5% false positives is way too high to be usable for anybody. Even 1% false positives is not acceptable, you'd still have to check your spam folder daily to ensure there weren't any legitimate email there.

The best filters combine lots of signals. A signal that in itself could give 5% false positives and 50% false negatives would not be useful on its own. But if you had a hundred independent signals each with that accuracy, then a combination of them is very useful. The hard part then is to find out if they really are independent, and what kind of threshold to use for each of them.

This new method just like any of the others is useless on its own, but it might be a good signal, and it could really be that if you have implemented detection of the signal described in this article then you might actually already have done 1% of the work required to implement a good filter.

This isn't a new technique...and it's inaccurate (1)

Arrogant-Bastard (141720) | more than 2 years ago | (#38496538)

First, we've known for many years that IP-level techniques can deal with a lot of spam. For example, using the Spamhaus "DROP" list in perimeter devices is so incredibly effective that anyone who isn't doing it may summarily be declared incompetent. As another example, perhaps more germane to this paper, see http://use.perl.org/~merlyn/journal/17094 [perl.org] -- which demonstrates how to use passive OS fingerprinting in the BSD pf firewall to throttle traffic from Windows systems. (I presume everyone is well aware that bots are nearly always hosted on Windows systems; my own research indicates that despite inroads by attackers into non-Windows hosts, the probability that any given bot will be found to be on a Windows system is still comfortably above 99.999%.) The technique shown by poster "merlyn" in that example from 2004 can readily be extended and combined with others.

Second, 95% true positive rate is impressive for a single measure, BUT we must also consider the false positive rate, and we have to consider the resource cost necessary to achieve this number. Frankly, doing this inside SpamAssassin is very inefficient -- this is a function that can be handled either in the firewall or in the MTA, or perhaps in a combination of the two. There's really no need to invoke something as heavyweight, slow and complex as SA. (Nor is this desirable: the more complex the anti-spam architecture, the more difficult it is to tune properly and the more susceptible it is to gaming.)

Here's the TL;DR version: if a host passive-OS-fingerprints as Windows then it's suspect. If it does that AND (lacks rDNS OR has generic rDNS) it's a bot.

Re:This isn't a new technique...and it's inaccurat (0)

Anonymous Coward | more than 2 years ago | (#38504116)

What I do is detecting these SMTP and TCP level characteristics of senders in the MTA, and then writing an X- header in the received message with the characteristics found during the transfer.
This header is then evaluated by SpamAssassin to assign spampoints to the received message.
The advantage is that faults in the sender's SMTP implementation do not immediately lead to hard blocks, but can be valued together with other characteristics of the message.
It also means the message ultimately ends up in the spam folder instead of being rejected. Over time, it has become clear to me that no matter how spammy a message may appear to be, it can always be that valuable message that the boss really wanted to receive.

In other news 99% (1)

stabiesoft (733417) | more than 2 years ago | (#38496672)

of all spam comes from dynamic addresses. Their method (95%) is worse than simply rejecting all email from dynamic IP's. I find greylisting dynamics for 36 hours and statics for an hour filters over 99% of spam. If one gets thru, I just blacklist the IP.

Been doing this for years (1)

1s44c (552956) | more than 2 years ago | (#38501412)

I've been doing this for years.

I use p0f to detect connections coming from windows and greylist them. Very little genuine mail comes from windows based mail servers.

I find there is little point greylisting mail from unix machines as very little spam comes from them.

ssshhh... (0)

Anonymous Coward | more than 2 years ago | (#38501482)

So basically to identify a spambot, you have to look at the network traffic?
Hmm, that's interesting. What else would there be to look at?

Check for New Comments
Slashdot Login

Need an Account?

Forgot your password?

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>