×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Smart Spam Filtering For Forums and Blogs?

timothy posted more than 5 years ago | from the world-will-beat-a-path-to-your-door dept.

The Internet 183

phorm writes "While filtering for spam on email and other related mediums seems to be fairly productive, there is a growing issue with spam on forums, message-boards, blogs, and other such sites. In many cases, sites use prevention methods such as captchas or question-answer values to try and restrict input to human-only visitors. However, even with such safeguards — and especially with most forms of captcha being cracked fairly often these days — it seems that spammers are becoming an increasing nuisance in this regard. While searching for plugins or extensions to spamassassin etc I have had little luck finding anything not tied into the email framework. Google searches for PHP-based spam filtering tends to come up with mostly commercial and/or more email-related filters. Does anyone know of a good system for filtering spam in general messages? Preferably such a system would be FOSS, and something with a daemon component (accessible by port or socket) to offer quick response-times."

cancel ×
This is a preview of your comment

No Comment Title Entered

Anonymous Coward 1 minute ago

No Comment Entered

183 comments

Akismet (4, Informative)

seifried (12921) | more than 5 years ago | (#26252239)

Akismet

Second that! (5, Informative)

_merlin (160982) | more than 5 years ago | (#26252349)

Akismet [akismet.com] is the best thing for blog spam prevention ever. I can't believe you've never stumbled across it before. It uses statistical analysis to identify spam, and the more people use it, the better it gets. If everyone used it, the blog spammers would just disappear because their attacks would be completely ineffective.

Re:Second that! (4, Informative)

seifried (12921) | more than 5 years ago | (#26252417)

Add to which it has an API/etc. It really is what you should be using.

Better than Askimet? (3, Interesting)

Jeremiah Cornelius (137) | more than 5 years ago | (#26252529)

Arguably, it is Mollom. Especially if you are using Drupal.

Askimet is 'rotting on th evine' in many ways - including development updates. Mollom is a commercial web service, with a free version for non-profit and small volume sites/users.

The Drupal module is explained here:
http://drupal.org/project/mollom [drupal.org]

The Mollom site:
http://mollom.com/ [mollom.com]

Re:Better than Askimet? (4, Informative)

ceejayoz (567949) | more than 5 years ago | (#26253465)

I seem to get Mollum captchas on every site that uses it. My IP, user agent, etc. are almost completely static. My comments are grammatically correct, never spammy, etc.

If their system hasn't identified me as safe by now, there's something wrong.

In contrast, to my knowledge Akismet has never flagged me. My comments go straight up on blogs using it. On my personal site, I've had maybe 10 false positives out of several thousand caught.

Mollom, IMO, has a long way to go.

Re:Second that! (0)

Anonymous Coward | more than 5 years ago | (#26252471)

Why is there even an article on the front page of /. about this? lame.

Re:Second that! (5, Interesting)

Indefinite, Ephemera (970817) | more than 5 years ago | (#26252499)

The difficulty in evaluating Akismet - I speak not as a user but as someone who ended up apparently blacklisted and having to try their appeals system - is that everyone I see praising it is by definition the kind of person who pays attention to the filter and therefore will train it effectively. Since your average wordpress.com user more likely lets false positives pile up, I'd love to know how effective it is for people who don't wonder how effective it is.

Re:Second that! (4, Informative)

_merlin (160982) | more than 5 years ago | (#26252727)

I've used it for a few years now. In that time, it has caught tens of thousands of spam comments. It has missed about ten spam comments (i.e. allowed them through). It has misidentified two legitimate comments as spam. Yes, I realise I'm keeping an eye on it, and someone who doesn't may not notice that it's causing problems for them. But the stats are pretty good in my case. I'm aware of the allegations of corruption and using it to gag people, but that hasn't affected me yet.

Re:Second that! (0)

Anonymous Coward | more than 5 years ago | (#26253471)

I've used it for almost a year. Caught just shy of 1300 spam messages. About 5 false positives and also around 5 spams let through (all of them in the same week - my guess is that it was something new and it took the filter a few days to catch up).

Re:Akismet (1, Informative)

Anonymous Coward | more than 5 years ago | (#26252683)

Hit Freshmeat for "bayesian" and PHP; that's the statistical method for calculating the probability that a given post is similar to a body of spam examples.

There's a couple of PHP-based ones there, both open source.

Re:Akismet (1, Informative)

Anonymous Coward | more than 5 years ago | (#26253373)

according to the wiki article http://en.wikipedia.org/wiki/Akismet [wikipedia.org] if you say something bad about Matt Mullenweg you will be blacklisted.

I fear being blacklisted hence the anon post :>

Oh, my (-1, Flamebait)

Anonymous Coward | more than 5 years ago | (#26252257)

Did your poor little blog get spammed?

I always thought (1, Informative)

davebarnes (158106) | more than 5 years ago | (#26252263)

Re-Captcha was fairly effective and easy to install and useful.

Re:I always thought (0)

Anonymous Coward | more than 5 years ago | (#26252825)

I always thought Recaptcha was a captcha, not a spam filter. The challenge is to filter spam _without_ bothering the mod nor the legitimate commenter.

D.I.Y. (2, Insightful)

Zsub (1365549) | more than 5 years ago | (#26252281)

Or am I misunderstanding what FOSS really is about?

Re:D.I.Y. (3, Informative)

Korin43 (881732) | more than 5 years ago | (#26252533)

Yes. The point of FOSS is that one person can do it and no one else needs to do it again unless they want to make it better. This guy is looking for a solution, and the solution already exists. He would be wasting his time if he did it himself.

Re:D.I.Y. (2, Insightful)

Trahloc (842734) | more than 5 years ago | (#26252543)

Not everyone is a programmer, some of us assist in less direct ways.

Re:D.I.Y. (1)

Firehed (942385) | more than 5 years ago | (#26253159)

It's not even a matter of programming skills. If you look at why spam never gets through to your Gmail inbox, it's because Google has a database of billions if not trillions of messages to run analysis on. When you have a twelve-digit sample size to work with, matching can be done much more accurately than with a couple hundred messages. It's pretty easy to slap together a system where you manually flag messages as good or bad. Being able to call $akismet->isCommentSpam() and have everything done for you programmatically takes not only a huge amount of additional coding skills, but a huge sample set to be even close to accurate.

So indeed, saying that "some of us assist in less direct ways" couldn't be more accurate with spam filtering. Merely flagging incorrectly-marked messages helps everyone.

Re:D.I.Y. (1)

dubl-u (51156) | more than 5 years ago | (#26252551)

Or am I misunderstanding what FOSS really is about?

If your first instinct is to build it yourself, then yes, you are kind of missing what FOSS is really about. To jointly improve shared solutions, you first have to find the solutions that are already out there.

Re:D.I.Y. (1)

truthsearch (249536) | more than 5 years ago | (#26252597)

For one large site I took an open source Bayesian filter and customized it. This site was large enough to get spam that's only posted there, so a DIY Bayesian filter worked extremely well. They have staff to remove spam and illegal content, so the filter simply aided the staff, who were able to train the filter very quickly.

However, this solution would be useless without enough content and without people properly training the filter. If you get generic spam in a common scenario than a more generic solution might suffice.

the solution is here .. (0)

rs232 (849320) | more than 5 years ago | (#26252305)

"Does anyone know of a good system for filtering spam in general messages?"

Yea, design an email system that is immune to spam and make the ISPs responsible for blocking spam, phishing and such attacks ..

Re:the solution is here .. (3, Funny)

D Ninja (825055) | more than 5 years ago | (#26252337)

Yea, design an email system that is immune to spam and make the ISPs responsible for blocking spam, phishing and such attacks ..

So, I know people sometimes DRTFA. It happens. Life is busy. But, you know, it's always good to RTFS because it has fancy little tidbits of information such as:

While searching for plugins or extensions to spamassassin etc I have had little luck finding anything not tied into the email framework.

Re:the solution is here .. (1)

Firehed (942385) | more than 5 years ago | (#26253229)

It's honestly not that difficult to do - the real problem would be getting people to switch over. Require all incoming messages to have a micropayment attached (even a tenth of a cent). Ignore messages without the payment. If you read the message and don't flag it as spam, the payment is refunded. If you mark it as spam, you keep the money.

It doesn't stop people from sending spam out, but it kills all of the cost-effectiveness. And to preempt the "but businesses have legitimate bulk mailings!" - places sending out messages to 1m people spend a good chunk of money just designing the newsletter for salaries and everything else. $1k to send a message to a million people, 99.9% of whom will not mark the message as spam and thus refund the micropayment makes it a complete non-issue.

Good luck over-riding two of the most-used protocols on the internet to put it in place (not to mention creating an effective system for micropayments), but at least at a conceptual level it would remove the cost-effectiveness of spam so it would go away.

I think the same principle could go a long way for Youtube comments too. They could finally have enough money coming in to pay for bandwidth :)

Re:the solution is here .. (0)

Anonymous Coward | more than 5 years ago | (#26253487)

Except that since botnets are responsible for a tremendous proportion of spam, the spammers aren't going to be the ones paying anyway.

Re:the solution is here .. (3, Insightful)

AnyoneEB (574727) | more than 5 years ago | (#26253565)

This is not exactly a new proposal, and it has been shot down on Slashdot before. One major problem is that a lot of spam is through botnets and the spammers would not get charged the e-mail fees, people with zombied computers would. I suppose this would make people with zombied computers notice, but why would they agree to sign up for such a service in the first place? Also, tying e-mail to payment means that the payment is probably traceable to a real person, which a lot of people do not want.

these might be overkill ... (0, Offtopic)

yossarianuk (1402187) | more than 5 years ago | (#26252315)

2 apps that come to mind - providing you are not running on shared hosting They may to overkill - they both have spam rules and can help web apps from hackers. modsecurity - http://www.modsecurity.org/ [modsecurity.org] snort - www.snort.org both are free and GPL - the snort does have a paid for subscription service - however if you just free register you get pretty up to date rules

Service.. (1)

bigattichouse (527527) | more than 5 years ago | (#26252327)

I've been thinking about modifying my VSDB software [bigattichouse.com] to do something like this...

Various ideas (1)

mlts (1038732) | more than 5 years ago | (#26252333)

There are a number of things you can try:

For a small site I helped set up, they went to complete SSL and client certificates, where users had to obtain a cert from Verisign or Comodo before they would get access. This stopped spam, and one can obtain a client cert for free or a low cost. However, this can't be done for most forums or blogs.

For larger sites, a lot have ended up moving to an approval type of system where a human approves the creation of the user, then a limit on how many posts a first time user could do, and how many features the user can access.

Finally, one site just went to a paid subscriber system where for any access at all, people had to pay $5 to $10 via PayPal. This at least forced spammers to pony up cash (or commit credit card fraud) before they would get access.

To filter posts like this? (-1)

Anonymous Coward | more than 5 years ago | (#26252365)

Niggers!

mollom not so free (2, Interesting)

jeffstar (134407) | more than 5 years ago | (#26252391)

mollom [mollom.com]

i discovered this one through drupal. I thought it was completely free but apparently for high traffic sites it isn't.

I think all your user generated content is sent to them and checked for spaminess against the other submissions they are receiving and they give you back a rating.

Re:mollom not so free (1)

yelvington (8169) | more than 5 years ago | (#26252601)

I use Mollom because of its excellent integration with Drupal. It's free for up to 100 legitimate posts per day, 30 euros/month after that.

It works very well for stopping spambots without annoying real humans (which a plain captcha will do).

Human spammers still slip through, but when you delete their work, it's fed back to the Mollom database, protecting you and others from repeats.

DIY or it will be broken (5, Interesting)

loony (37622) | more than 5 years ago | (#26252407)

Any method you use can be broken. Your only chance is to reduce the likelihood that your site is worth the effort.

Basically, if you use a common solution - no matter of FOSS or commercial - then there will be a thousand other sites that use it too. This attracts attackers because they know when they hack it once, they can re-use it.

However, if you handcode something, no matter how primitive, it likely lasts a lot longer because nobody bothers hacking into your site...

Of course that doesn't work if you have a large site like myspace - there, a single site is worth the effort by itself.

Anyway - then there are two things - a really fast moving animated gif and silly things where you ask people to identify items usually work.
I help out with a site that randomly takes five pictures of cats and dogs and it asks you to identify which of the images contains the highest number of kittens... We barely ever get spam through - and that with almost 20K attempted submissions by non-humans a day makes us pretty happy

Peter.

Re:DIY or it will be broken (1)

Sir_Lewk (967686) | more than 5 years ago | (#26252757)

Sounds like security through obscurity too me, if someone actually tries to target *your site* (which will happen if it's popular enough) then chances are it'll be broken in no time.

Re:DIY or it will be broken (4, Interesting)

dattaway (3088) | more than 5 years ago | (#26252761)

However, if you handcode something, no matter how primitive, it likely lasts a lot longer because nobody bothers hacking into your site...

Simply renaming the .php files worked 100% for me.

Re:DIY or it will be broken (0)

Anonymous Coward | more than 5 years ago | (#26253821)

What do you mean?

Re:DIY or it will be broken (1)

boyter (964910) | more than 5 years ago | (#26252921)

Thats what I did. I added an extra field which you need to enter the word "fatty" into. I went from 200 spam comments a day to 0.

Re:DIY or it will be broken (1)

martin-boundary (547041) | more than 5 years ago | (#26253139)

Any method you use can be broken. Your only chance is to reduce the likelihood that your site is worth the effort.

That's only one approach. The other approach is to increase the response time dramatically: once you've been spammed, if you can clean up very quickly and reconfigure to prevent similar attacks in future, then visitors are unlikely to notice anything. This is the key advantage of statistical spam filters, as they make it relatively painless to respond and reconfigure to handle whole classes of messages, unlike the handcoding approach which is much more work intensive.

(does not apply to extreme high volume sites, but if you've got one of those and are looking to slashdot for a solution, then you've got bigger problems).

Re:DIY or it will be broken (0)

Anonymous Coward | more than 5 years ago | (#26253735)

I help out with a site that randomly takes five pictures of cats and dogs and it asks you to identify which of the images contains the highest number of kittens... We barely ever get spam through - and that with almost 20K attempted submissions by non-humans a day makes us pretty happy

That would mean a 20% chance of guessing the answer? Immunity by diversity won't really work because most modern spammers quickly make templates for individual sites. I have seen spambots barf out code in the comments occasionally and the code seem site-specific, as if generated by recording a human doing it first. Is answering a multiple-choice question really enough?

-swedish coward

Re:DIY or it will be broken (0)

Anonymous Coward | more than 5 years ago | (#26253739)

Amen to this. I run a wiki site that was being bombarded with huge amounts of link-spam and other garbage. I solved the problem by making just a few very minor modifications to the wiki software itself and the spam problem disappeared entirely (even though the server logs show that the site is still under constant, unsuccessful, attack). And users can still use the site without captchas or other security measures.

It wouldn't be hard for someone who was dedicated to break the scheme I came up with, but it hasn't happened in the last two years. And no, I'm not telling anyone what I changed in the software. Sorry.

Re:DIY or it will be broken (2, Interesting)

edmazur (958154) | more than 5 years ago | (#26254225)

I'll second this.

My friend runs a smaller site and was having a problem with forum spam. He edited the registration page to include a checkbox that said something along the lines of "check this box if you are not a bot". His problems went away instantly. Obviously this does not scale well, but for smaller sites being targeted randomly by automatic spam crawlers, it appears to be very effective.

require a bit of human intuition (1)

cpearson (809811) | more than 5 years ago | (#26252423)

Forums and blogs are susceptible to post spam because most are large opensource or commercial scripts. The fact that there are thousands identical scripts makes them a prime target to spammers. The trick to prevent forum spam is to confuse the spam bots. Most all bots will fail to register a user if given something unexpected. For forums I administer, I wrote a vbulletin mod that requires a bit of human intuition to solve (not much mind you) such as have a text input field that must remain blank. This simple measure has prevented nearly 99% of forum post spam.

4 Tests Stopped 30,000 Comments For Me (5, Interesting)

WebmasterNeal (1163683) | more than 5 years ago | (#26252443)

I have a series of 4 tests to block spam on my website. So far it has stopped over 30,000 attempts in the last year.

Test one is, does the last name = the first name. For some reason almost all spammers do this.

Second, do they use a keyword from a list of about 15 words.

Third, do they fill out a hidden inputbox? This is sort of the reverse captcha.

Finally do they use more than 4 "http" in a post. Almost all comment spam is an SEO effort to increase their pagerank.

Hidden Input Box (5, Informative)

waldoj (8229) | more than 5 years ago | (#26252557)

Third, do they fill out a hidden inputbox? This is sort of the reverse captcha.

This is really a very good test. As others have mentioned in this thread, it's the sort of thing that spammers will circumvent if it becomes widespread, but for now it's great.

There's something else I've found to be really quite effective: deliberately misnaming my form fields. For instance, give the input field that's labelled "First Name" an input name of "phone number." Humans don't use input names to determine what text to enter, but spambots do. Then check that inputâ"if the first name field contains a phone number, you know you've got yourself spammer.

I've used solely the combination of these two things to run one of my websites for two years now, and I get a vanishingly small amount of spam.

Bad Idea (4, Insightful)

erlehmann (1045500) | more than 5 years ago | (#26253325)

As someone who once used text browsers, I can only advise everyone not to do this - it breaks accessibility at a fundamental level: I got banned from a forum once because they mislabeled fields.

What however, works really great for comment spam is a simple question like "What is the name of Barack Obama ?".

My 3 tests also work (5, Interesting)

lalena (1221394) | more than 5 years ago | (#26252787)

I have implemented something similar, but I haven't been checking the number of blocked messages. All I know is that I used to get spam, and now I haven't gotten any for years. I use this for Formus and the Contact Us page.

My rules are:
1) The text boxes for things like name and subject are actually called junk.
2) There are hidden textboxes called name and subject (1 hidden by javascript and one by CSS) that if they are populated the post is ignored.
3) A third hidden field is the result of a simple javascript math equation that is checked on the server side. If the value is wrong, the post is thrown out.

As others have said, if your site is small these types of things are good enough to prevent spam because the spammers won't bother to figure it out. These concepts would never work for any of the larger sites or 3rd party forum software.

Re:My 3 tests also work (4, Informative)

lalena (1221394) | more than 5 years ago | (#26252809)

As a follow up to myself, I didn't come up with these ideas on my own. I read them on Slashdot a couple of years ago.

Re:4 Tests Stopped 30,000 Comments For Me (2, Insightful)

WebmasterNeal (1163683) | more than 5 years ago | (#26253165)

Oh I also forgot, if you have a static URL that your form posts to, it is a good idea to rename that page every now and then, especially if it is getting a tremendous amount of spam. Also you can do a check to see if the referring URL is on your own domain as a lot of spammers are posting from a copied version of your form.

Re:4 Tests Stopped 30,000 Comments For Me (5, Interesting)

Magic5Ball (188725) | more than 5 years ago | (#26253579)

Background: One of my sites is a custom job which kills a spam comment every 3 seconds or so, and has done so consistently for the past four years.

OP's suggestions are very good, especially limiting the number of 'http's. We've given up on the keyword lists since they are costly to maintain and aren't as effective as some other methods.

Currently, the most effective kill rules for us are:
1) We write the client's IP address, the ID of the thing being commented on, and random stuff to a cookie from the legitimate page from which the client clicked the "post reply" link. If the IP address doesn't match, or if the ID missing, or if the parameter for the random junk aren't in the cookie, then fail. This rule traps non-browser scripts and limits spam throughput, but does not affect humans.

2) The client's IP address is a hidden form variable. If that IP address does not match the IP from which the POST originates, fail. This rule traps the browser-based scripts, and operators who proxy through botnets for testing.

These two rules catch all but about two spam-like messages a month (spam operator not using proxies to test their scripts), and have mislabeled two legitimate messages (from a local ISP's poorly-configured proxy) in the last three years.

There are other things at play, such as salted hashes of the above, and some other heuristics on hidden and unused fields which sort and categorise the spam for our own research (including point of origin, topic, etc.). One finding is that IP/geographic blacklists are ineffective. I'll post new findings and methods in another two years.

I'm also evil in that the apparent failure modes are non-deterministic, and include such things as random HTTP response codes, random modes of connection failure, and spam messages that apparently go through, but are only visible for the IP that posted them, or for one minute after they are posted.

Your move, "RosarioRush".

HTTPBL (2, Interesting)

Anonymous Coward | more than 5 years ago | (#26252445)

Project Honeypot's HTTPBL has been good to me:

See: www.projecthoneypot.org/httpbl.php

G to the oats! E? (-1, Troll)

Anonymous Coward | more than 5 years ago | (#26252467)

I defeated your filter. AGAIN!

Fast way is to slow it down (3, Interesting)

Todd Knarr (15451) | more than 5 years ago | (#26252501)

The fastest way is probably to just slow down user registration. Permit anonymous posting, but make it moderated/screened by default (ie. not visible to other users until the forum owner flags it as OK). When a user goes to register (so they can get their posts visible immediately), do not send them the confirmation e-mail immediately. Batch your confirmations up and send them out twice a day at odd times (ie. not midnight and noon, something like 3:47am and 3:47 pm) (you could do it 4 times a day, but not much faster than that since the idea's to introduce a delay in the registration process). Make sure to tell the user on the registration screen what sort of time-frame they can expect their confirmation to arrive in. Ordinary users who plan on using the forum long-term won't be inconvenienced much by this. Spammers... won't tolerate the delay, they want to get their message in fast and get out. With their automated scripts they might not even notice things are failing. Also, don't include a direct confirmation link in the e-mail. Include a URL to a form and make the user copy-and-paste the confirmation number from the e-mail. That'll be trivial for humans, but not easy for an automated script to handle without human assistance.

None of that will stop a determined spammer, but most of them are more interested in volume than anything else and they won't bother spending time/effort on just one forum when they could hit 10 others instead.

Re:Fast way is to slow it down (0)

Anonymous Coward | more than 5 years ago | (#26253411)

As someone who works in Internet marketing I would like to offer free advice to everyone and recommend against everything the parent said.

The main idea behind boosting the conversion rate (how large percentage of people who come to the site register, read articles, post comments, buy products, do whatever the site's owner wants them to do in order to make the site working) is to make things as simple and quick to the user as possible.

Ideal situation is always that user will not have to register at all. Need to register drops conversion rates dramatically. For every fields of info you need to enter in order to register the conversation rates drop a lot again. E mail confirmation drops them by a bit but not much any more at that point. But the suggested "send confirmation emails twice per day at odd times" is just horrible idea.

I've met some websites that use that. I think. I DM DnD as a hobby and googled for guides to make better maps. I hit cartographers' guild's forums. There were nice tutorials but before trying them I wanted to see what end result looked like. The forum software didn't let me see the images (uploaded to forums) before registering. I nearly left but decided to register instead. But the confirmation mail didn't arrive. I decided not to wait for hours, googled five minutes more, found another great tutorial site (a lot of free video tutorials) that didn't require me to register, bookmarked it and am still an active user.

The question is then "But do we want to have users who don't have interest to even register?" and my experience is yes. Wanting to comment on something shows that the user is already promising lead. If he can do that easily, he is more likely to check back to see responses to that and stay as active visitor. Even if he didn't his comment is now extra material for other visitors to look upon. Unless we assume "Registered users give automatically higher quality comments" and again based on what I've seen, there is nothing to imply that.

Honestly, don't make things harder, slower and more annoying for your visitors just to fight spambots when there are other ways too. You'll only hurt your site.

Re:Fast way is to slow it down (2, Insightful)

Todd Knarr (15451) | more than 5 years ago | (#26254047)

Internet marketing brought us forum spam in the first place, I'm afraid.

For most forums it's not about getting the most users at any cost. It's about getting the most interested visitors without scaring an unacceptable fraction away, while at the same time keeping the number of spammers at a manageable level (which, given their proclivities, is pretty close to zero). And the simple fact is that, if it can be automated, spammers can and will automate it. And as long as it costs them little or no time or effort, they'll continue to flood the forum. Getting around filters can be automated.

I'm reminded of an exchange:
Merchant: I need some way to keep people from stealing my merchandise all the time!
Consultant: <looks over the racks of high-value jewelry sitting outside the store> Well, why don't you move your merchandise inside, instead of leaving it out along the sidewalk?
Merchant: I can't do that! People couldn't see my merchandise, and I might lose customers!
Consultant: ... so why exactly did you hire me again?

"I am a robot" field (4, Informative)

casualsax3 (875131) | more than 5 years ago | (#26252511)

The ZSNES boards employ a neat trick: http://board.zsnes.com/phpBB2/profile.php?mode=register&agreed=true [zsnes.com]

It's got a field that says "I am a robot" checked off by default. A human should obviously see that and uncheck it. Those registrations that come in with it checked are blackholed. It's definitely cut down on the SPAM accounts since they enabled it.

Re:"I am a robot" field (4, Funny)

slimjim8094 (941042) | more than 5 years ago | (#26252633)

Great idea, and it has the side-effect of keeping idiots out too :)

Re:"I am a robot" field (5, Funny)

Anonymous Coward | more than 5 years ago | (#26253475)

And the robots. Here I am, brain the size of a planet, and I keep getting banned from forums. *sigh*

Re:"I am a robot" field (0)

Anonymous Coward | more than 5 years ago | (#26252637)

You have just checked the following...

I have a cat in a bag:

[ ] keep it in
[X] let it out

Re:"I am a robot" field (0)

Anonymous Coward | more than 5 years ago | (#26252721)

That's all good for robots and all, and there are a number of alternatives for such a thing, but our problem is real people registering on our PHPbb forum and spamming it. There is no good option like Akismet for PHPbb (though I believe there was an attempt at a plugin, I don't think it worked so well). So while Wordpress and Drupal are covered, PHPbb is not, which is a shame since its a very popular forum platform.

Re:"I am a robot" field (1)

noidentity (188756) | more than 5 years ago | (#26253917)

It's got a field that says "I am a robot" checked off by default. A human should obviously see that and uncheck it.

What have you got against sentient robots taking part in discussion (you insensitive clod)? Or are we...er...they forced to lie about their true identity?

easy way to do it (1)

indy_Muad'Dib (869913) | more than 5 years ago | (#26252513)

not so easy for you of course make the forum moderated and require post approval. then get good mods to approve posts.

Probably not doable now (1)

Deleriux (709637) | more than 5 years ago | (#26252523)

But some PHP tinkering and you could probably do something to pass comments through spamassassin using a socket or something.

Spamassassin would need updating though to work with content-only data.

Wonder if anyones ever thought of this before?

If you're willing to pay... (1)

Lord Byron II (671689) | more than 5 years ago | (#26252539)

...there are companies out there that use a Bayesian filter to sort posts into low scoring and high scoring, and then they have their employees manually sort through the high scoring messages.

Message board spam. (4, Informative)

JWSmythe (446288) | more than 5 years ago | (#26252573)

    I had a similar problem in the comments area of my site. It was all fun and games, until one day I checked, and there were something like 1000 spams for every real message.

    I wrote my own system to deal with it. It's not very hard, assuming you know how your site works (of course you do, right?)

    I ended up making two blacklists. One was for words and phrases. The spammers tend to post (and repost, and repost) the same crap. My blacklist rules had some simple regular expressions that I could run queries with. Like, "%http://%spamsite%" and "%v%gra%". You get the idea. The second list was IP's that were known spammers.

    At the time, I allowed both anonymous comments, and comments from logged in users. I eventually did away with the anonymous comments, as they were a headache. This was the best cure.

    So, when my script ran (once a minute), if it matched a message, it would delete the message, and append the IP to the IP blacklist. If it was posted by a user account, the user account got suspended, so they could no longer log in, nor post.

    After it's detection and cleanup run, it then ran back over the IP list, and pruned out every post by that IP. Sometimes they'll do practice runs saying silly things like "nice site". I thought they were real user complements at first, until I saw the same posting verbatim coming from the same IP to multiple news stories, and then that IP would start spamming later.

    Some people will argue that the IP cleanup run was not nice, polite, or even fair. People use proxies. Sure, they do. We got a lot of abuse from anonymous proxies, and no real messages from them. The spammers didn't seem to like to use AOL.

    When I implemented this, I posted a very brief description of what I was starting ("We're starting advanced anti-spam protection"), with an apology for real messages that were deleted. I never received one complaint about real comments disappearing.

    How brutally you do it is really up to you. I built my method by manually doing it for a while, and then letting the script do it on it's own. Occasionally, I would have to go in and add new words and/or site names to the words blacklist.

    I noticed the spammers hit more common software more often. It's worth it for them to make automated systems to abuse a piece of software that's deployed on tens of thousands of sites. When I rewrote my site from scratch, then abuses dropped down to 0 for a long time. Now, they manually submit "news" items which are just ads for their own sites. It appears to be manual, and since we won't run them as news stories (our editorial staff decides what does or doesn't show up as news, and if it needs to be edited first), they give up pretty quickly.

honeypot (1)

OrangeTide (124937) | more than 5 years ago | (#26252689)

I like honeypot links that blacklist anyone who clicks it. Seems to take out spam spiderbots effectively, until they learn how to avoid the honeypot links.

drupal techniques (0)

Anonymous Coward | more than 5 years ago | (#26252705)

I've had good luck using Drupal with a couple of techniques.

First is the Spam module. Pretty much out of the box.

Second is to have all comments subject to moderation.

Third is IP-address filtering via the .htaccess file. Blocking entire continents for some Drupal sites with a US geographic audience has keept the spam down to a very low level.

Moderation (0)

Anonymous Coward | more than 5 years ago | (#26252817)

Maybe use something that lets users moderate posts up or down with labels like Informative, Funny, Flamebait, and Offtopic. Also, don't forget include one involving a fictional bridge dweller, that will be misused by just about every moderator for anything they don't like.

gmail (1)

wangi (16741) | more than 5 years ago | (#26252865)

On phpbb boards I run the most productive things are:

1. Do not allow external links in profile of newly registered / non validated users

2. Do not allow registrations with gmail.com email addresses

3. Ensure "valid" timezone and country settings are selected by users.

L/

Re:gmail (5, Insightful)

siyavash (677724) | more than 5 years ago | (#26253295)

"Do not allow registrations with gmail.com email addresses"

That is one of the most stupid things I heard this year.

Re:gmail (1)

wangi (16741) | more than 5 years ago | (#26253697)

You'd be surprised. It is trivial for spammers to get a gmail account. Anyone who genuinely wants to contribute to a forum will have another email address, if not they will be able to explicitly email...

For small boards, passwords (1)

DMUTPeregrine (612791) | more than 5 years ago | (#26252917)

I run a site for my rennisance faire guild to talk at and plan things. We had tons of message board spam until I implemented a simple solution: a password is required to register. If not entered, registration fails. The password is posted elsewhere on the site in my case, but you could communicate it only to people who need access if the site is small enough.

Pivot open source blogging (2, Interesting)

macraig (621737) | more than 5 years ago | (#26252991)

The comment- and trackback-spam blocking techniques in Pivot blogging software are, from my limited personal experience, 100% effective. There's even an extension that uses the enormous Project Honeypot database (http:BL) to weed out IP addresses of identified harvesters and comment spammers. That's just for entertainment, though, since the basic techniques are completely effective.

Mollom (1, Redundant)

kbahey (102895) | more than 5 years ago | (#26253089)

Mollom [mollom.com] is free for low to medium traffic sites. They have plugins [mollom.com] for the major CMSes out there (Drupal, Joomla, Wordpress, and a bunch of others).

It is relatively new, but I use it on several sites and it works well. See the score card [mollom.com] for some fun.

The founder of Mollom is Dries Buytaert, the founder of Drupal, the CMS.

public filtering is an unsolved problem (1)

beefubermensch (575927) | more than 5 years ago | (#26253293)

If a spammer really wants to, he can test his attacks against the site until he beats your filter. Filtering works impossibly well, but only if the output of the filter is private. Spammers may not be doing these attacks now, but if everyone started using Akismet, no doubt they would start.

Reverse CAPTCHA (0)

Anonymous Coward | more than 5 years ago | (#26253363)

What I did in the past is to do a reverse CAPTCHA. Basically just a regular one but with a little text below: "Enter the letters shown in the image _in reverse order_". Stopped 100% of all spambots.

CRM114 is an option. (1)

Dr. Crash (237179) | more than 5 years ago | (#26253367)

CRM114 is an option you might want to consider.

Plusses and minuses:

+ REALLY FREAKIN' ACCURATE. trains to 10x better than human.
+ REALLY FREAKIN' FAST. 20-50 milliseconds/posting without even being demonized.
+ REALLY OPEN SOURCE. GPLed. Free forever.
+ REALLY FLEXIBLE. Has about a dozen built-in classifiers, most of which work on any human language (including
        chinese, japanese, korean, etc in their native formats).

- Arcane control language. "like 'awk' on meth". "grep bitten by a radioactive spider". You get my drift here?
- Not a drop-in solution for blogposting. You'll have to do some coding.
- Needs to be trained, with both positive and negative examples. When it wakes up, it knows _nothing_.

It's at "crm114.sourceforge.net"; there's mailing lists as well as an IRC channel (#crm114) on freenode.

How does Slashdot do it? (2, Interesting)

Ye_Gads (985366) | more than 5 years ago | (#26253561)

I rarely see spam here...or is it just quickly modded down to oblivion?

StopForumSpam.com works very well. (1)

asackett (161377) | more than 5 years ago | (#26253621)

Take a look at StopForumSpam.com [stopforumspam.com] . I've got it installed on a vBulletin forum and it works very, very well to prevent spambots from registering. Every now and then one sneaks through, but it's a lot less than I was seeing before.

The old "Fake Textarea" trick (1)

Dwedit (232252) | more than 5 years ago | (#26253631)

I'm using the old "Fake Textarea" trick. If anyone fills in the fake textarea, the post is rejected. The fake textarea comes up first, but is hidden with CSS. I also modified the forum software so that the fake text field has the same form name as what the forum traditionally uses for the real field.
I'm also using this in conjunction with blocking posts containing URLs from guests or users with no posts.

Of course, this is all useless against Stock Ticker symbol spammers.

spot the cat works for humans (1)

johnjones (14274) | more than 5 years ago | (#26253665)

hey there

if you want to filter for humans simply present a bunch of images and have the person spot the cat among the dogs

then apply the spam filtering (simple stuff really works you can even just use spam assassin plugins for content ) to get rid of the spammers posting urls and rubish and denie based on IP if you catch spam unless they contact you somehow

regards

John Jones

http://www.johnjones.me.uk [johnjones.me.uk]

use robbIE's patentdead PostBlock censoring device (0)

Anonymous Coward | more than 5 years ago | (#26253687)

it doesn't work well/isn't very smart, either. better days ahead. keep it to yourself, to avoid the embarrassment of being 'filtered'.

RBLs are annoying but effective (1)

erroneus (253617) | more than 5 years ago | (#26253725)

Recently, one of my users got infected with some spam-spewing bot malware which resulted in my company being listed at least four RBLs. It is annoying, but I can't hold it against the list services as I use them myself in my own filtering.

I have to wonder if RBLs of some sort could also be applied to web browsing especially on forums? But since most people are on dynamic IP addresses, I can only assume that without some very clever ideas to go along with it (perhaps some sort of cumulative scoring + fingerprint method?) RBLs for dynamic IPs is a rather bad idea. Still, it would be nice if there were some means of simply blocking "infected" IP addresses or at the very least calling the problem to the user's attention in some way.

My Own Experience (0)

Anonymous Coward | more than 5 years ago | (#26253803)

Last year I wrote my own software for a blog to be viewed by just family and friends. I figured security by obscurity. Was I ever wrong. As soon as I was discovered the spam just flooded in at a rate of about 250 posts a day (which completely blew me away).

I noted that it was all designed to raise pagerank. Just random words and several links. So I wrote a simple function that every time a post was submitted it would be checked to see if there was more than two links. After all Grandma won't be sending me tons of links anyway.

If it detects more than two links the function spawns a separate thread which waits ten seconds then deletes the post. In short the post last just long enough for the spammer to test if the posts are still sticking. After I installed the filter about 16k posts were filtered out (which also blew me away) until the spam stopped all together.

I was actually pleased to realize that I was spammed that much, because it meant 16k posts were filtered absorbed by my blog rather than an un-protected one.

If I could think of a better way to take up a spammer's bandwidth/processes I would do that as well.

Fr0st pi5t (-1, Offtopic)

Anonymous Coward | more than 5 years ago | (#26253959)

beyond the scop3 of get tough. I hope people's faces is

Custom solutions (1)

ericlondaits (32714) | more than 5 years ago | (#26253967)

The phpbb forum I administer fell victim to spammers more than a year ago so I tried a bunch of MODs that implemented a couple of changes to foil attempts of automatically registering. Spam slowed down a bit, but still was strong enough to be a problem... it seems that whatever script spammers use to post in phpBB already implements most standard MODs.

PhpBBs own Captcha is no good either... ... So what I did was implement my own validation, which requires to enter a fixed word ("Dragon") in a text box. It's not a captcha... it simply states "Enter the word 'dragon':" in plain text. That did the trick and spammers completely dissapeared from the forum. ... So as long as you're not running a high profile site, a custom mod should be enough.

Load More Comments
Slashdot Account

Need an Account?

Forgot your password?

Don't worry, we never post anything without your permission.

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>
Sign up for Slashdot Newsletters
Create a Slashdot Account

Loading...