Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Spam Through HTTP Referrer Logs

timothy posted more than 10 years ago | from the laundering-kiddie-porn-money dept.

Spam 52

Max Romantschuk writes "This morning while doing my usual log review of reader activity on my weblog, I discovered some rather strange sites, porn sites, which were linking to me. Closer inspection revealed that they weren't linking to me at all, but that someone had falsified the HTTP referrer header to inject the links into my logs." (Read more below.)

Max Romantschuk continues: "It took a moment to realize what was going on, but then it dawned to me, I was being spammed through my referrer logs! A quick google search on the words "referrer spam" confirmed my suspicions, this was indeed a widespread practice, and not new at all. In fact, Wired had an article on the subject dating almost a year back. It turns out the spammers aren't after blog authors, but what they are actually doing is targetting people which publish their referrer logs on their sites automatically. Fortunately, I don't.

I run a very small site, and get about 20 to 50 visits a day, and I don't publish my logs. Not exactly a likely target, am I? Clearly these spammers seem to do this in volume, and the phenomenon is bound to increase as email spamming is becomming increasingly hard. With email spam, IM spam, Windows Messaging spam (NET SEND popups) and HTTP referrer spam, how long will it take until every open technology has to be locked down? I hate to say it, but I doubt Wikis and similar systems will stay open for very long if things keep going in this direction."

cancel ×

52 comments

Sorry! There are no comments related to the filter you selected.

Ummm...Hello? (0, Offtopic)

GTRacer (234395) | more than 10 years ago | (#7567972)

Well, this article's been up for like 5 minutes, so I might as well...

fr0st p1st!

Oh yeah, SPAM is bad! Speaking of SPAM, I saw the official SPAM-mobile yesterday on I-95...Not nearly as cool as the Weinermobile!

GTRacer
- huhuhuhuh...he said weiner...

The idea behind a Wiki (3, Insightful)

jjshoe (410772) | more than 10 years ago | (#7567976)

The idea behind a Wiki is that anyone can maintain it. The more people that maintaining something, (Linux) means all the more people to remove nasties. In this case the nasties just happen to be spam. As long as copies of the Wiki are kept after every N changes all should be good, just in case a spammer deletes everything...

Re:The idea behind a Wiki (2, Interesting)

cfradenburg (592693) | more than 10 years ago | (#7568388)

Due to the fact that anyone can maintain it spammers can add and change it. Now, can any number of people find and delete spam in a Wiki faster than however many bots the spammers decide to throw at it?

Re:The idea behind a Wiki (2, Informative)

RyoSaeba (627522) | more than 10 years ago | (#7568710)

I'm contributing to Wikipedia [wikipedia.org] , and we have some ways to deal with vandalism. We weren't (yet !) victims of determined spammers with bots, so it's theoritical, but here are things we can use:
  • first, all changes appear in a special page, so anyone can see them, and switch back to a previous version in history. Anyone can in one click see differences with the previous version
  • all contributions of users (anonymous or not) are easily viewable by anyone, thus cleaning after finding a spammer is made easier
  • sysops (contributors with some maintenance rights) can revert last changes of anonymous users in a few clicks
  • sysops can delete pages (to clear new pages created by bots, in this case)
  • sysops can block IPs if needed, preventing the edition of pages from those IPs
  • sysops can also block usernames
  • sysops can protect pages, preventing any edition (to protect main page for instance, in case of repeated vandalism)
  • worse case, a filter can be added to the computer's firewall settings.

And, given the number of contributors and sysops, it's almost certain there's a sysop nearby at any time. Of course, if spammers attack from 50 IPs, one sysop alone will have a hard time to fight & clean the mess :)

Re:The idea behind a Wiki (1)

jonadab (583620) | more than 10 years ago | (#7577348)

This is all not enough, if bots can sign up for accounts. You're going to need
eventually to set things up so that only a verified human can get an account,
and only a logged-in user can edit anything.

Not Always Spam (2, Interesting)

Anonymous Coward | more than 10 years ago | (#7567984)

If you snoop on people, you get what you deserve.

Personally I don't like people tracking my referrer links. Mind your own business. If you want to see who is linking you, you can do that with google. I know people disagree, since your website is your business. But I don't like being monitored that closely.

Maybe I'll set my referrer to goats.cx.

BTW, this story has been seen on Slashdot before.

Re:Not Always Spam (0)

Anonymous Coward | more than 10 years ago | (#7568259)

>Personally I don't like people tracking my referrer links

Re:Not Always Spam (0)

Anonymous Coward | more than 10 years ago | (#7568428)

> Maybe I'll set my referrer to goats.cx.

Cool. Easier to lock your sorry paranoid ass out. Some of us just like knowing what internal links people are using so we know what menus are working on the site and which aren't. Without using a cookie, which I'm sure you also fuck around with on principle.

So blank it (2, Insightful)

J_DarkElf (602111) | more than 10 years ago | (#7568653)

Personally I don't like people tracking my referrer links. [snip] Maybe I'll set my referrer to goats.cx.


Just leave your damn referrer blank then. I suppress the referrer through Opera everywhere, and only enable it on sites which are foolish enough to believe I want to leech their images, and on those maybe one or two sites where I know they use my referrer info for something useful.

But don't set it to some bogus info, or you're no better than these crimina^H^H^H^H^H^H^H spammers.

Re:So blank it (1)

GoRK (10018) | more than 10 years ago | (#7569554)

It would be useful if a browser had an option to set the referrer to:

1) Blank
2) Constant value
3) Same URL that is being retrieved
4) "base" URL of the site being accessed -- ie if you were acccessing http://www.yahoo.com/some/path/some/file.html the referer would be "http://www.yahoo.com/"

Re:So blank it (2, Informative)

Anonymous Coward | more than 10 years ago | (#7570295)

4) "base" URL of the site being accessed -- ie if you were acccessing http://www.yahoo.com/some/path/some/file.html the referer would be "http://www.yahoo.com/"

privoxy [privoxy.org] can do this.

Re:So blank it (2, Informative)

Carnildo (712617) | more than 10 years ago | (#7571733)

The Proxomitron does #3 -- with the side benefit of letting me view images that people have hotlinked from Geocities and other free hosting providers.

Re:So blank it (0)

Anonymous Coward | more than 10 years ago | (#7571953)

Proxomitron used to use a URL for the author's favorate band.

Re:So blank it (0)

Anonymous Coward | more than 10 years ago | (#7570010)

But don't set it to some bogus info, or you're no better than these...spammers.

Oh please. Fucking around with your referrer information isn't even close to spamming.

Re:So blank it (1)

mikeswi (658619) | more than 10 years ago | (#7572774)

> sites which are foolish enough to believe I want to leech their images

Clearly you don't run a site yourself. That happens. There is nothing foolish about checking for it.

It costs me hundreds of MB per month if I don't keep an eye on my logs. If my bandwidth use suddenly goes up and I start seeing the same forum showing in my log hundreds of times, going to one of the URLs inevitably shows some asshat using an image from my site in his avatar or sig.

> But don't set it to some bogus info, or you're no better than these crimina^H^H^H^H^H^H^H spammers.

Agreed. Some firewalls actually advertise there. I'm tempted to start displaying a message to the effect of "Did you know your firewall is telling me exactly what software you're using?" Since it's a "privacy" option, that oughta shake a few people up.

Re:So blank it (0)

Anonymous Coward | more than 10 years ago | (#7582326)

Here's a possible solution for you: RFC 2397 [ietf.org] . The basic idea is that you can embed your graphic data right in the page. There's no separate URL to the images, so there's no way they can link to you from offsite.

Obvious problems: your pages become huge, people who don't want images have to suck them down anyway, and it only works in a couple of web browsers. I guess it depends on just what you're trying to do and how bad the problem is.

I'd paste the example in here, but it would probably trigger a filter. If you're running Mozilla, load that RFC page, highlight the whole mess starting with "data:image...", then load it just like it's a URL. You should see a face.

Re:So blank it (1)

maximilln (654768) | more than 10 years ago | (#7586939)

-----
going to one of the URLs inevitably shows some asshat using an image from my site in his avatar or sig.
-----
I had no idea that referrer IDs and URLs were embedded in pictures. Not that I have a sig or an avatar (a what?) but it's an interesting bit of information for me.

At what point are we going to start tracking our pee after it's in the ocean?

Re:So blank it (1)

mikeswi (658619) | more than 10 years ago | (#7588767)

No, these people are loading these images directly from my site on the offsite forums and that runs up my bandwidth.

Re:So blank it (1)

maximilln (654768) | more than 10 years ago | (#7592391)

I had no idea people were that uncouth. If I'm going to use a pic for something it's going to reside on my system.

Re:Not Always Spam (1)

Brandon T. (167891) | more than 10 years ago | (#7570414)

Snooping on people is not really the problem. I don't really care if people blank out their referrer or put something bogus instead. The problem is that by having your logs constantly spammed, your log data becomes useless. If you're using a log analysis program like webalizer, your total hits, visits, etc are way out of wack because only 1 out of every 3 or so hits is legitimate. You can't get an accurate picture of how many hits your site is actually getting. I don't know how it happened, but my site has gotten so bad that it is constantly being bombarded with these hits from a bunch of different ip addresses and a ton of different domains.

Re:Not Always Spam (1)

dtfinch (661405) | more than 10 years ago | (#7570487)

Referral logs are a valuable tool for webmasters. For one, it'll tell you which search phrases are getting you the most hits.

Re:Not Always Spam (1)

Max Romantschuk (132276) | more than 10 years ago | (#7575392)

Personally I don't like people tracking my referrer links. Mind your own business. If you want to see who is linking you, you can do that with google.

True, but if you get a spike in visits, your referrer logs can often tell you who linked to you... I like to know whats happening, when it's happening ;)

I know people disagree, since your website is your business. But I don't like being monitored that closely.

As was already said, then block your referrer :)

BTW, this story has been seen on Slashdot before.

Maybe it has, but the Slashdot search engine failed to find it for me... and the editors published it, didn't they?

Well, (2, Interesting)

noselasd (594905) | more than 10 years ago | (#7568153)

Last time I asked people about this, I was told this was script kiddies
scanning for open proxies and similar things, using some certain scripts/whatever which annoyed the logs with falsifyed referes.

New spammers everywhere... (1)

hlh_nospam (178327) | more than 10 years ago | (#7588172)

... check out what the Herbalife and other MLM scumbags are doing to Monster.com. This phenomenon appears to be spreading over the entire net.

I have used Monster.com [monster.com] on several occasions, and even found a contract there a couple of times, and I was even considering advertising on their site. In just the last week or so, however, I have noticed a new trend that is rapidly rendering Monster.com completely worthless. Seems that my current job search agents (for C++/C#/Java programming) are returning dozens of hits -- but almost all for Multi-Loser-Marketing scams (mostly Herbalife [herbalife.com] , aka Global Online Systems [winnersstartnow.com] --this is one of several thousand of their replicated websites) and ads for services that purport to teach me how to "work at home" for a membership fee.


I have complained to Monster, and they have replied that yes, this is a violation of thier TOS, and yes, they would remove the ads that I called to their attention. Just for grins, I checked this morning to see just how many such ads there were on Monster, and found over 5,000 of the Herbalife ads, and about 1,000 of the "work at home" membership ads. This appears to be primarily the output of 3 organizations, with Herbalife 'distributors' responsible for the largest portion. If this is the beginning of a trend, then every MLM and suckerbait outfit on earth is going to be putting their crap there by the end of next week, drowning the legitimate job ads in the noise. For example, my last search produced exactly one legitimate job opening in the last 2 days, and 10 listings for a "work at home" [onlylegitw...mejobs.com] service. (How many legitimate businesses actually use the word 'legitimate' in their names, anyway?)


While tracking down the perpetrators of the most egregious ads, I came across this description [cockeyed.com] of just what Herbalife is, and the damage being done to the Sacramento area by Herbalife 'distributors'. Very interesting read. These scumbags are making spammers look good... OTOH, it sure seems to me that Monster needs to clean up its act, too. They obviously can't remove these fraudulent ads as fast as the MLM victims post them, so they need to start preventing them from getting there in the first place.

Small site? (4, Funny)

Hell O'World (88678) | more than 10 years ago | (#7568285)


I run a very small site, and get about 20 to 50 visits a day,
until I posted a link to it on Slashdot.

Re:Small site? (1)

Sam Lowry (254040) | more than 10 years ago | (#7568395)

He handles the load pretty well now, it is probably because he carefully prepared for the assault ;-)

Re:Small site? (1)

Max Romantschuk (132276) | more than 10 years ago | (#7570117)

I run a very small site, and get about 20 to 50 visits a day, until I posted a link to it on Slashdot.

He handles the load pretty well now, it is probably because he carefully prepared for the assault ;-)

"The assault" consisted of around 20 people who visited my page so far ;) Actually, one of my stories was featured on the front page, and even then there was only minor traffic.

Then again, in neither story was my site in the actual focus.

Check the referer? (1)

phorm (591458) | more than 10 years ago | (#7572064)

if ( $ENV{HTTP_REFERER} =~ /slashdot\.org/ )
{
mail("me@mycellphone.com", "help!", "I'm meeeelting!");
init_throttle();
pray();
}

Check this link for a suggestion to stop it (5, Informative)

Brandon T. (167891) | more than 10 years ago | (#7568408)

I was having the same problem; getting literally thousands of hits to my site from referrers for all kinds of porn and other random domain names. I did a google search and found this site: http://www.spywareinfo.com/articles/referer_spam/ [spywareinfo.com] . It shows how to use mod_rewrite with apache to block the most frequent domains. I took Mike's blacklist and created this page [resynthesize.com] , which automatically creates the .htaccess file for you. The problem is that they seem to be registering tons of new domain names so it's hard to keep up a decent blacklist.

Re:Check this link for a suggestion to stop it (1)

mikeswi (658619) | more than 10 years ago | (#7572855)

Glad you like it. I broke my stats program, so I have no idea how well my own method is working until I switch servers and reinstall awstats.

Target Audience (2, Funny)

Alethes (533985) | more than 10 years ago | (#7568611)

they are actually doing is targetting people which publish their referrer logs

Hmmm, who reads the logs that aren't published? Geeks with no girlfriends, maybe? Sounds like a good target audience for a porn site to me.

"Hey, why is [insert favorite porn site here] linking to my geek portal/blog? They must be a good site if they link to mine, and I can easily explain my visit to the boss!"

Re:Target Audience (1)

statusbar (314703) | more than 10 years ago | (#7571636)

Then the logs are parsed via webalizer and the webalizer results are indexed by google as lots of links.

Jeff

Since when (2, Funny)

Anonymous Coward | more than 10 years ago | (#7568967)

I don't think porn sites are strange at all, in fact there are lots of them.... how silly to think of them as strange...

Legality of wiki spam? (2, Interesting)

TimButterfield (16686) | more than 10 years ago | (#7569267)

Web sites can be defaced. This is typically thought of as illegal. Does the level of security on that site affect the legality of the defacement? Just because a wiki is more easily editable than an otherwise non-secure site should not automatically allow hijacking of that site for purposes other than those intended by its owner. Would the appearance of 'specific wording' on the site make enforcement of this easier?

Re:Legality of wiki spam? (1)

1iar_parad0x (676662) | more than 10 years ago | (#7578617)

Except that one guys defacement could be another man's legitimate posting. Take a look at the average message board. People make trolling, yet related comments everyday. Who is to say that is or isn't vandalism? Perhaps a better course of action would be to limit the number of posts in a given day. I would think 10 wiki posts (they should be insightful) would be more than enough. Sure bots could trash the site, but it would be too slow and painful for the average spammer.

I don't publish my logs (1)

NickDngr (561211) | more than 10 years ago | (#7569730)

I don't publish the logs on my very small, low-traffic site and I get quite a bit of this as well. All of the non-legitimate referrers on my site seem to be weblogs. No porn so far. I just ignore them. Referrer stats are the least useful part of my logs anyway.

links to resumes (2, Interesting)

displague (4438) | more than 10 years ago | (#7569957)

I would like to know who goes around posting links to their resumes as referers to your website?

Is it the people looking for jobs, or is it some resume posting service? I get about a half-dozen of these per month.

Re:links to resumes (0)

Anonymous Coward | more than 10 years ago | (#7575858)

Actually, I find looking at the referrer logs for my resume to be interesting. (That is, who is linking to my resume.) Shocking numbers of fairly obvious google searches will return my resume on the front page.

MovableType Blogs (2, Interesting)

ceejayoz (567949) | more than 10 years ago | (#7570041)

It's becoming a rather large problem on MovableType blogs. Apparently, the spammed referrers are usually fake blogs, that are front sites to get a porn webcam link high in Google PageRank.

http://echo.ashpool.org/blog/305/ [ashpool.org]
http://www.idly.org/2003/11/14/porn_sites_hiding_b ehind_blogs.php [idly.org]
http://www.jayallen.org/comment_spam/2003/11/alert _referral_spamming [jayallen.org]

Re:MovableType Blogs (3, Interesting)

Dachannien (617929) | more than 10 years ago | (#7572695)

Fortunately, Google is working on this problem. [slashdot.org]

As for solving the issue of false referrers, why not just modify where the referrer ends up based on whether the specified referring page actually has a link to you or not. The distributed effects of zillions of bloggers all spamming the spam site with automated HTTP requests should be enough to dissuade the spammers from continuing :)

Re:MovableType Blogs (1)

valdis (160799) | more than 10 years ago | (#7575691)

OK... I'll bite.. How do you tell if the page actually has a link to you without trying to fetch the page and seeing if there's a link? This gets particularly interesting when you deal with content generated on the fly - there's a very good chance that my Slashdot page has links on it that aren't on yours, for instance, and which also won't be on the page your proposed automatic verifier will get if it blindly chases the Referer: URL back.

Spider them before publishing log (2, Interesting)

Anm (18575) | more than 10 years ago | (#7571462)


I would think that it would easy enough to send a spider to the referrer page and search for the referred page. If you don't find it, delete it from the log. In fact, you wouldn't even need the spider because the link should be the exact page anyway.

This also becomes a means to maintain the blacklists other have mentioned.

Isn't this simple to do?

Re:Spider them before publishing log (1)

Kris_J (10111) | more than 10 years ago | (#7572941)

it would easy enough to send a spider to the referrer page and search for the referred page
The fairies bring you magic bandwidth each night do they?

ACK (1)

sploxx (622853) | more than 10 years ago | (#7571567)

Thanks for pointing out that this is spam!
I also get these "referers".

The sad thing is, that it is nowadays half-criminal to do a ping/traceroute to a certain host (Considered preparing an attack) but these spammers can generate their high volume(!) traffic, out of every RFC borders, and don't get problems at all.

So why are you posting this? (1)

s88 (255181) | more than 10 years ago | (#7571832)

"A quick google search on the words "referrer spam" confirmed my suspicions, this was indeed a widespread practice, and not new at all. In fact, Wired had an article on the subject dating almost a year back."

Thats not clue enough that maybe your lack of knowing about this isn't newsworthy?

Re:So why are you posting this? (1)

Max Romantschuk (132276) | more than 10 years ago | (#7575443)

"A quick google search on the words "referrer spam" confirmed my suspicions, this was indeed a widespread practice, and not new at all. In fact, Wired had an article on the subject dating almost a year back."

Thats not clue enough that maybe your lack of knowing about this isn't newsworthy?

My lack of knowing about it may also be an indication of this being a legitimate issue despite of being less than common knowledge. By managing to get this article published I may have raised public awareness of this issue, and thereby affecting the odds that something will be done about it.

The article also provided me with good ideas on how to battle this issue, benefiting both me and probably others as well.

Happened to me, too (1)

deja206 (711205) | more than 10 years ago | (#7572099)

Apparently I was linked by a porn site... I also got my first comment spam today, from a Turkmen guy... Deleted it anyway... =)

Google Has The Solution (1)

angedinoir (699322) | more than 10 years ago | (#7573119)

For those of you who don't fully understand the problem. You have to understand how google works a little, or at least one major point.
Google considers that if someone links to your site, you're probably better than someone who has nobody/less people that link to their site.
(If anyone has more detailed info on this, please feel free to post a reply)

One of the main reasons why spammers are stuffing their urls into your referrer logs is to boost their page rank in google. To combat that, google has a simple method for page designers to instruct its bots to not follow links from a certain page. Thus taking away the benefit of spamming your referrer log.

How can I prevent Googlebot from following links from a particular page or archiving a copy of a page? [google.com]

I would suggest adding these if you insist on keeping your referrer log on your web-site.

Time to start over (1)

Frisky070802 (591229) | more than 10 years ago | (#7574720)

With all the unforeseen problems of email spam, web links, blogs, etc., it's clearly time to

Throw out the internet and start over.

PS. Does this mean we have to curse Tim B-L [w3.org] in the same breath as Microsoft?

Re:Time to start over (1)

1iar_parad0x (676662) | more than 10 years ago | (#7591629)

Unfortunately, as long as everyone has rights to post on the internet (in one way or another), somebody is going to abuse that priviledge. Since when are HTTP Referrer logs considered good content. You don't really have control over what goes into those logs anyway. We need to find more ways of filtering out bad content. It's like free speech. Ham radio and usenet have their share of nuts. Most people don't turn to those places for news. We tend to filter information on our own.

Re:Time to start over (1)

Frisky070802 (591229) | more than 10 years ago | (#7592626)

I'm in violent agreement about needing ways to filter bad content, but not about filtering info on our own. Usenet not only has its share of nuts, its signal/noise ratio is awful. So isn't Slashdot a way of addressing that shortcoming, by having top-level stories moderated by trusted users, and comments trusted by experienced ones, allowing people to filter not just by reading stuff but by automatically avoiding stuff?

Lest anyone think this is offtopic, let me point out that Slashdot has in some sense redefined usenet, and it seems that similar approaches to redefining other internet applications --- making them accessible but not completely open --- would be a big improvement. Spam through referrer logs is an example of an open system getting abused.

Blog sites often show referers (1)

tronicum (617382) | more than 10 years ago | (#7585094)

Not only blog software is intended do present referrers even blog meta sites like

http://www.bloogz.com/ [bloogz.com]

shows referrers on their starting page. The only good thing about it is that they have to provide a working URL to get "return on investment".

I found this site in my blog-referrer stats, but I dont know if they crawled me or if my blog-provider sends information about new blogs to them....

Check for New Comments
Slashdot Login

Need an Account?

Forgot your password?

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>