×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Hotmail & Yahoo Mail Using Secret Domain Blacklist

timothy posted about a year ago | from the it-looks-like-you're-reading-a-newsletter dept.

Censorship 345

Frequent contributor Bennett Haselton writes: "Hotmail and Yahoo Mail are apparently sharing a secret blacklist of domain names such that any mention of these domains will cause a message to be bounced back to the sender as spam. I found out about this because — surprise! — some of my new proxy site domains ended up on the blacklist. Hotmail and Yahoo are stonewalling, but here's what I've dug up so far — and why you should care." Read on for much more on how Bennett figured out what's going on, and why it's a hard problem to solve.

On December 7th I sent out a normal batch of emails to the Circumventor mailing list, where I send out new proxy sites for getting around Internet filters. I registered seven new domains and sent each domain to one seventh of the list; the list contains about 420,000 addresses, so each one went to about 60,000 people. (Each new site is only sent to a random subset of the list, so that a blocking company can't just subscribe one address to the list and block all new sites as soon as they're mailed out.)

The list is also comprised of 100%-verified-opt-in addresses, meaning that a new subscriber has to reply to a confirmation message in order to be added to the list. That's considered the gold standard for responsible mailing, but major email providers keep finding new ways to block the emails as "spam," which sometimes provide interesting insights into how the filters work behind the scenes.

After the last mailing, for example, all of my newly registered domains got disabled by the registrar because two of the domains had been incorrectly blacklisted by the Spamhaus Domain Block List. It took two days to discover the problem and then several hours to trace the problem to Spamhaus, although once I found Spamhaus's automated form I was able to get the domains un-blacklisted immediately. So the registrar re-enabled the domains a few hours later, although the traffic to the domains never returned to its previous levels. Spamhaus, meanwhile, continues to claim the DBL is a "zero false-positive" list, and has yet to acknowledge the error or contact me to help get to the bottom of how it happened. Well, they know how to reach me.

At least this time around, my domains didn't get disabled. Instead, the messages rolled out for a few hours with no problem (replies from users indicated that at least some hotmail.com and yahoo.com users were receiving them), until bounces abruptly started coming in from hotmail.com and yahoo.com addresses saying:

----- Transcript of session follows -----
... while talking to mta5.am0.yahoodns.net.:
>>> DATA
<<< 550 Message Contains SPAM Content
554 5.0.0 Service unavailable

After pummeling my address with bounce messages (to the point where my own Gmail account started bouncing because it was getting hammered with so many bounce messages from Hotmail and Yahoo), when the dust finally settled, I tried reproducing the error by sending test messages from my server's IP address to a test Hotmail account. It turns out that out of the seven different URLs that I had been mailing to our users, four of the domains in those URLs would generate a "550 Message Contains SPAM Content" error when sent from my IP to a Hotmail address, and the other three did not. The message didn't have to contain the banned domain in the From: address; the message would get blocked if it even mentioned the domain anywhere in the message body. (This only happened when sending from my own IP address at peacefire.org. It didn't happen if I tried sending a message from my Gmail account to a Hotmail address, even if the message contained one of the four banned domain names, so the issue probably won't reproduce if you try sending a test message yourself.)

But interestingly, Yahoo Mail started bouncing my messages at about the same time — out of the seven domain names, the same four domain names were being bounced by Yahoo Mail as by Hotmail, also with the error "550 Message Contains SPAM Content." That's far too unlikely to be a coincidence, so it looks as if Hotmail and Yahoo Mail are using a common secret blacklist of domain names that cause a message to be blocked as spam. (As it happens, the other three domains were also being bounced by Yahoo Mail with the error "Message Contains SUSPECT Content" — as opposed to "SPAM Content" — while those three domains were not blocked by Hotmail at all. That of course is aggravating, but the real clue lies in the fact that both Yahoo Mail and Hotmail were giving "SPAM Content" errors to the exact same subset of domains.)

I don't want to publish the list of all seven domain names here, so as not to make it too easy for censorware companies to block them all, but one of the four blacklisted domains was 'golflanding.com.' (All of the new domains I register are nonsensical two-word combinations, since those are the only .com domains that are likely to be (1) still available and (2) easy to remember.) As soon as it seemed like Hotmail and Yahoo Mail were working off of a common blacklist, I checked to see if Spamhaus had screwed up again and listed our domains, but none of the seven domains were on Spamhaus's lists.

I looked up golflanding.com on the blacklistalert.org service, which checks against all major spam blacklists, but no hits were listed there either (except for on some defunct services which haven't been updated in years).

So if Hotmail and Yahoo Mail are both using the domain blacklist, perhaps it's a list compiled by one company and then licensed to the other, or perhaps it's a third-party list not widely known to the public. (Hotmail uses their own SmartScreen filter, but I've found nothing online about Yahoo using it as well.) It's conceivable that one or more of the domains might have gotten blacklisted as a result of Hotmail or Yahoo users clicking their "This is spam" button. However, Hotmail allows newsletter publishers to view data about what percent of their messages to Hotmail users are being flagged by users as "spam," and when I looked up the stats for our IP, they showed a "complaint rate" of less than 0.1% (usually the rest of people hitting 'Junk Mail' to unsubscribe from the list). Assuming that the complaint rates are similar for Yahoo Mail, it's unlikely that the domains got blacklisted as a result of user complaints, unless the blacklist trigger has a ridiculously low complaint threshold.

Neither the Hotmail postmaster site nor the Yahoo postmaster site mention anything about a list of domain names that could cause a message to be blocked for mentioning the domains in the message body. Yahoo Mail does provide a support form for newsletter publishers to send inquiries about why their mail is being blocked; I submitted that on Saturday and started a thread with email "support," although so far their response has just been to copy and paste articles from the Postmaster site, with tips like "Send email only to those that want it." Each time, I reply saying, No, this is not the problem, the problem is that the domains in the messages are getting incorrectly blacklisted, and each time, support cheerfully sends me another article. If I'm not literally talking to a bot, I might as well be.

I opened a similar ticket with Hotmail, and they sent me a form letter saying that the emails were being blocked because of SmartScreen, and that as a matter of policy, they would refuse to fix any errors being made by the SmartScreen filter. Waiting to see if I get a reply from a human next.

So why should you care? Well, for one thing, if you care about users in China and Iran being able to receive proxies to get around their Internet blockers, right now Hotmail and Yahoo are thwarting these proxies more effectively than those countries' own censors are. Yes, these are real people who really do write back to me after a mailing goes out, telling me about how they were able to use the proxies to receive banned political information, and sometimes how long the proxy lasted before the censors blocked it. This week, they had to do without.

But more importantly, this is an example of a general problem: That there are certain types of issues, like blocking of legitimate mail by spam filters, where the "free market" does not deliver the best experience to consumers, and the costs get passed on to everybody. Sometimes the problems could be solved with some effort, but the effort does not get made, because people believe that the free market will solve the problem, or that it already has.

In theory, if consumers have enough information about different companies and their services, the companies can compete to provide the best product to users. The problem is that if one type of information is systematically hidden from users — in this case, the fact that their mail provider is blocking mails from reaching them — then the "theory" falls apart. Since spam getting into your inbox is a visible problem, but missed email messages are an invisible problem, Hotmail's incentive is not to give the user the best experience, but rather to err on the side of blocking legitimate messages — even if the user might prefer to get slightly more spam, than to miss one important email that they were waiting for.

This means we're not just talking about a few messages getting caught in filters, which could happen even in an efficient marketplace. We're talking about a permanent equilibrium where the user gets a sub-par experience by default — a trade-off that causes them to miss more messages than they want to — and senders have to pay the cost of overcoming the marketplace inefficiencies. (Which means if the sender is a business you buy from or a charity you support, the costs get passed on to you.)

Pretty much the entire financial cost of sending email, is attributable to the failure of the "free market" to motivate email providers to deliver non-spam emails into their user's inboxes. If a company or organization uses an email list hosting company like AWeber or Constant Contact to email their users, they pay a fee of about $1 per month for every 100 users on their list (which would run me about $4,000 per month). That fee doesn't go towards bandwidth — even a 1-million-subscriber list, emailed once a month, would use less than 3 GB per month of bandwidth, which is what GeoCities was was giving away for free 10 years ago. What you're paying for is the fact that AWeber and Constant Contact have friends in the right places at Hotmail, Yahoo, and Gmail, so if your mails are getting blocked, they know the people to call to fix the problem. If you run your own list instead of paying a hosting fee to AWeber or Constant Contact, you'll end up paying other costs indirectly, through loss of income when your messages don't reach recipients, or in time and money spent trying to fix the issue. (I have to take this option anyway, since I send different URLs to different random subsets of my list, which is not supported by AWeber or Constant Contact.)

On the other hand, if the market actually "worked" — if email providers did reliably deliver non-spam messages to their users — a company or charity could run their own list for virtually zero cost, and would be able to keep all of that money. (I incur no up-front fees for running my own list; all of the costs are the time spent trying to get Yahoo, Gmail, and Hotmail to stop blocking it.) So every time you donate to a charity or buy from an online retailer, a little bit of that money goes towards the cost of that organization having to fight past marketplace failures in order to get their email to you.

I don't think there's an easy algorithmic solution, like crowdsourcing Facebook complaints or using random-sample voting on Digg. Generally, I just think we need more awareness of the fact that, under certain conditions (including those surrounding email deliverability), the "free market" is virtually guaranteed to arrive at a non-optimal solution. One manifestation of that awareness would be if Hotmail, Yahoo Mail, and Gmail created public points of contact where legitimate email publishers could find out why their emails were blocked, and had real humans responding to the messages and fixing the problems. By default, the imperfect information in the marketplace leads toward an equilibrium that errs on the side of blocking too much legitimate email, so anything that pushes the equilibrium back towards more legitimate messages getting delivered will improve the experience for users and lower costs for senders.

Besides, there's a more basic ethical issue here. If you're Hotmail and you tell your users that you're providing them with "email accounts," then those users expect those accounts to work — including having the ability to receive mails from mailing lists that they've signed up for. Helping legitimate emails get through to users is not just a matter of addressing a marketplace inefficiency, it's a matter of honesty.

Larry Lessig's book "Code is Law" describes how default choices built into the architecture of the Internet and other environments — the "code" — can steer our behavior in ways that we might not choose otherwise. I'm making essentially the same point in saying that some problems are not fixed by market forces, because people are not aware of the problem at all. I think the evidence and the reasoning are straightforward in this case, but it's hard to convince people who have adopted it as an axiom that whatever the free market arrives at, must be the solution. My favorite single sentence in Lessig's book was, "Put your Ayn Rand away." I could imagine the years of pushing against dogmatic fanaticism that led him to write that sentence, and I knew how he felt.

cancel ×
This is a preview of your comment

No Comment Title Entered

Anonymous Coward 1 minute ago

No Comment Entered

345 comments

conspiracy ! deliver my spam (0)

Anonymous Coward | about a year ago | (#42275475)

If it wasn't for Viagra ads generation US $50 a day, you could pretty much have unfetter delivery.
Do the proxies on your list relay smtp?

Re:conspiracy ! deliver my spam (1)

Anonymous Coward | about a year ago | (#42275827)

If it wasn't for real Viagra costing $25 a pill, there wouldn't be as hot a market for the spam.

Summary (5, Insightful)

sorensenbill (1931240) | about a year ago | (#42275493)

Is there a summary of the summary available?

Re:Summary (-1, Troll)

Anonymous Coward | about a year ago | (#42275653)

Bennett Haselton's a spammer. Any more questions?

Re:Summary (5, Insightful)

TheMMaster (527904) | about a year ago | (#42275767)

According to TFA his list is opt-in only, so unless he's lying about that he doesn't appear to be a spammer.

I've had similar experiences with Spamhaus btw, they decided to nix my upstream provider and when I complained I was told that I should use another ISP because mine wasn't well liked.

I can assure you I have never sent a single spam email in my life.

This is the whole point of TFA though, there's no incentive for companies running mail services to ensure that legitimate mail gets delivered. It's simply cheaper to not bother with false positives at all because the cost of non-delivery is placed squarely on the shoulders of the sender.
This is why Spamhaus could easily force me to switch ISPs, it doesn't cost them anything to put my IP range on a shitlist, but it cost me money and effort to migrate my service.

Re:Summary (1)

Anonymous Coward | about a year ago | (#42276757)

"Opt-in" covers multiple sins. Sometimes, it's a genuine "I want you to send me this stuff". Sometimes, it's "enter your email address to have a chance to win a magic unicorn. By the way, we'll also send you some emails", and sometimes it's an "opt-in" with the box checked by default, the way that everyone tries to get you to install browser toolbars or demo antivirus products.

Then you get people, who generally get sent marketing junk from any number of companies that they may or may not remember having dealt with. Typically, people don't go to any effort to try to unsubscribe from these lists, but just hit th e"report as spam" button because they don't want to read it.

Re: Summary (5, Informative)

Urza9814 (883915) | about a year ago | (#42276705)

As a long-time subscriber to his list (at least 6 years), no, he's absolutely not. He provides a fantastic service and does a damn good job of ensuring only those who want the messages are receiving them. And I get less than one message per month from that list. If he's a spammer, so is literally every single person or organization that has ever sent me an email.

Re:Summary (2)

TubeSteak (669689) | about a year ago | (#42275775)

Is there a summary of the summary available?

We call them "titles"
Here's one example: Hotmail & Yahoo Mail Using Secret Domain Blacklist

Simple summary (5, Informative)

Pollux (102520) | about a year ago | (#42275909)

He's saying that Hotmail, Yahoo, and GMail are running a cartel of free online webmail services.

He's trying to get opt-in email to accounts on these systems, and it's not going through. He has evidence indicating these services operate a common hidden blacklist service keeping those emails from getting to the accounts. He cannot reach people within these organizations to open up emails coming from his domains, as he does not have an inside contact to "assist" him with this problem. This leads him to speculate that Hotmail, Yahoo, and GMail are operating like a cartel, where only "approved" email list hosting service companies with inside contacts are able to do business with these services.

Better?

Re:Simple summary (4, Interesting)

niiler (716140) | about a year ago | (#42276117)

Bingo. Good summary. I gave up using my own server to send email a couple of years ago for precisely these reasons. It wasn't worth trying to get de-blacklisted every few weeks because my server had an obscure domain name. If I recall, when I sent out more than 10 emails in a batch (we're talking maybe as many as 30) to members of a class, this triggered the anti-spam bots. When I did it from gmail or from other major providers, things worked beautifully. I had too many irons in the fire to deal with this, and while I would love to use my own server's email capability, it's not worth it anymore.

Re:Summary (2)

TheRealMindChild (743925) | about a year ago | (#42276791)

Spamhaus != 0 false positives. This guy sends the same email out to tens of thousands of people who tend to use Yahoo or Hotmail. They both block the messages as spam.

Just FYI, I seen this guy bitching about it MONTHS ago. Apparently he still hasn't made a lot of headway. However, if you operate like a spammer (sending the same email to multitudes of folks, while relaying information about open proxy servers as information), then you will be treated like a spammer

Dude (-1, Troll)

Anonymous Coward | about a year ago | (#42275505)

You're a spammer... Hotmail and Yahoo are doing us good... Get lost!

Re:Dude (0, Troll)

sexconker (1179573) | about a year ago | (#42275665)

You're a spammer... Hotmail and Yahoo are doing us good... Get lost!

Yup!
Can't believe this kid has the audacity to complain about it on Slashdot with a wall of text to hide the fact that he's a fucking spammer.

Re:Dude (-1)

Esandman (141148) | about a year ago | (#42275779)

Agreed...you send out 60k e-mails with the same web address in them and expect to NOT get picked up by a spam filter? This has everything to do with sending an e-mail bomb and nothing to do with oppression (not to say that oppression isn't happening).

Re:Dude (3, Informative)

Anonymous Coward | about a year ago | (#42276405)

Do you people not understand the concept of an email newsletter? For instance, I am subscribed to NASA Tech Briefs 's email newsletter, which purports to have an audience of over 77,000. Being a newsletter, of course those emails all have "the same web address in them" -- they're the same bloody content. This has been going on for decades (they've been a big thing since home users who never heard of usenet started getting internet access...), and as long as it ONLY GOES TO PEOPLE WHO VOLUNTARILY SUBSCRIBED, it's NOT MOTHERFUCKING SPAM! If your spam filter flags this, your spam filter is broken. Spam= UNSOLICITED bulk email, not all bulk email.

Re: Dude (4, Informative)

Urza9814 (883915) | about a year ago | (#42276871)

FWIW, I'm on that list. And if I was using hotmail or Yahoo I would be PISSED about missing those messages. Been on it since highschool where I used them to bypass the school's web filters (occasionally teachers would even promote these sites because we literally couldn't do our work without them); today I still use them for testing and occasionally at work if, for example, I need a document from scribd (why that is blocked I'll never understand...)

Re:Dude (1)

lister king of smeg (2481612) | about a year ago | (#42276199)

Its not Spam if you opt in. Spam is unsolicited. For this you have to request. Now is it possible the guy is bull shitting that part sure, however if we accept that the articles are bull why bother to read them?

Re: Dude (2)

Urza9814 (883915) | about a year ago | (#42276801)

I've been on his list for around six years, and as far as I can tell, everything he says in the article is 100% accurate.

Also worth noting that he submits articles about these things to Slashdot quite regularly. I recall one a few months back where he was first considering this exact experiment. I'd go find it, but I'm posting from my phone.

Re:Dude (2)

jellomizer (103300) | about a year ago | (#42276007)

I hate to use the if you were legit then you wouldn't need a proxy argument. However If he was using email the way most services want you to use it, he wouldn't have a problem.

Email was meant for a Person to send a message to another person or a small group of people, usually with people that you have some connection too.

Re:Dude (0)

Anonymous Coward | about a year ago | (#42276367)

> some connection too.

I'm sorry, your sentence abruptly ended. The connection also what?

Re:Dude (2)

crypticedge (1335931) | about a year ago | (#42276461)

I have to use a mail proxy, not because I spam (we send about 20 emails a month) but because verizon blocks port 25 outbound, and won't let me get a static IP at home for my mail server.

I pay 20/year for my mail proxy, gives me 200/mo that we never hit.

Re:Dude (0)

Anonymous Coward | about a year ago | (#42276759)

Why not use Verizon's mail server?

Re:Dude (5, Insightful)

magic maverick (2615475) | about a year ago | (#42276435)

After the last article I signed up for the service of getting emailed the proxy sites. Guess what, I've had no problem. I've not recieved any spam to the email address I used. I've only received emails that I specifically requested.

So, ah.

Dude, you're a fucking idiot. Hotmail and Yahoo are not doing anyone good... Get lost!

If someone is running an incredibly popular opt-in email list, that doesn't automatically make them a spammer. In fact, because it's all opt-in it makes them the opposite. It's solicited, not unsolicited. Mr Haselton is one of the good guys, and you are a moron if you can't see that.

Re:Dude (0)

Anonymous Coward | about a year ago | (#42276549)

Parent calls him a spammer, gets +5 Informative.
I call him a spammer a ways upthread, get -1 Troll.

yeah, spam blacklists are a poor solution (1, Insightful)

Trepidity (597) | about a year ago | (#42275511)

I could maybe see their necessity 10 or 15 years ago, but statistical classification techniques are good enough these days that a blunt tool like a domain blacklist doesn't really make much sense. Heck, Paul Graham was arguing that seven years ago [paulgraham.com], and it hasn't gotten less true.

Server load (1)

betterunixthanunix (980855) | about a year ago | (#42275785)

Blacklists are nice because they reduce server loads. Sure, running a statistical classifier for one user is not so hard, but if you have to process hundreds of millions of messages per day, that is a lot of CPU time spent on spam.

Now, I agree that blacklists are bad, but we do need some system that doesn't require large amounts of CPU time or other resources. Hashcash is interesting here, in that the CPU time is mostly spent by clients; one might be able to slow spam down enough to let a combination of statistical filtering and greylisting take over.

Re:yeah, spam blacklists are a poor solution (1)

Anonymous Coward | about a year ago | (#42275929)

I wonder how many job opportunities I've missed or friends I've drifted apart from because of email dropped by statistical classification techniques. That's why everybody uses Facebook to keep in touch now.

Re:yeah, spam blacklists are a poor solution (1)

ColdWetDog (752185) | about a year ago | (#42276087)

I wonder how many job opportunities I've missed or friends I've drifted apart from because of email dropped by statistical classification techniques. That's why everybody uses Facebook to keep in touch now.

Friends? An AC on Slashdot?

Jobs? An AC on Slashdot?

Not to worry.

Re:yeah, spam blacklists are a poor solution (1, Insightful)

niiler (716140) | about a year ago | (#42276165)

Mod up. This is a very good point. Closed systems like Facebook seem to work.

Re:yeah, spam blacklists are a poor solution (1, Insightful)

pixelpusher220 (529617) | about a year ago | (#42276455)

yes but maybe not for who think they work for...

Re:yeah, spam blacklists are a poor solution (0)

Anonymous Coward | about a year ago | (#42276817)

Pick your poison.

Facebook is 100% reliable in getting a message through.
Gmail and Yahoo are pretty good.

For everything else, losing a single email to a relative, friend, or employer is simply not acceptable. We're not talking about pictures of kittens, here.

I used to use my own domain for email, but now I'm lucky if messages end up in the receiver's spam box as opposed to just silently dropped. And even there, nobody checks their spam folder anymore.

So let Facebook and Google mine your messages, or wonder if anything you send ever makes it to the recipient.

Re:yeah, spam blacklists are a poor solution (1)

AlphaWolf_HK (692722) | about a year ago | (#42276933)

The spammers have found various ways around these. Often they throw a bunch of the "high target" key words (e.g. viagra, cialis, penis enlargement) in as images, or they'll use computer generated text that looks somewhat real enough to even fool some human readers in order to throw off those filters. This works because the more words you have, the less likely the small terms will be snagged.

No Comparison To China and Iran (1)

Anonymous Coward | about a year ago | (#42275541)

The blacklists and censorship dealings in China and Iran are directly attributable to their respective governments, there is no similiar connection in hotmail and yahoo's blacklists.

Stop this, you look like fools.

Re:No Comparison To China and Iran (1)

rudy_wayne (414635) | about a year ago | (#42275893)

if you care about users in China and Iran

You had me up till there. At that point I realized you're an asshole and stopped reading.

Spam is like cancer (2, Insightful)

Anonymous Coward | about a year ago | (#42275567)

The only treatment is a deadly poison that you hope kills off the bad parts before the good suffers too much.

Distribute the load (2)

betterunixthanunix (980855) | about a year ago | (#42275995)

Part of the problem with spam fighting is that we are not distributing the spam fighting load. Hashcash distributes the load somewhat, in that it forces spammers to use more resources to send out their message and can slow them down somewhat. A distributed filtering system that allowed people to volunteer CPU time and bandwidth to filter spam (with some system of gaining the trust of an email server) might also work; imagine if hundreds of millions of people were relaying / filtering 100 messages per day.

People still use Yahoo mail? (1)

Anonymous Coward | about a year ago | (#42275679)

I think you just wanted to go on a political rant there. Seriously, you spend the post talking about the failings of two companies, ignoring the fact that there are other companies out there (well, you do mention GMail once, but you don't give any supporting evidence for it not being "open"), and act like two companies doing particular things is some kind of "failure of the free market."

So what's your solution? What's to stop a government-owned email provider from using this SmartScreen thing "as a matter of policy?"

but, really: (-1, Flamebait)

cellocgw (617879) | about a year ago | (#42275697)

Who uses hotmail or yahoo mail for anything other than an anonymous dead drop anyway? Almost any other service is better.

You're a bleeding moron (1)

Anonymous Coward | about a year ago | (#42275717)

Seriously? It's fucking news that there might be domain blacklists that aren't public knowledge?

"Free market" scare quotes (2, Insightful)

Freddybear (1805256) | about a year ago | (#42275755)

What's with the gratuitous complaints about the "free market" not giving some mythical "optimal solution" that lets you send your "100% guaranteed opt-in" spam without interference? I call bullshit. If Hotmail isn't accepting your "really honest it's not spam" mailing list stuff, maybe you should try contacting them about it. The "free market" doesn't magically solve problems without people doing what it takes to address the problems.

Re:"Free market" scare quotes (0)

ADRA (37398) | about a year ago | (#42276107)

No, the free market 'says' that if Hotmail is an inferior product then people will find a different product to use. You shouldn't give one hair arse if you're being blocked. Its those using inferior products that should feel sad about it. Email is about as far from monopoly terratory as you can get.

Re:"Free market" scare quotes (1, Insightful)

Freddybear (1805256) | about a year ago | (#42276363)

That's just silly. If you can't be arsed to do something about your "honest it's not spam" emails getting blocked, you don't have any business complaining about the people who do the blocking. Stop complaining about "the free market" as if you'd prefer an unfree one.

Optimal != Perfect (1)

Anonymous Coward | about a year ago | (#42276827)

The "Optimal solution" isn't "perfect".

There are always tradeoffs, and the power of the free market is that it is relatively effective at weighing different options.
It basically brute forces the answer to any question. It's messy, ugly, often inefficient, but it works.

You are a spammer (-1, Troll)

mlwmohawk (801821) | about a year ago | (#42275811)

Your behaviors are those of a spammer. 420,000 addresses? You are surprised that you were blocked? There does not need to be any conspiracy, it only means that there is similarity in their algorithms.

Anyone that wishes to deliver 420K k-emails in a batch SHOULD be shut off. That volume of email can not contain any valuable information. Its nothing but the crap we have to endlessly delete. You are the type of email abuser that makes spam filters nessesary.

Re:You are a spammer (0)

Anonymous Coward | about a year ago | (#42276073)

I love how you obviously do not understand the purpose of the emails this guy is sending...

Re:You are a spammer (0)

Anonymous Coward | about a year ago | (#42276115)

All 420,000 signed up for it, and confirmed their email addresses. How can it be unwanted? If they didn't want it, they could click the unsubscribe link.

How can you think that there aren't 420,000 people in the world who may have a common interest, and want to receive the same newsletter?

Re:You are a spammer (0)

Anonymous Coward | about a year ago | (#42276259)

420k emails is a lot? tha tis nothing, grupon sends 10m plus everyday, but wait they are own but same people as gmail, hotmail,yahoo etc...

Re:You are a spammer (5, Informative)

glaurungn (1253152) | about a year ago | (#42276307)

He sends proxy address to people that requested that information. He send it weekly because the proxys are blocked.

You missed the point. (2)

CaptainNerdCave (982411) | about a year ago | (#42276357)

The issue is that no one on the list of recipients got the chance to refuse the message.

How can you be certain he is not part of an internet forum dedicated to anonymity? What if he were sending an email with updates on domains that are security risks to a long list of subscribers to his IPsec newsletter?

There is a very long list of possibilities for what he could have been doing that was perfectly legitimate. Basically, USPS, UPS, FedEx, DHL, $common-carrier should not read your text-only message to determine if there is any information they don't like, and refuse to deliver it based on that alone.

Sigh (0)

junkgoof (607894) | about a year ago | (#42276403)

Blocklists are not a bad thing. I dealt with a number of them when I inherited an SMTP open relay 10 years ago or so. People tend to hate them because they rant at the (generally unpaid) people running the blocklist instead of taking steps to show they are mailing sanely. I configured my SMTP server and got the IP removed from all (and there are a lot of them) blocklists including a number with a reputation for being unreasonable. Politeness goes a lot further than ranting.

This guy may say what he's doing is normal and reasonable but it sounds as though he's blatantly spamming. If the guy does not want his stuff flagged as spam he should try sending e-mails with the same address people opted in for.

Re:You are a spammer (5, Interesting)

niiler (716140) | about a year ago | (#42276583)

His behaviors are _similar_ to those of a spammer in number only. Having visited his site: http://www.peacefire.org/ [peacefire.org] it seems that he gets his email list from people subscribing to it on his site. If I understand it correctly, people who sign up for this list are looking for regular updates to proxies so that they can avoid censorship. As proxies are discovered by governments or certain companies , they are blacklisted, and new proxies must be created and sent out to the interested masses:

"Of course, employees of blocking software companies have gotten on this list as well, so they add our sites to their blocked-site database as soon as we mail them out, but in most places it takes 3-4 days for the blocked-site list to be updated. So the latest one that we mail out, should usually still work. "

Now it could be that there is a better way of doing this, but it seems to me that no matter how this game is played, constant updates to users should be the norm...

Now that I think of it, perhaps a Firefox extension could do the trick. Signed extensions can be updated automatically. The extension could have obfuscated URLs that are decrypted with something like this: https://addons.mozilla.org/en-US/firefox/addon/domcrypt/ [mozilla.org] and then wired in to automatically select an available proxy from the current batch. Not perfect by any stretch of the imagination, but it solves the "spam" problem. Also, it maybe easier for users and harder for censors? Crap... now I'm not going to get any work done...

Re:You are a spammer (0)

Anonymous Coward | about a year ago | (#42276739)

I know that these days it's usually too much to expect people to RTFA. So instead I tell you to RTFS.

If you had actually read and understood the summary, then you would realize that, if the story in the summary is true, this person's actions were very legitimate and not the actions of a spammer.

Have a good day sir. Don't let the door hit you on the way out.

It's op-in (0)

Anonymous Coward | about a year ago | (#42276835)

I wouldn't be surprised if some large companies' customer email lists have that many subscribers, all of whom EXPECT their incoming mail to be delivered without errors.

Prob. just a syntax error (-1)

Anonymous Coward | about a year ago | (#42275891)

Send me a copy of your email list, it may be a formatting issue.

Question that was never answered last time... (5, Interesting)

Anonymous Coward | about a year ago | (#42275911)

Are the proxy servers you are sending out on these lists capable of relaying mail onwards on port 25? If so this is probably a significant factor in these blacklistings. If you block outbound connections to port 25 when you set up these proxies, you'll probably find your blacklist problems are significantly reduced.

5 second summary (1, Insightful)

IamTheRealMike (537420) | about a year ago | (#42275919)

Blah blah blah ...... I sent craptons of mail to people who I'm sure want to receive it ..... but the system is telling me people don't .... blah blah ..... free markets suck.

I have worked on spam filters before. I've heard this story a million times. In case the article poster reads this, here's the blunt reality:

Those half-million people you think really really want new proxy sites all the time? Guess what, many of them don't. They are reporting your mail as spam which is why you're getting blocked (this is domain reputation). You may not understand why, but they are, so deal with it. Expire addresses that signed up a long time ago - some people won't unsubscribe when it's no longer useful for them. Make sure it's a simple, obvious one click operation to unsubscribe, and I mean really one click - not "click, log in, go to preferences" etc. Being able to unsubscribe should be the easiest thing in the world.

If SpamHaus is blacklisting you, they probably think you're sending mail to their spamtraps. Hence the "zero false positives" claim. Are you sure every single address on your list replied to a confirmation mail? All 400,000+ of them? Because it sounds unlikely.

Re:5 second summary (4, Informative)

DRJlaw (946416) | about a year ago | (#42276415)

Those half-million people you think really really want new proxy sites all the time? Guess what, many of them don't. They are reporting your mail as spam which is why you're getting blocked (this is domain reputation). You may not understand why, but they are, so deal with it.

You assume that this is case, yet the poster provides a link to management data which at least appears to show that your assumption is incorrect. Did you read the post where it mentions that "[it] showed a 'complaint rate' of less than 0.1% (usually the rest of people hitting 'Junk Mail' to unsubscribe from the list)," or are you simply going to deny any version of reality that doesn't align with your assumptions.

Expire addresses that signed up a long time ago - some people won't unsubscribe when it's no longer useful for them.

Apparently, deny any version of reality that doesn't align with your assumptions.

BAD 'EXPERT'!

If I sign up to a mailing list, I expect to receive the output of that mailing list until I unsubscribe. I certainly don't want the mailing list silently dropping me, and I'm not very interested in the ISP offloading its mailing list problem onto me by making me affirmatively renew my subscription. Especially when you offer no evidence that 'addresses that signed up a long time ago' make up a disproportionate fraction of the alleged 0.1% spam report rate.

Pushing the problem onto the 400,000+ individual users instead of dealiing with it at the ISP level is exactly the sort of free market failure tha the poster complains of.

If SpamHaus is blacklisting you, they probably think you're sending mail to their spamtraps. Hence the "zero false positives" claim. Are you sure every single address on your list replied to a confirmation mail? All 400,000+ of them? Because it sounds unlikely.

Again, deny any version of reality that doesn't align with your assumptions. He isn't being blocked by SpamHaus. He's being blocked by Hotmail and Yahoo. Just admit that you haven't actually read the post, that you're spouting off about your own personal bugbear, and that your advice has almost no bearing on the actual problem. It'll make you feel better, honest.

Re:5 second summary (2, Interesting)

Pope (17780) | about a year ago | (#42276543)

Why does he need to send 400,000+ emails in the first place? If it's just a list of proxy domains, why not just have an RSS feed that people can subscribe to? No emails needed.

Re:5 second summary (1)

Kergan (780543) | about a year ago | (#42276753)

+1. TD;DR the article, but the parts I did made this whole story reek of "your unsubscription method isn't braindead obvious enough to end-users, so they're unsubscribing by hitting the Spam button until your emails go away for good."

Re:5 second summary (2)

amicusNYCL (1538833) | about a year ago | (#42276897)

They are reporting your mail as spam which is why you're getting blocked (this is domain reputation). You may not understand why, but they are, so deal with it.

That's one possibility, and may even be likely considering his subject material. In this example he says he sent a total of 7 new proxy domains to 420,000 addresses, but only sent 1 domain to each person. So each domain got sent to a random 60,000 people, his reasoning being so that a censor could not subscribe and get a list of all new proxies, they would only get one (per address, at least).

But, instead of them getting those emails and blocking the proxies, it may be more effective for the censors to always report his emails as spam, thereby getting them blocked, and then no one gets any of the 7 new proxies. So the people reporting spam aren't doing it because they don't want the mail, they're doing it to stop other people from getting it.

Obviously, this is 100% speculation.

Really? (1)

Anonymous Coward | about a year ago | (#42275951)

Oh, someone is still using Hotmail and Yahoo?

Wouldn't it be just easier to do vice versa, block both and it would be a favor to us all.

This guy's got balls (-1)

Anonymous Coward | about a year ago | (#42275953)

F'ing spammer

gold standard for responsible mailing (4, Informative)

joostje (126457) | about a year ago | (#42275959)

Yes, verified opt-in is one requirement. But if you don't want to be marked as sender of SPAM, you should also make it *very* simple to unsubscribe. I know I've subscribed to a few lists, and at first read the emails, then ignored them, and eventually thought "should unsubscribe". But if that unsubscribing is difficult, I'll just hit "spam" in gmail (or whatever). I don't see the emails and more, and the sender gets blocked as spammer.

Re:gold standard for responsible mailing (0)

Anonymous Coward | about a year ago | (#42276159)

I work for a company that sends 2 email blasts per week to confirmed opt-in subscribers. We have a 1-click unsubscribe. We maybe 3-5 unsubscribes per week ( for a blast about 10,000 email addresses strong ). Believe it or not, most of those are our members CALLING us to have them removed. Our unsubscribe link is at the top & bottom of every email we send yet they call us for it. Granted, unlike the OP's business, we have a lot of non-tech savvy subscribers. I just wanted to point out that one-click solutions still don't mean anything.

I have had an email bill that we sent out get marked as spam by MSN & AOL. The reasoning had nothing to do with the content, but was actually because we sent X emails to their server in a short period of time and tripped their spam filters. It took an email and a phone call the first time. Since then, we were added to their respective whitelists and haven't had an issue since.

Re:gold standard for responsible mailing (0)

Anonymous Coward | about a year ago | (#42276325)

Even with a really simple unsubscribe, you'll have idiots that can't be bothered to understand how it works. On hlds, Valve's mailing list for server operators, there was this dude that wrote to the list to demand to be removed from this list or he would spam everyone in it. The unsubscribe URL is below every single message sent to the list. It's a list for server operators, it's opt-in only with a verification email. You would think he would be able to handle something as a simple unsubscribe. The point is, no matter how easy it is to unsubscribe, you'll have stupid people that will mark a message as spam even when they asked to receive it. There is just no way around that.

Re:gold standard for responsible mailing (2)

Revotron (1115029) | about a year ago | (#42276735)

You do realize you're talking about Valve servers, and this person who threw the temper tantrum on the mailing list is probably 12 years old and bought his server with daddy's credit card? It's no surprise, really. If you're looking for foolish, overdramatic, hot-headed people, look no further than Counter-Strike players.

Re:gold standard for responsible mailing (5, Informative)

magic maverick (2615475) | about a year ago | (#42276539)

Here's the latest email I got from Mr Haselton (with the email addresses changed though).
It's apparently very easy to subscribe. (Though it's not one click as you do need to enter your email address if you use the webpage option.) Is that good enough for you?

From: Bennett Haselton at Peacefire.org <webmaster@yahoo.com>
Reply-to: "Bennett Haselton at Peacefire.org" <webmaster@yahoo.com>
To: webmaster@hotmail.com
Subject: new Circumventor, in a new format
Date: Fri, 07 Dec 2012 04:00:02 -0500 (07/12/12 10:00:02)
Envelope-To: webmaster@hotmail.com

[You are receiving this because you subscribed to the Circumventor distribution list.
To unsubscribe from this list, click here:
http://www.peacefire.org/circumventor/cv-unsub.html [peacefire.org]
or reply with the word "unsubscribe" in the subject.]

Happy Holidays everybody -- your early Christmas gift enclosed:

https://www.kitepuddle.com/smart/ [kitepuddle.com]

This Circumventor site is in a different format but it should work as well as the others. You *must* access this one with 'https' at the beginning of the Web address; it won't work with 'http'.

You can attempt to access the "regular" Facebook through this one, for example, but it might not work correctly; the most reliable way is to enter http://m.facebook.com/ [facebook.com] on this Circumventor site, which will take you to mobile Facebook. Unfortunately Youtube still isn't accessible yet but we're working on it.

Don't waste too much time on those school computers - Santa's watching!

Bennett

***

"When I was in high school these twins got mono. They got stereo." -Demetri Martin

Peacefire.org
14615 NE 30th PL #10D, Bellevue WA 98007/blockquote.

No Coincidence (-1)

Anonymous Coward | about a year ago | (#42275971)

"That's far too unlikely to be a coincidence, so it looks as if Hotmail and Yahoo Mail are using a common secret blacklist of domain names"

That doesn't follow. All it means is that your mail met some common criteria that both use for SPAM. Since you sent a similar set of mail to each of them, its not really surprising that they triggered a similar response.

Is this a repeat? (1)

rudy_wayne (414635) | about a year ago | (#42275991)

I could swear this same guy was complaining about problems with his "I swear it's not spam" mailing list several months ago.

Re:Is this a repeat? (0)

Anonymous Coward | about a year ago | (#42276183)

Not enough people bought it last time, so he figures he needs to tell the world again.

Yahoo outgoing mail filter (0)

Anonymous Coward | about a year ago | (#42276035)

Yahoo has various keywords blacklisted too. Try sending an email from Yahoo containing the words "Western Union" or "Bitcoin" and it throws up a captcha.

Independent verification of verified/double opt-in (1, Interesting)

bersl2 (689221) | about a year ago | (#42276055)

I used to work security at a major hosting provider. If we got complaints about your mailing list, the first thing we'd do is ask you about how you got your list, to see if it complied with our requirement for verified opt-in lists only. We'd also sign up ourselves or check logs and code, because customers always lie (except when they don't).

Right now, I'd apply the same standard of skepticism. I understand that revealing such things would make your proported aim of censorship circumvention hard, but I'd still like to hear independent verification from someone who can reasonably demonstrate the depth of their commitment to opting in.

apple has a "secret list" too it seems (1)

crisper (12620) | about a year ago | (#42276065)

Apple has a "secret list" too it seems, I had one case of this with one domain. When I called I explained to normal tech support the issue, they had me escalated where I explained the issue in a bit more detail. Within an hour or two I had a call back from Apple support telling me that the domain had been removed, I didn't pry any more I just figured since they have the right to deny email for whatever reason then have the right to do this. This came after looking over logs, and some packet captures, to make sure it was being delivered to their servers before making the call to Apple. Nothing indicated any type of failure/deferred/blocked from looking at those logs/captures.

Great! (0)

Anonymous Coward | about a year ago | (#42276069)

I think it is a great feature!
So much spam is sent these days with only a short cryptic text and a URL that it is a necessity to block on domain names mentioned in messages.
Apparently it works fine.

All I can say is (-1)

Anonymous Coward | about a year ago | (#42276081)

Good. Maybe if you weren't a spammer, you wouldn't be put on a spam list?

Not a hard problem to solve for PGP. (1)

DamnStupidElf (649844) | about a year ago | (#42276137)

Even S/MIME might meet your needs in this case. Encryption is cheap enough even for mailing lists now.

Re:Not a hard problem to solve for PGP. (0)

Anonymous Coward | about a year ago | (#42276865)

Wait, how can you encrypt an email for a mailing list? You would have to encrypt the session key with each recipient's public key, otherwise how on earth would they access it? If you tried to do it the other way around, then that means literally anyone can read the message, so what is the point of encrypting it?

Even the article comes across as SPAM! (-1)

Anonymous Coward | about a year ago | (#42276167)

Oh my god. This guy just spammed slashdot and got away with it. Way to go timothy.

Not a spammer! (0)

Anonymous Coward | about a year ago | (#42276477)

Please keep in mind that Bennett has a legitimate purpose for his email patterns- he is trying to distribute proxy domains to people living in parts of the world where the internet is censored.

did you think of... (1)

sithlord2 (261932) | about a year ago | (#42276495)


- Implementing DKIM?
- Implementing SPF?
- Make sure the sender address doesn't bounce?
- Make sure you don't open thousands of connections to the receiving party for each recipient ? (in case of yahoo, hotmail, gmail, ...)
- The contents of the e-mail is not considered spam? (provide unsubscibe link, no big images included, etc...)


Setting up a mass-mail infrastructure is not to be taken lightly. There are lots of reasons why you could be listed as a spammer. That's why most companies outsource their their mass-mailing to 3rd parties like MailJet, MailChimp, SendGrid...

Hotmail and Yahoo (0)

Anonymous Coward | about a year ago | (#42276585)

Seems to me the problem is people are still using these for e-mail accounts.

I don't understnd the animosity here (3, Insightful)

Anonymous Coward | about a year ago | (#42276591)

Early on (before I quit reading) the OP said:

  It turns out that out of the seven different URLs that I had been mailing to our users, four of the domains in those URLs would generate a "550 Message Contains SPAM Content" error when sent from my IP to a Hotmail address, and the other three did not. The message didn't have to contain the banned domain in the From: address; the message would get blocked if it even mentioned the domain anywhere in the message body.

It seems to be treating his email as spam even when he sends one email to a single address.That isn't spam.

Use DKIM (1)

pr0gr3sR (463347) | about a year ago | (#42276603)

Had a similar problem with Yahoo... Implemented domain keys and signed all my outbound mail and it fixed the problem.

420,000 addresses (-1)

csumpi (2258986) | about a year ago | (#42276687)

I stopped reading there. You should get blocked. I hope my ISP blocked you, too. If I want to know what you're doing, I'll subscribe to your RSS feed or check your website. No need for you to be knocking on my email box.

If you behave like a spammer... (0)

Anonymous Coward | about a year ago | (#42276779)

Spammers rotate domains to avoid filters all the time. Legitimate senders of mail don't.

Bennett is behaving like a typical spammer, rather than like a legitimate user of the internet. Rather than changing his behaviour, he wants to whine about it. None of this is new behaviour, he always thinks that his goals are so noble that he can do whatever he likes and not be called on it - which dates back at least a decade, when he was promoting the idea of abusing insecure servers so that you could hide your activity by channelling your traffic through them.

Web page (0)

stabiesoft (733417) | about a year ago | (#42276803)

With 420K users, why would you not have a web page that gets updated with the same info. Users could check it at any time for the latest version. A "pull" instead of a "push" approach.

tell me more, (0)

Anonymous Coward | about a year ago | (#42276869)

so i can feed the whois results straight into iptables.
i note that this spammer has a psychopathic sense of entitlement to leech off other people's work.
how about a proper day's work, ideally something not involving networked computers ?

Load More Comments
Slashdot Account

Need an Account?

Forgot your password?

Don't worry, we never post anything without your permission.

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>
Sign up for Slashdot Newsletters
Create a Slashdot Account

Loading...