Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Ask Slashdot: How Do You Handle SPF For Spam Filtering?

samzenpus posted about a year and a half ago | from the false-positive dept.

Spam 187

An anonymous reader writes "Our organization had had a decent SPF record of our own for a long time. Recently, we decided to try using SPF for filtering inbound mail. On the up side, a lot of bad mail was being caught. On the down side, it seems like there is always a 'very important' message being caught in the filter because the sender has failed to consider all mail sources in writing their record. At first, I tried to assist sending parties with correcting their records out of hope that it was isolated. This quickly started to consume far too much time. I'm learning that many have set up inaccurate but syntactically valid SPF records and forgotten about them, which is probably the worst outcome for SPF as a standard. Are you using SPF? How are you handling false positives caused by inaccurate SPF records?"

cancel ×

187 comments

Sorry! There are no comments related to the filter you selected.

Forget about them (0, Troll)

Anonymous Coward | about a year and a half ago | (#42815393)

If they are too stupid to set it up correctly then they don't deserve to work with you.

Re:Forget about them (3, Insightful)

BradleyUffner (103496) | about a year and a half ago | (#42815693)

That works fine until the CEO misses an email from a prospective client.

Re:Forget about them (3, Interesting)

BlueBlade (123303) | about a year and a half ago | (#42816599)

The way I do is is that I send an NDR on rejected mail caused by bad SPF records (with some anti-flood limits). So far, as far as I know, bounced emails always eventually reached us. What happens is the person gets a NDR mail ("We have rejected your email due to bad SPF records"). The person getting the bounce forwards it to their IT dept, where it's usually taken care of rather quickly. If it's a small business without a dedicated IT staff, I'm sometimes asked to explain how this works, but usually, such companies do not have SPF records at all, which means the mail won't bounce.

Re:Forget about them (3, Insightful)

Cajun Hell (725246) | about a year and a half ago | (#42815769)

If they are too stupid to set it up correctly, then they aren't the fools whom you are supposed to part with their money?!

"Too stupid" is exactly the kind of person I'm looking for!

It's the smart people I don't want to hear from. You know the people I'm talking about: the ones who are so smart that they don't have to work, they just program their botnet to send viagra spam and sit back and collect the money. I admire them, but they're useless to me.

Re:Forget about them (2)

smash (1351) | about a year and a half ago | (#42815781)

Because of course SMTP administration competence of the company's (possibly hosted) email is directly proportional to competence in the field the company works in.

Pull your head out of your arse - in the real world, businesses need to communicate.

Re:Forget about them (1, Insightful)

pla (258480) | about a year and a half ago | (#42816203)

Because of course SMTP administration competence of the company's (possibly hosted) email is directly proportional to competence in the field the company works in.

Yup, pretty much. If you walk around - alone - wearing an "i'm with stupid" t-shirt, I don't care if you make Stephen Hawking look like Forest Gump, people will steer clear of you.


Pull your head out of your arse - in the real world, businesses need to communicate.

Yes. Yes, they very much do. And if they don't take that function seriously enough to make sure their audience can hear them, do you really want to do business with them?

They also need to make pay their bills - Do you also overlook your customers just "forgetting" to pay you because they have their AP system set up poorly?


Unless, of course, your core business depends on a steady stream of "bigger idiots", in which case, just reverse the polarity of the SPFion flux.

Re:Forget about them (5, Funny)

Anonymous Coward | about a year and a half ago | (#42816657)

If you walk around - alone - wearing an "i'm with stupid" t-shirt, I don't care if you make Stephen Hawking look like Forest Gump, ...

Oh come on! Forest Gump would run circles around Stephen Hawking, any day of the week!

Re:Forget about them (1)

Lefty2446 (232351) | about a year and a half ago | (#42817595)

LOL Modpoints, tha sarcastic humor is killing me!

Re:Forget about them (5, Insightful)

Anonymous Coward | about a year and a half ago | (#42817481)

Yes, people would steer clear of someone who looks superficially stupid, that is a given. That isn't a defense of why you should steer clear of such a person though, especially if their skills and services are of particular use, or if they are a potentially big customer. By rejecting providers, you are going to tend to pay more for services because now you only look at a subset of potential providers, or by rejecting customers, you are rejecting sources of income. Either way, there is a cost associated with that. If it turns out it only costs you a couple hundred dollars in the long run, then maybe it is worth the saved effort by your IT department. If it is going to cost you much more, is it really worth it? How many thousands of dollars in extra costs or lost revanue is acceptable because you don't feel like dealing with such people?

It reminds me of when a coworker once started up a simple online store in his free time and offered me a bit of money to look over the front and backends while he still learned web development. I found out his website redirected IE users to a page telling them to get Firefox and didn't let them get to the actual store. As this was around 2005 or 2006, the logs showed that 90% of his traffic was being redirected to that page, including cases of people trying multiple times to get back to the product list. His response, "I don't want to deal with people too stupid to use IE, and I would have to waste time to make my site work on IE too." My response, "First, if I copy-paste your pages, they completely functional in IE as is. Second... you sell hot sauce, what do you care what browser people use?" He just brushed it off, and continued to insist that he didn't want to bother with such people. Considering he was able to get a couple hundred dollars a month after a bit of local promotion, and assuming there isn't some massive correlation between browser use and hot sauce purchasing, he chose not to turn that into couple thousand dollars a month over such superficial, trival BS.

So, exactly how much money would you give up because you want to tell a customer, "Sorry, we don't want to accept your email," even if your business dealt with products that has nothing to do with email otherwise?

Re:Forget about them (4, Insightful)

MightyMartian (840721) | about a year and a half ago | (#42815877)

And meanwhile in the real world where nailing some important email because the sender was sending all his email through a local MTA because his ISP doesn't have an externally accessible MTA, your boss is right now handing you your walking papers.

The only sane way to use SPF is to drop a spam score of an email. Outright filtering on bad or missing SPF records is just a recipe for a large number of false positives.

Re:Forget about them (2)

jht (5006) | about a year and a half ago | (#42816619)

This. SPF is fine as something you can use to tag messages and increase their filter score. If they lack a valid SPF there's a slightly higher risk it's spam. But to block based on SPF records is just goofy. It's a good idea, but nowhere near universally adopted and there's plenty of valid reasons why mail would go through a different source.

On my own mail server, I add 1 to the scoring, with tag at 3.5 and block at 5.5. Those have worked pretty well (I use Kerio Connect, which has a Spamassassin-based system).

Do the right thing (1)

mhab12 (1180139) | about a year and a half ago | (#42815403)

We're experiencing the identical problem, but keep on doing what you're doing. It is scary to see 'valid' mail from Fortune 500 companies who can't keep track of their own SPF records being dropped by Postini, but it is happening on a regular basis. We're doing what we can to help out those who are affected and eventually we will help reduce spam and get everyone into the 21st century with their outbound mail and SPF records.

Re:Do the right thing (3, Insightful)

Anonymous Coward | about a year and a half ago | (#42815453)

Some would say you are doing a disservice to your customers by continuing a practice that is hurting their business in an effort to promote a technology standard that is not working.

Re:Do the right thing (1)

icebike (68054) | about a year and a half ago | (#42815507)

Some would say you are doing a disservice to your customers by continuing a practice that is hurting their business in an effort to promote a technology standard that is not working.

Others would say you are doing a disservice to your pocketbook by turning away mail just because your customers aren't always up to speed, or have contracted their mail handling to someone else.

Re:Do the right thing (4, Insightful)

MightyMartian (840721) | about a year and a half ago | (#42815899)

Anyone who understands SMTP and spam knew from the very moment that SPF and its cousins/descendants were proposed that it was a hopeless measure. That, after ten years, guys like me are still having to explain "setting your SMTP server to reject because of SPF" tells you just how badly SPF failed.

Id10ts (-1)

Anonymous Coward | about a year and a half ago | (#42815405)

Those people should not have mail servers. But those people are paying clients so good luck.

don't reject based solely on SPF (5, Insightful)

jjeffries (17675) | about a year and a half ago | (#42815419)

Or anything else, for that matter--mark it as "spf failed" and score it as you feel appropriate for your filtering setup.

Re:don't reject based solely on SPF (4, Insightful)

smash (1351) | about a year and a half ago | (#42815801)

This.

If email filtering was as simple as dropping non-SPF approved mail, spam would not exist. There is no single silver bullet in the war against spam. And besides, when domains cost a couple of dollars to register, it's entirely possible to set up an SPF enabled domain and spam from that.

Re:don't reject based solely on SPF (1)

marka63 (1237718) | about a year and a half ago | (#42816871)

SPF is not a spam solution. Spammers can have legitimate SPF records.

SPF is design so that the recipient can reject forged emails without the blow back impacting the person whose email address is being forged. This only works if the published SPF records reflect reality.

Re:don't reject based solely on SPF (1)

hedwards (940851) | about a year and a half ago | (#42817287)

You're supposed to combine SPF with DKIM so that you know who is sending messages from where. The point isn't to kill spam completely, just to make it as inefficient as possible. If you make it inefficient enough that they're all losing money, they'll move onto another scam.

Spamassassin (5, Interesting)

icebike (68054) | about a year and a half ago | (#42815469)

Spamassassin handles SPF, reasonably intelligently, that is, not trusting it completely, not giving it more weight than it deserves.
Hanging your spam fighting hat on any single hook is problematic. and SA uses a wealth of tools with constantly updating itself via
scripts. Its been largely trouble free, and we have it set up so that it will learn false positives and false negatives when users
move these to the corresponding folders.

I've been well served by Spamassassin [apache.org] for some time now, it runs quietly
on our mail server. SA does not block mail. It flags it. Our mail server will evaluate these flags and trash outright the most
egregious spam, but we have the limits set low enough such that we will allow the questionable things through.

We error on the side of caution, but we still dump a lot of mail right after SA flags it. (Our business can do that, your business
may not be able to do that.)

Re:Spamassassin (2)

MightyMartian (840721) | about a year and a half ago | (#42815935)

Yup. We run SpamAssassin along with ClamAV and postgrey on top of Postfix to filter all the incoming mail and it does a pretty good job. Here and there a spam will get through, but I can look at my logs to see all the rejections. It's not rocket science, and the solution has been tested and proven for over a decade now.

Re:Spamassassin (1)

Albanach (527650) | about a year and a half ago | (#42816251)

I think this makes sense where your organization cannot devote additional resources. However it's important to work out how much staff time is wasted by your spam volume.

Personally I would reject if the sending server lacked a PTR record, or if the PTR record was invalid, or on SPF failure. I use templates to quickly reply where someone had trouble contacting us, directing them to contact their IT department and referencing RFCs where necessary. Even large ISPs would get it wrong sometimes, but invariably folk got it fixed within a day or so.

Almost no spam got through, so there are security gains and productivity gains. I found that to be worthwhile, but it will differ for every organization.

Use it for scoring, not blocking (5, Informative)

Anonymous Coward | about a year and a half ago | (#42815479)

Too many users are still careless with it. This is because it was proposed as a DNS standard, but was poisoned by Microsoft cluttering it with the entirely distinct "DomainKeys" project, then deliberately mislabeled the SPF version that they use. (See the history at spf.pobix.bom)

The result is that SPF has been less useful than expected. Also, SPF does not work well over mail relays unless they're configured to to indexing and re-writing of the "From " field, and pass the bounces through the relay server on its way back to the original sender. But it has been enormously effective to add to spam *scores*, to give anything with a bad SPF result a much lower score as a potentially valid email message.

Re:Use it for scoring, not blocking (4, Informative)

dshk (838175) | about a year and a half ago | (#42815703)

A small correction: Forwarders must rewrite the reverse-path, which is the address they submit in the MAIL FROM SMTP command, not the From field in the mail content. They leave the From field as it is. Actually they should not tamper with the mail content at all. I believe that all large forwarders have been SPF compatible for years. Otherwise they could not deliver their mail to a very large percentage of recipients.

Re:Use it for scoring, not blocking (3, Interesting)

MightyMartian (840721) | about a year and a half ago | (#42815965)

Less expected by whom exactly? I was part of the major discussions a decade ago surrounding SPF and there was a large contingent of damned good mail admins and SMTP experts who said the whole thing was flawed.

Re:Use it for scoring, not blocking (1)

thogard (43403) | about a year and a half ago | (#42816711)

SPF was broken from the start but it was a step towards a better solution.

I liked the idea of using dns or snmp to ask the server that should have sent the message "did you send message id foo_xyz? That alone would have fixed a massive amount of spam if you tie it back to mx records of what should be sending the messages.

Re:Use it for scoring, not blocking (1)

marka63 (1237718) | about a year and a half ago | (#42816893)

MX records are for inbound traffic. SPF is for outbound traffic. Don't mix the two.

Re:Use it for scoring, not blocking (1)

thogard (43403) | about a year and a half ago | (#42817005)

MX are for inbound delivery. SPF are for outbound authentication so it is even a different concept. MX records are nearly required for mail yet SPF isn't and will never be so the only authoritative way to find out if a server is involved with mail at this point in time is to look at an MX record... which is pointless for any real verification of an outbound message.

The asymmetrical nature of email has always been the problem that will always allow spam. The only real fix is to force the inbound mail servers (which can be determined via MX records) to verify a given message that it may not have any info about. There were proposals for things like looking up example.com._.user.message-id or even just example.com._.user to see if they are approved to send messages. The problem is that if you do not centralize outbound email, you can not stop spam. That is one reason why X.400 mail systems worked they way they did.

Re:Use it for scoring, not blocking (1)

marka63 (1237718) | about a year and a half ago | (#42817201)

It's perfectly possible to authenticate email with distributed senders. It just requires willingness to deploy the tools to do it. This includes updated clients. Whether that email is spam or not is a orthogonal matter.

Re:Use it for scoring, not blocking (1)

thogard (43403) | about a year and a half ago | (#42817395)

The distributed authentication tools that use cryptography that would be required to be deployed are illegal in many parts of the world so I don't see that as a decent solution to the problem.

Re:Use it for scoring, not blocking (1)

shirikodama (2670887) | about a year and a half ago | (#42817619)

Microsoft sponsored SenderId. Domainkeys was from Yahoo and eventually was standardized into DKIM (rfc 4871)

Filter based on IP geographic region instead (1)

Anonymous Coward | about a year and a half ago | (#42815483)

The best mail filtering is based on geographic region of the envelope IP address. Most spam comes from countries without any proper anti-spam legislation, so simply junk them all until they get their act together.

Re:Filter based on IP geographic region instead (1)

dshk (838175) | about a year and a half ago | (#42815717)

AFAIK most spam come from the USA. Or maybe they are second.

Re:Filter based on IP geographic region instead (1)

Anonymous Coward | about a year and a half ago | (#42815815)

Oh thank you for reminding me. Back when we had scads of ever angrier customers in the mail because we wouldn't answer them. Or rather, we couldn't. They were verizon customers, we were in europe, and verizon in their wisdom had decided that "too much" spam was coming out of yurp so they geoblocked it at the router level.

Right on the heels of a then-contemporary report saying that most spam (by far) came out of 'merica. Good show, verizon, I won't soon forget that little stunt.

Re:Filter based on IP geographic region instead (1)

Anonymous Coward | about a year and a half ago | (#42815843)

I have never seen any spam from the USA. Lots of spam might purport to be from the USA but if you check the envelope you will discover that it is probably coming from Moldova, Kazakhstan, Poland, Spain, Brazil, Argentina, Romania, Iran, or Taiwan.

Re:Filter based on IP geographic region instead (1)

dshk (838175) | about a year and a half ago | (#42816809)

Spam from Iran? Strange. I checked my last four spam mails: 1 China, 1 France, 2 USA.

OK, it is a pathetic statistics, but still... The country is based on IP address of the server which directly connected to our server, and I lookud up the IP address on ip2location.com.

Re:Filter based on IP geographic region instead (1)

smash (1351) | about a year and a half ago | (#42815823)

If you are in the business world this will simply NOT work. Presumably, you are talking about russia and china - big businesses are doing a lot of work both in those regions and with companies located in those regions. For your home connection, sure....

Re:Filter based on IP geographic region instead (1)

_merlin (160982) | about a year and a half ago | (#42815957)

Most spam I have to deal with comes from US, although Japan has increased its proportion of spam recently. I keep reading people complain about spam from China and eastern Europe, but I never receive much of it. I still think it's a case of racism causing blindness.

Re:Filter based on IP geographic region instead (0)

Anonymous Coward | about a year and a half ago | (#42816801)

Do GeoIP lookups on the sending SMTP servers, you'll see.

I get India, Kazakhstan, Russia, "East Ossetia", Poland, Czech Rep, Romania by the shovelful.
Actually not terribly much spam from China, but plenty of SNMP sweeps and ping sweeps.

The Truth About WWII and FEMA (-1)

Anonymous Coward | about a year and a half ago | (#42815505)

Since WWII, bigfoot and yeti sightings have been constant-- but FEMA still wants us to think of them as "rumors" and "hoaxes." This is just one of many cover-ups in FEMA's long and patchy history.

Top-ranking generals in Iraq authorized a drone strike on multiple news agencies critical of JWoww-- but cancelled the operation when a connection to FEMA was exposed.

Old classmates say that JWoww associated with Communists during college.

One prominent reporter discovered an unmarked surveillance device under his car after he published an article on this topic.

Next time you're in a major US university's library, try having the library fetch you their collections of prominent publications about the stock market. Don't be surprised when they say those books have been "checked out" since WWII. Somebody doesn't want you reading them!

If you speak out about this, you are practically assured to go missing. Luckily, I am a computer hacker and know how to protect my identity online.

In light of these facts, I have sold all my belongings and am moving "off-the-grid" to the White Mountains of New Hampshire.

whitelist (3, Insightful)

shentino (1139071) | about a year and a half ago | (#42815517)

Use SPF as part of a whitelist/blacklist scheme.

For sources that have their shit together, trust their SPF records as an absolute metric.

SPF does work if set up correctly.

Re:whitelist (2)

urbanriot (924981) | about a year and a half ago | (#42817097)

I'm fully in agreement and I'm questioning whether some of the commenters here suggesting to use scoring over blocking, for senders that have enabled SPF, are actively involved in the administration of high volume email servers. If a company has created an SPF record, clearly they want people to block email that is not originating from their servers. This does not result in 'false positives' as its an SPF record with the DNS server - the address doesn't change.

Re:whitelist (0)

Anonymous Coward | about a year and a half ago | (#42817493)

There are several reasons why an e-mail legitimately sent from the approved outgoing servers will end up with an altered path that causes it to fail the eventual check by the receiver.. the most common and obvious one is forwarding. SPF isn't followed strictly, because these things happen quite frequently and drop alot of legitimate incoming mail.

There are solutions like the extra fields inserted into the mail that keep the chain of hops intact, so it can be unwrapped by the receiver... but all parties have to stick to that as-of-yet non-standard method and must use the *same* style of indicating that chain.

Don't rely on SPF (-1)

Anonymous Coward | about a year and a half ago | (#42815525)

What we do is treat it as an extra part of spam filtering rather than trusting it 100%.

The spam filtering to quite high but if it has a valid SPF it counts a lot towards it not being spam. If it has a valid DKIM then it passes 100% and we also whitelist certain senders who we get a lot of e-mails from. So far this has been working well for us, not too many false positives.

The curial part for us is that we don't deduct points for not having a valid SPF, only treat it as less likely to be spam if it is valid.

We don't reject, but we send some "helpful info" (3, Informative)

neiras (723124) | about a year and a half ago | (#42815553)

We had the same problem.

Our solution: if the message makes it past all our spam filters and the only problem is that the sender's client isn't allowed by their SPF record, we send a friendly, plain-english informational message back to the sender, then deliver their message to our users.

"Please tell your company's IT staff that your mail server isn't set up correctly. This may cause your messages to be rejected or classified as spam. Please forward the following information to your system administrator: [SPF record] [sender's IP]. Thank you!"

Usually the issue is that the sender has set up GMail to send "from" their company email without sending through the company SMTP servers, usually without authorization.

We do the same thing with broken DKIM signatures.

This has worked well for us, since senders get tired of the message and seem to get things dealt with. We considered adding a "threat" saying that if the problem wasn't fixed, we'd block the sender, but that was rightfully shot down as pretty mean.

Re:We don't reject, but we send some "helpful info (1)

hawguy (1600213) | about a year and a half ago | (#42815773)

We had the same problem.

Our solution: if the message makes it past all our spam filters and the only problem is that the sender's client isn't allowed by their SPF record, we send a friendly, plain-english informational message back to the sender, then deliver their message to our users.

"Please tell your company's IT staff that your mail server isn't set up correctly. This may cause your messages to be rejected or classified as spam. Please forward the following information to your system administrator: [SPF record] [sender's IP]. Thank you!"

That's nice if you don't rely on email for communication with your customers. Generally, customers (and potential customers) don't respond well to nagging -- especially if whatever you're asking is outside of their control. They may be too small to have anyone to complain to (and no idea how to fix it themselves), or they may complain to the corporate helpdesk who says "Nope, our server is set up the way we want it".

If I got an email like that from a vendor I was trying to reach, I wouldn't be doing service with that vendor.

One vendor that we were paying support to was trapping our entire domain's email in a spam filter - none of our emails could go through, we ended up faxing problem reports to them. This went on for 2 months before they finally admitted that there was a problem in their (outsourced) email spam filter, they ended up crediting us 2 months of our monthly support contract because their SLA promised 24 hour response to emailed support requests, and we didn't get any response at all.

Re:We don't reject, but we send some "helpful info (1)

neiras (723124) | about a year and a half ago | (#42817455)

If I got an email like that from a vendor I was trying to reach, I wouldn't be doing service with that vendor.

One vendor that we were paying support to was trapping our entire domain's email in a spam filter - none of our emails could go through...

I see what you're saying. Just wanted to emphasize that email messages are never dropped in our system, they are always delivered. The SPF check is still part of the spam checks, but by the time we get to the "send informational message" stage we've already decided the message isn't spam. We also check the sending IP against the Backscatterer RBL, which hopefully keeps us from sending to domains that have been hijacked.

The informational message is only sent once per week to any given sender. We do have an "don't email me about this again" link in the email. Finally, we don't send the message to anyone sending from a rather large commercially-maintained list of known webmail address and ISP mail servers, which exempts a large portion of senders from ever seeing the message (these users can't do anything to fix a broken spf record).

Re:We don't reject, but we send some "helpful info (1)

smash (1351) | about a year and a half ago | (#42815849)

Of limited usefulness. MD walks in "i'm expecting this email for a potential tender, the client tells me we are rejecting their email, and submissions close today!".

The MD receiving the odd spam is not a career limiting prospect. Missing a multi-million dollar contract due to non-delivery of business correspondence (or that reason being used as a scapegoat by your company's tendering department) potentially is.

Re:We don't reject, but we send some "helpful info (2)

GumphMaster (772693) | about a year and a half ago | (#42816627)

Relying on any form of email for multi-million dollar tender delivery is the career limiting move here.

Re:We don't reject, but we send some "helpful info (0)

Anonymous Coward | about a year and a half ago | (#42817107)

Email is normally considered more reliable than fax or mail, and it's much faster too. We typically sign things and run them through our copier, which emails out a scanned image of the document that we can forward on. As a bonus we can make sure that the signature was visible and the text legible.

The point is that it has to be that reliable because there is an expectation of reliability for a tool that people use more often than the telephone.

Re:We don't reject, but we send some "helpful info (1)

smash (1351) | about a year and a half ago | (#42817165)

Sure. But shit rolls down-hill, and it's not the MD who's going to get fired.

Re:We don't reject, but we send some "helpful info (1)

smash (1351) | about a year and a half ago | (#42817177)

Put it this way - in an ideal world, I agree with your premise. But the reality is that people expect email to work, and don't care WHY it failed. The fact that you bounced their legitimate correspondence is unacceptable to the business side of the company. The one that brings IN money.

Re:We don't reject, but we send some "helpful info (1)

MightyMartian (840721) | about a year and a half ago | (#42815975)

In other words you're generating soft bounces and making your server vulnerable to being used as an attack vector.

Re:We don't reject, but we send some "helpful info (1)

neiras (723124) | about a year and a half ago | (#42817343)

In other words you're generating soft bounces and making your server vulnerable to being used as an attack vector.

Nope, we do limits - one message per week per sender. So no effective attack vector.

Re:We don't reject, but we send some "helpful info (5, Insightful)

Bogtha (906264) | about a year and a half ago | (#42816557)

we send a friendly, plain-english informational message back to the sender

Please don't do this. One of the problems SPF solves is that spammers pick some random domain then spoof emails from that domain to send to millions of people. If you happen to be one of the unlucky people whose domain is targeted, you get a million bounces in your inbox.

The whole point of SPF is that if an email fails an SPF check, the email may not have come from the purported sender, and you should not treat it as genuine. You're completely undermining what SPF is for by doing this.

Re:We don't reject, but we send some "helpful info (1)

neiras (723124) | about a year and a half ago | (#42817379)

The whole point of SPF is that if an email fails an SPF check, the email may not have come from the purported sender, and you should not treat it as genuine. You're completely undermining what SPF is for by doing this.

Yeah, if everyone did this it would be bad - zillions of polite informational messages. You make a good point.

On one hand we're a small shop with a low-traffic email server. We've had good success getting sending domains to fix their record. On the other hand, perhaps we should go back to passive mode. I'll mention it to the system administrators.

SPF-SOFTFAIL = Supid (1)

krelvin (771644) | about a year and a half ago | (#42815571)

We actually mark on SPF checks on the outside but accept them. They then pass through a spam quarantine system which if they are SPF-FAIL or SPF-SOFTFAIL they are quarantined. Users can release them if they want to but they have to check their "Junk" reports to look for them or go to the portal itself. They have up to 15 days to release them or they are gone. However since these are a policy based rule, they can't white list senders that are messed up. Doesn't take long for a major vendor to learn they need to do email right.

Biggest problem is people with SPF records ending with ~ALL which basically means here is where we think our mail is coming from but if it is from somewhere else don't block it. Kind of stupid to not know where your email is being sent from. Especially annoying when a known company is setup like that that gets used as a phishing attack which is why we now quarantine SPF-SOFTFAIL now.

Google does whatever it does (1)

kwerle (39371) | about a year and a half ago | (#42815585)

Thanks, Google!

I can't be bothered to run my own damn SMTP/IMAP/etc any more.

Re:Google does whatever it does (3, Insightful)

smash (1351) | about a year and a half ago | (#42815861)

See megaupload. If you're a business, using cloud a service for email storage is just WAY too legally grey right now. IMHO.

Re:Google does whatever it does (0)

Anonymous Coward | about a year and a half ago | (#42815893)

Funny seeing geeks brag that they outsourced.

Re:Google does whatever it does (0)

Anonymous Coward | about a year and a half ago | (#42816061)

Funny seeing geeks brag about outsourcing.

Don't bother fixing the world, obviously (0)

Anonymous Coward | about a year and a half ago | (#42815621)

You can't fix everyone else's mail server and you can't fix everyone else's SPF records. SPF was a poorly thought out idea with a poor implementation, and it's never ever going to become relevant. The only thing it's good for is wasting your time like this.

If you have allo day to spend on this, you might try fixing the world's HELO responses too. You can catch tons of spam this way. And put your users out of business by blocking their essential mail coming from incompetently run mail servers.

Did you know? (1, Funny)

wbr1 (2538558) | about a year and a half ago | (#42815657)

Here is how I increased my SPF using one weird trick!

Re:Did you know? (0)

Anonymous Coward | about a year and a half ago | (#42816831)

~Obey!

Add receivers automatically (0)

Anonymous Coward | about a year and a half ago | (#42815691)

Give sender a check box to manually block it.

Automatically add all receivers to the SPF.

Depends on the weather (0)

Anonymous Coward | about a year and a half ago | (#42815705)

If it's going to be hazy out I prefer SPF 25. If it's full on sun I prefer SPF 40 or above.

DMARC (4, Interesting)

MillerHighLife21 (876240) | about a year and a half ago | (#42815735)

That's what DMARC is for. It let's companies specify exactly how to handle their SPF (and DKIM) rules based on how thoroughly they have covered their bases. The company I work for deals with a ton of phishing against our user base and implemented SPF, DKIM, and DMARC with great success.

Google has excellent documentation on the protocol.

Google (1)

Anonymous Coward | about a year and a half ago | (#42815759)

We just get Google to worry about spam for us.
We use Postini for archiving
We have low TTL MX records and a back up provider in case we have to worry about Google (again)
We have an account with Socket labs, in case Google screws up sending mail.
We have too few legitimate origins for maintaining SPF to be a problem

This works, is robust, piss easy to manage and dirt cheap. 150 - 200 employee company.

SPF is worthless, unfortunately (1)

realmolo (574068) | about a year and a half ago | (#42815761)

You CAN'T reject based on SPF, as you have learned. You'll lose too much valid mail.

So the best you can do is use it as part of a spam-scoring system, like SpamAssassin. Unfortunately, anti-spam systems that try to assign a "score", or try to analyze mail content, are ALSO worthless. Spammers have *completely* figured out how to get past any anti-spam system that analyzes content.

The sad fact is that the only way to effectively stop spam is to pay for an anti-spam service like Postini (which is going away, I hear), or buy something like an IronPort.

Basically, professionally-maintained, commercial blacklists are the ONLY really effective anti-spam system. If you are doing anything else, you aren't doing enough.

Re:SPF is worthless, unfortunately (1)

dbIII (701233) | about a year and a half ago | (#42815915)

Basically, professionally-maintained, commercial blacklists are the ONLY really effective anti-spam system.

I've been on the wrong end of a few of those since I have a mail server on an address that used to be a dynamic address something close to ten years before I got it and it gets blacklisted every now and again when people accidently dredge stuff out of old lists. To sum up those events, commercial rarely means "professionally-maintained" with spam blacklists.

Re:SPF is worthless, unfortunately (1)

smash (1351) | about a year and a half ago | (#42815931)

I manage to cut over 75% of spam before it gets to content filtering by greylisting, and the spamhaus zen block list. The vast majority of spam doesn't even get to my content filter.

No, it's not perfect, but if your boss isn't willing to shell out for an ironport or you're trying to do it yourself, cutting 75% out before it even sends the content to your server is a win in my book.

DMARC... (1)

Phoenix Rising (28955) | about a year and a half ago | (#42815771)

Add DMARC to your processing (http://www.dmarc.org/). It's a 'yes, I really meant it' notification for senders to communicate to receivers.

Other than that, the other suggestions already posted are about as good as it gets: use SPF as one element in scoring a message. Mark the message if your e-mail system allows it (e.g. label:authenticated in green, or label:authfail in yellow).

Yes, be a hardass about it. (1)

Anonymous Coward | about a year and a half ago | (#42815835)

Hells yes. I use postifx and reject connections that fail SPF with a 500 error. For example..

"550 5.7.1 : Sender address rejected: Please see http://www.openspf.org/Why?id=dassergey%40tadka.com&ip=103.22.182.230&receiver=mail.server.com%20:%20Reason:%20mechanism [openspf.org] "

I have found that most senders just forward the bounce message to their administrator who (you would hope) have the skills to rectify the problem.

I also use DKIM http://www.dkim.org/ [dkim.org] and DMARC http://www.dmarc.org/ [dmarc.org] . They're worth checking out too.

Reject them immediately (4, Insightful)

dshk (838175) | about a year and a half ago | (#42815837)

We reject mails which fail the SPF check immediately within the mail session. That is the only safe way, because then the sender will receive a bounce message from his own mail server.

We never received complaints regarding SPF rejects, but maybe because we do not have large incoming mail traffic.

Even if there were false positives, it would not hurt anybody, because the sender is guaranteed to be immediately notified that his message had not reached its recipient. He could contact us using a different method, not mail - in addition to complaining to his (so called) system administrators.

Re:Reject them immediately (0)

Anonymous Coward | about a year and a half ago | (#42816261)

Unless the email goes through a forwarder, which will then fail to bounce it back to the sender *unless* the forwarder has SRS set up. Very few mail servers have SRS setup up.

Never getting complaints because they never arrive in the right place does not count as successfully sending a complaint.

Re:Reject them immediately (2)

dshk (838175) | about a year and a half ago | (#42816735)

When a forwarder - actually any SMTP server - accepts a mail, than that means that it accepts the responsibility of either
1) delivering it, or
2) notifying the original submitter about the failure.

If a server fails to fulfill such a basic requirement of the SMTP protocol, than it is not an SMTP server, and anybody who depend on it is in serious trouble. And this is completely independent of SPF.

Regarding getting complaints: we have phone numbers and HTML contact forms. Our postmaster account is monitored and it is exempt of any kind of filtering. If somebody really wanted to complain then had the methods to do so.

Re:Reject them immediately (1)

Anonymous Coward | about a year and a half ago | (#42817331)

"When a forwarder - actually any SMTP server - accepts a mail, than that means that it accepts the responsibility of either
1) delivering it, or
2) notifying the original submitter about the failure."

This is not part of the RFCs.

Re:Reject them immediately (1)

dshk (838175) | about a year and a half ago | (#42817361)

Maybe you should read RFC 5321 more carefully:

"In other words, message transfer can occur in a single connection between the original SMTP-sender and the final SMTP-recipient, or can occur in a series of hops through intermediary systems. In either case, once the server has issued a success response at the end of the mail data, a formal handoff of responsibility for the message occurs: the protocol requires that a server MUST accept responsibility for either delivering the message or properly reporting the failure to do so (see Sections 6.1, 6.2, and 7.8, below)."

Re:Reject them immediately (1)

urbanriot (924981) | about a year and a half ago | (#42816523)

I don't believe I've ever heard of an SPF false positive failure but the sender would be notified with a bounce back, at which point the issue could be corrected, so I don't think false positives should be a concern.

Re:Reject them immediately (1)

Anonymous Coward | about a year and a half ago | (#42816859)

"We reject mails which fail the SPF check immediately within the mail session. That is the only safe way, because then the sender will receive a bounce message from his own mail server. "

This is false. The mail server that attempts delivery to your sever could be several hops along the mail path. There is no guarantee that any bounce message will be received by the sending party.

Re:Reject them immediately (1)

dshk (838175) | about a year and a half ago | (#42817345)

There is no guarantee that any bounce message will be received by the sending party.

No, there IS a guarantee that the bouncing message will receive the sending party. That is a basic requirement of the SMTP protocol.

Otherwise the sending domain in question has not only messed up its SPF records, but in addition it has a horribly broken mail system too. Maybe it worths keeping a safe distance from them. :)

Seriously, if we meet somebody with
1. broken SPF record and
2. broken mail system and
3. never noticed the above previously and
4. send us an important message and
5. negligent enough to not follow it up
then we will miss an important message.

But, that is a bit too much ifs, is not it? Honestly, I am more worried about the spam filtering heuristics in Thunderbird and in Gmail, not to mention other important things in life. I believe we benefit more from SPF than the risk it causes. But your mileage may vary.

Re:Reject them immediately (0)

Anonymous Coward | about a year and a half ago | (#42817237)

Bouncing is the appropriate response, especially for explicit hardfail. By publishing SPF, the sender domain takes responsibility for ensuring validity. it is Sender policy framework after all. The bounce will let them know either of client misconfiguration or a rogue or otherwise unmanaged sender.

Re:Reject them immediately (1)

GreyFoxx (13337) | about a year and a half ago | (#42817249)

> We reject mails which fail the SPF check immediately within the mail session

Try doing this as a colo/hosting provider servicing thousands of domains and hundreds of thousands of incoming emails (if not millions a day) from all all over the world and you quickly learn that immediately rejecting by SPF is something that can really only be done by small companies who get very little mail from few sources.

It's just not something a support desk is ready to handle since your hosting customers want the ability to send email from wherever is convenient so rarely want an SPF record in their own domain, and will scream bloody murder if you bounce "important" email because someone buggered an SPF record or never had one in the first place because they've never had anyone else reject their mail over it.

It's just not a viable option for large scale email/hosting.

Now as part of a scoring scheme like spamassassin, that's a different beast. Then it becomes at least slightly useful.

Re:Reject them immediately (1)

dshk (838175) | about a year and a half ago | (#42817479)

If you are talking about your customers, than you set their SPF record correctly, do you? So their mail will not bounced by others. If they want to use random email servers of the world, than you setup an "allow all" SPF record for them. Or you do not setup a record, but I think in this case you are doing a disservice to them, because their mail will be more suspicious. And no, if a domain has no SPF record, then its mails will not be rejected by any SPF compliant mail server.

Regarding spamassasin: that is a heuristics, in contrast to the well-defined SPF system. It is strange that you are afraid of SPF, but trust in a Spamassasin score.

However, I understand, that if you are neither a small organisaton, nor a big, influential company, like Google, then you may not want to take the effort to educate your users / other system administrators. But now, in 2013, maybe SPF is mature enough (thanks to the efforts of both the small and the large organisations), and it is not that big effort, and there are more benefits for customers than disadvantages, even in the short term.

SPF is authoritative. (0)

Anonymous Coward | about a year and a half ago | (#42815919)

If a site publishes an SPF record, they have requested that you take the record into account when accepting mail from their domain. Especially if they publish it as hardfail, you should definitely bounce any hardfail. If their SPF record is broken, the onus is on their site to deal with it, so bounce at the MX.

Try not to rely on true/false spam filtering (1)

FridayBob (619244) | about a year and a half ago | (#42815941)

Of course SPF filtering is not perfect; no one filter method ever is, thanks to all the badly configured mail servers out there. So, try not to build mail servers that reject incoming mail based on any one test, like one for SPF, or else you'll probably end up maintaining a lengthy whilelist, which will not only make you unpopular with the users, but is also a waste of time.

Instead, I built a mail system, using Exim (an extremely flexible MTA), that keeps track of a total number of 'strikes', as I call them, for positive scores for a number of filter categories, e.g. broken reverse lookup, bad HELO response, bad sender domain, blacklisted domain, no callout response, etc. Most of my filters are applied during the recipient phase of the delivery, before the MTA agrees to receive the message body (the data phase). If a message gets at least a certain number of strikes (in my case 3), it's rejected. If only 1 or 2, it ends up in the spambox of the intended recipient. These filter tests are separate from the ClamAV and spamassassin tests, both of which can still lead to an outright rejection.

In my experience, this method of counting strikes and using spamboxes to route messages to if they have only failed one or two filter categories has proved to be highly reliable and virtually maintenance free.

SPF Sucks (1)

dskoll (99328) | about a year and a half ago | (#42816041)

The idea is good, but in practice it's horrible. Here's what we do:

We never give a bonus to anything scoring SPF "pass" unless its from one of a few large domains that we know have good SPF records.

We add a small score (1 SpamAssassin point) for SPF softfail.

We add a moderate score in most cases for SPF fail.

For specific, often-phished domains like paypal.com and ebay.com, we add lots of points for SPF fail and softfail.

Huh? (0)

Anonymous Coward | about a year and a half ago | (#42816225)

How are you handling false positives caused by inaccurate SPF records?

The only "false positives" is accepting SPF -all when that is the policy the domain holder has in place. If you're going to check the records, do it right and reject at SMTP time.

We use it - but we keep postmaster@ out of it. (0)

Anonymous Coward | about a year and a half ago | (#42816383)

We use it - blocking on 'fail' and 'softfail'. We use ORF [vamsoft.com] , and the SMTP response explicitly includes "Contact postmaster@mydomain for help if needed" (and we've configured ORF to always whitelist emails to that address irrespective of SPF status.)

Yes, the spec is a bit broken (softfail?) but it's wider accepted than DKIM.

Educate your users, and SPF mungers (1)

jhealy1024 (234388) | about a year and a half ago | (#42816427)

We use Postini (now Google message security), and we turned on their SPF checks. Unfortunately, there are no knobs for this, so SPF pass is "good", SPF softfail is "bad", and SFP fail is "always quarantine, even if the address is whitelisted".

Despite some high-profile mail getting snagged by this, we educated our users and let them know that while we're more stringent with SPF than most places, it's not "our fault" that other people can't configure their domains correctly. After an initial spate of 10 or so messages where we had to call other IT departments to set them straight, it's more or less died down now and our users know how the problems get resolved.

We also created this page to send to the e-mail originator, so we didn't have to craft well-thought-out e-mail responses each time:

    http://web.suffieldacademy.org/~jhealy/spf.html

All of this makes for a little more work for us, but it has cut down on spam. Also, it makes the internet a little better of a place as we slowly drag other domains into compliance by catching their mistakes. ;-)

Re:Educate your users, and SPF mungers (0)

Anonymous Coward | about a year and a half ago | (#42816647)

I love all the snarky comments by asshole admins that work at a place that probably has more admins than needed. Sure, I bet you have all the time in the world to work on your email spf records when you charge more than twice the median yearly wage for tuition. Others (most others) work at places where the admins have to...

ah fuck it you loser.

Think of the senders! (1)

urbanriot (924981) | about a year and a half ago | (#42816511)

For the past year, one of our business domains has been a patsy for a continual bombardment of spam against the internet. Typically our catch-all account would receive at least 200 bounce back failures a day. Since then, we enabled SPF on our domain and it drastically increased the amount of bounce-backs considerably... which is good, as it means less people are seeing our domain in a malicious light as they're immediately denying the spam as it's caught by an SPF filter before it goes through other filters.

I encourage all mail server admins to enable SPF to hamper the ability of spammers to hit their targets.

I contract it out... (0)

Anonymous Coward | about a year and a half ago | (#42816567)

I find that usually Aveeno works the best, although Burt's Beeswax has some more natural ones that are better for the skin.

What the H* is SPF? (0)

Anonymous Coward | about a year and a half ago | (#42816603)

For those of us in the rest of the universe, what the H* is "SPF"? Sounds like some kind of inaccurate Spam Filter. ("FPS" is First Person Shooter games where you shoot at the enemy; maybe "SPF" is where the enemy shoots at you?)

Re:What the H* is SPF? (1)

dshk (838175) | about a year and a half ago | (#42816969)

SPF means Sender Policy Framework [openspf.org] . It is a method to specify and check the precise list of servers which are allowed to send mails in the name of a domain.

Receiving servers use it to check the validity of the reverse-path of the mail. Reverse-path is the address to where bounce messages should be sent.

It is nothing more, nothing less. It causes great confusion because many thinks that it is intended to be a final SPAM prevention method, when it is clearly not.

Alone, it makes sending SPAM only a bit more difficult. On the other hand it does make easier to catch spams with other methods. As more and more domains and receiving servers start to use it, it becomes more and more effective in this role.

The simplest benefit of setting up SPF for a domain that your domain name cannot be forged as the reverse-path of spam mails. So you will get less back-scatter spam and angry mails. SPF almost completely eliminated back-scatter spam for us, which was once quite an annoying issue.

SPF is designed to be implemented aggressively (1)

abelb (1365345) | about a year and a half ago | (#42816625)

SPF's great benefit is protecting your users against phishing attacks. Common phishing targets such as Paypal, eBay and Gmail, as well as banks and other finance organisations have very well managed SPF records. The onus for ensuring SPF records are correct falls on the sender, not the recipient. If someone inputs an SPF record for their server, then sends an e-mail via a different server you're doing exactly as requested by blocking the message. Problems of misconfigured SPF records will ease once more organizations implement it aggressively so that administrators will not be able to assume their mail is flowing correctly when delivery is initially successful to a few lax servers.

Dont forget Microsoft's lame attempt (1)

alanshot (541117) | about a year and a half ago | (#42816763)

I havent seen it in a while, but for a while we kept having issues with some implementations of Exchange refusing email based on our perfectly valid SPF records. The problem only occurred if we had both SPF AND SenderID (M$' so called "SPF 2.0") records for our domain. If we deleted our SPF records and only left the SenderID record, the mail would be delivered.

Way to go Microsoft in taking a perfectly good standard and deciding its not QUITE good enough and muddying the waters with your own crap.

Use SPF as a hint (1)

manu0601 (2221348) | about a year and a half ago | (#42816921)

You can use SPF as a hint. A bad SPF record could trigger longer greylisting delay, therefore lowering the ability of a spammer to reach destination before been known by blacklists. milter-greylist [hcpnet.free.fr] allows such setup.

Don't confuse anti-spoofing with anti-spam. (0)

Anonymous Coward | about a year and a half ago | (#42816925)

Something to keep in mind is that SPF isn't intended as an anti-spam technology, it's intended as an anti-spoofing technology, and it only really applies to the email envelope HELO/EHLO and MAIL FROM fields. The idea behind SPF is that if you have a valid SPF record, some third party shouldn't be permitted to send mail that claims to be you at the envelope level. However, that doesn't prevent someone from claiming to be you in the FROM field which normally isn't validated by SPF. In addition, there's nothing preventing a bad actor from setting up his own spam domain with a perfectly valid SPF record and spamming the heck out of you, and all their spam will pass SPF just fine. DKIM isn't much help either because that just validates that the mail hasn't been modified since it was originally sent, it makes no judgement about the quality of the content. DMARC is actually your best bet as an anti-spoofing technology, as it's designed to validate the authenticity of the email address' domain displayed to the user in the FROM field using SPF and/or DKIM, but again, it says nothing about the quality of the content. Even so, DMARC runs into issues because it can't validate the sender information that most email clients readily display, the so called from or pretty name. We get mail all the time from a major ISP's mail server that passes the various authentication acronyms, yet the pretty name claims to be another company's billing department. Also, many many forwarders, mailing lists, and social group services will break/violate one or more of these various schemes - so you need to keep that in mind as well. I can tell you from personal experience that SEVP's don't like their Facebook mail being dropped on the floor because they're using an imperfect forwarding service. "But Sir, Facebook says to reject it based on their DMARC policy!" doesn't work as a get out of jail card :-)

Load More Comments
Slashdot Login

Need an Account?

Forgot your password?