Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Do Data Center Audits Mean Anything?

samzenpus posted more than 2 years ago | from the who-certifies-the-certifiers? dept.

Hardware 84

1sockchuck writes "Data center service providers often tout certifications such as SAS 70, SSAE 16 and SOC 2 as evidence that they meet lofty operational standards. But some of these certifications are based on self-defined standards, and the entire situation is confusing and frustrating to customers, according to one critic, who says data center shoppers are poorly served by the jumble of acronyms and standards. Do these certifications matter when users are seeking data center space? Should they?"

cancel ×

84 comments

Sorry! There are no comments related to the filter you selected.

Not really (4, Informative)

gweihir (88907) | more than 2 years ago | (#38755396)

Now, if you get your hands at the detail reports, the audit result may actually tell you something, at least if the auditors are good. But the certifications pretty much only ensure minimal standards low enough to be meaningless.

Re:Not really (1)

ackthpt (218170) | more than 2 years ago | (#38755580)

Now, if you get your hands at the detail reports, the audit result may actually tell you something, at least if the auditors are good. But the certifications pretty much only ensure minimal standards low enough to be meaningless.

The first thing you'll probably discover is that 'Certificate' isn't spelled with an 'S', like all these acronyms are starting with. :-\

Re:Not really (2)

jhoegl (638955) | more than 2 years ago | (#38755634)

Much like HIPPA, where its "best effort" at a minimum.
Them:What? I zipped it with a password, doesnt that protect the patient information?
Me: Sure, for about 5 minutes. *doublefacepalm*

Re:Not really (1)

fafaforza (248976) | more than 2 years ago | (#38757120)

But it's disclosed to you, and based on that you can make a decision on whether to give a certain datacenter your business. (not sure if there's as much choice in the medical field, but that's a topic for another time)

Re:Not really (1)

MightyMartian (840721) | more than 2 years ago | (#38755784)

The certifications amount to "This shell company we created to report whatever we want it to says we're tight and secure. Yay for us!!!!!"

Re:Not really (1)

ackthpt (218170) | more than 2 years ago | (#38755852)

The certifications amount to "This shell company we created to report whatever we want it to says we're tight and secure. Yay for us!!!!!"

And if you pay enough for our services, we'll frame the certificate, as soon as it comes off the laser printer.

Re:Not really (2)

wiedzmin (1269816) | more than 2 years ago | (#38756506)

Laser printer?! Man, you're high-rolling it! Last certificate we got (at something like $15K cost) came as a PDF attachment in email. But you know what, it made customers happy...

Re:Not really (3, Insightful)

Fluffeh (1273756) | more than 2 years ago | (#38755786)

Now, if you get your hands at the detail reports, the audit result may actually tell you something, at least if the auditors are good. But the certifications pretty much only ensure minimal standards low enough to be meaningless.

It depends on the purpose of the audit. If the purpose is to appease middle managers and the like, then the auditors (good or bad) will be able to read the request "We need to ensure we are certified for [insert current buzzword]." and see that this is nothing short of an easy way to make a costly fee. If on the other hand, the request is to find ways to break into the systems and comes from sysadmins or the like, then it is much more likely that the company wants to patch vulnerabilities.

Business is business. If a sales person sees easy money walking into the office, they will probably sell them overpriced and needless goods/services. If they see someone who knows exactly what it is they want, they will more likely give them exactly what they ask for and for a reasonable price.

Re:Not really (3, Insightful)

eth1 (94901) | more than 2 years ago | (#38763226)

These aren't intrusion tests they're talking about but certification audits.

My experience with those (ISO, SAS, etc.) is that a company hires someone to write up a bunch of documents to match what the auditors want to see, and tell the employees where to find it. Then the auditors come and get told/shown what they want to hear/see so they'll go away and let us get back to real work. The documentation isn't looked at again by regular employees until the next audit.

Those certs are just like professional certs like MCSE, CCNA, etc. They don't really have any bearing on whether or not you're good at what you do, but they sound good to customers/employers.

Re:Not really (1)

hakioawa (127597) | more than 2 years ago | (#38755806)

And yet the minimal standards are much higher than most people actually do.

My take: Do audits and auditors do anything to secure your systems? Rarely. Do having real hoops to jump through and jobs/salaries/bonuses on the line for failure prompt people to try to secure their systems? Frequently.

These "audits" are bogus. (1)

Anonymous Coward | more than 2 years ago | (#38755418)

It's like when any of the US government agencies "audit" themselves for instance, The Federal Reserve. Remember that "audit" recently?

Re:These "audits" are bogus. (2)

hedwards (940851) | more than 2 years ago | (#38757212)

Bad example, the Federal Reserve is a private entity.

ahem! (1)

Grindalf (1089511) | more than 2 years ago | (#38755444)

Ooooo, has someone failed a standard test? Give us the rest of the dirt, pleeeeeeeese!

No. (0)

Anonymous Coward | more than 2 years ago | (#38755466)

No. It means absolutely nothing. I have done the process myself and it has absolutely nothing to do with security as far as I can tell. (read: its a complete joke)
 

It's better than no cert at all (3, Interesting)

NemoinSpace (1118137) | more than 2 years ago | (#38755480)

not entirely unlike MSCE, but less so.

Re:It's better than no cert at all (1)

gregsmac (945663) | more than 2 years ago | (#38755516)

MSCE? Way to poop on a cert you cant even spell.

Re:It's better than no cert at all (1)

Anonymous Coward | more than 2 years ago | (#38755646)

No, he obviously meant Master of Science in Clinical Epidemiology (MSCE) [upenn.edu] Whoo boy, that cert isn't worth the paper it's printed on!

Re:It's better than no cert at all (3, Funny)

jhoegl (638955) | more than 2 years ago | (#38755766)

Yeah, how dare he use actual logic instead of Microsoft Logic to acronym!

Re:It's better than no cert at all (1)

viperidaenz (2515578) | more than 2 years ago | (#38756240)

Actual logic would make Microsoft Certified Systems Engineer MCSE though... I think the point is to make it a FFLA [urbandictionary.com] because they're superior to a TLA [urbandictionary.com]

Re:It's better than no cert at all (1)

NemoinSpace (1118137) | more than 2 years ago | (#38756184)

Actually, i was going for the "i can't spell MCSE but i are 1" gag. But my mind thinks faster than i type and something got lost in translation. No offense to cert holders out there - well maybe some.

yes (5, Funny)

nimbius (983462) | more than 2 years ago | (#38755490)

without data center audits thousands of datacenters across the country would have to forego tiny wooden plaques with things like "SAS70 CERTIFIED!" and "SSAE 16 READY!"
and I as a sysadmin would have to stop making the joke, "SAS70? oh thats for when we change the motor oil in the cloud."

Re:yes (0)

Anonymous Coward | more than 2 years ago | (#38756100)

You've obviously never seen a SAS70 type 2. The auditors remarks on control effectiveness are in particular useful.

Re:yes (0)

Anonymous Coward | more than 2 years ago | (#38757130)

The problem is that companies are rarely willing to release any actual audit data to their customers. They run around say "We're SAS 70 Type II certified!" but that's completely meaningless without access to the data. In short, these audits are genuinely good for businesses that actually take them seriously and utilize the data to make real internal improvements. That said, most businesses simply seem to use them to go through the hand waving motions required to appease ignorant customers.

Yes, I have seen both sides firsthand.

Re:yes (1)

narf (207) | more than 2 years ago | (#38758194)

Really? Wow. I've never had a company claim they have a SAS70 type II (or whatever it's called now) but refuse to release the actual report ... that just seems so non-sensical. Not having a SAS70 isn't the end of the world, but lying about it certainly would be.

Re:yes (0)

Anonymous Coward | more than 2 years ago | (#38764294)

Concur. SAS 70 reports (event point-in-time type I's) are available. Either auditor or compnay controlled, normally under some form of NDA.

I'd be leery of any entity that says they have passed SAS 70 and cannot produce the report.

Re:yes (0)

Anonymous Coward | about 2 years ago | (#38771734)

I've personally requested the actual reports from four different datacenter providers who were proudly proclaiming that they were "SAS 70 Type II certified." All four refused to release the reports.

Re:yes (1)

itcontrolsfreak (2557520) | more than 2 years ago | (#38766552)

Actually, the service organization is only supposed to release the SAS 70 report (and the newer SSAE 16, SOC 1) to existing customers and their auditors. It is not a "general release" report and should not be used for marketing purposes. This language is in the SSAE 16 standard and should be in the engagement letter with the CPA firm that conducts the attestation.

Re:yes (0)

Anonymous Coward | about 2 years ago | (#38771760)

I've personally requested the actual reports from four different datacenter providers who were proudly proclaiming that they were "SAS 70 Type II certified." All four refused to release the reports. These were very large datacenters, including one you've doubtless heard of (and probably visit sites hosted on servers that reside there five times a day). This was during my employment with a hosting company that has a very large presence in said datacenters.

Re:yes (1)

jamstar7 (694492) | more than 2 years ago | (#38757790)

Oh, where they talk about spending the extra 3 bucks for the heavy duty triple oil filter?

Re:yes (0)

Anonymous Coward | more than 2 years ago | (#38757168)

Motor oil in the cloud. Brilliant.

Re:yes (0)

Anonymous Coward | more than 2 years ago | (#38758982)

The truth is, it's a volleyball club founded in 1970 in the town of Uithoorn in the Netherlands. SAS means Samen Altijd Sterk, Together Always Strong.

Uses for Audits/Certifications (4, Insightful)

ackthpt (218170) | more than 2 years ago | (#38755548)

  • - Waving in face of prospective customers - ' Yes w certainly a certificate of certification granting certitude!'
  • - Finding things you actually did right
  • - Finding things you need to fix or wallpaper over
  • - Creating gainful employment for auditors, certifiers, pencil pushers, paper shufflers and rubber stampers.
  • - Sell more seminars and books for a certification industry
  • - Influence government to require certain certifications to keep an industry of auditing and certification on the gravy train for years
  • - Give significantly less benefit to people who disagree with the need for dubious audits and/or certifications.

Re:Uses for Audits/Certifications (1)

swb (14022) | more than 2 years ago | (#38755728)

Rent-seeking, it's not just about buying rack space.

Re:Uses for Audits/Certifications (0)

Anonymous Coward | more than 2 years ago | (#38759734)

wallpaper over... I won't say that we "fixed" things by temporarily stopping services on servers when we *requested* to get audited . We knew the audits were coming and the CEO would tell us to just smoke screen a ton of things. Basically the certs are meaningless. The one thing they do is keep criminal rift-raft out ... most of these certs clauses that say no felons can be employed within the company.

should they? no. do they? not to me. (2)

Narcocide (102829) | more than 2 years ago | (#38755570)

The fact of the matter is a lot of stupid certification acronyms were specifically designed to allow spenders to make decisions without being actually informed in any way about what they're spending their money on. That's actually the *point* here. The problem isn't the certifications, the problem is that to make an informed decision about which ISP should host your servers you shouldn't be the type of CTO who insists on using outlook express and ie6 still and can't even configure their own email client. You need to know bandwidth from ass-width.

Re:should they? no. do they? not to me. (0)

ackthpt (218170) | more than 2 years ago | (#38755738)

The fact of the matter is a lot of stupid certification acronyms were specifically designed to allow spenders to make decisions without being actually informed in any way about what they're spending their money on. That's actually the *point* here. The problem isn't the certifications, the problem is that to make an informed decision about which ISP should host your servers you shouldn't be the type of CTO who insists on using outlook express and ie6 still and can't even configure their own email client. You need to know bandwidth from ass-width.

I couldn't help but notice this post wasn't ESU 77A Certified - sign up for our Seminar[www.joesbarandtrainingcenter.com] only $1,500, availibility is limited

Like ISO 9000 (1)

Tteddo (543485) | more than 2 years ago | (#38755622)

Yeah, I paid the bribe. Now I am certified!

Re:Like ISO 9000 (0)

geekboybt (866398) | more than 2 years ago | (#38755660)

Why not come up with a similar certification, one that's available under Creative Commons? That way, anyone could actually read the specifications, and you can go so far as to have a third party certify the results, either by self-certifying and having that notarized, or having a trusted third party perform the audit.

Re:Like ISO 9000 (1)

marcosdumay (620877) | more than 2 years ago | (#38756004)

Will that certification be meaningless to not create any kind of trouble, and yet obscure enough for potential clients not discovering it is meaningless?

Re:Like ISO 9000 (1)

iluvcapra (782887) | more than 2 years ago | (#38756106)

The actual ISO 9001 standard only costs about $130 on the ISO's website, the verification labor is by far a bigger part of the cost. Nobody foregoes ISO 9001 certification on account of the non-libre status of the standard.

Re:Like ISO 9000 (0)

Anonymous Coward | more than 2 years ago | (#38757022)

That would remove the profit from the ISO 9000 people. One company I worked for paid $10,000 and a client I have now paid $5000. I actually wrote up some docs and when you are really vague, they are really happy.

SOPA - Our OWNERS continue to CONSUME our ENERGY (-1, Flamebait)

Anonymous Coward | more than 2 years ago | (#38755628)

Carlin - The Real Owners Of America

"The real owners are the big wealthy business interests that control things and make all the important decisions. Forget the politicians, they're an irrelevancy. The politicians are put there to give you the idea that you have freedom of choice. You don't. You have no choice. You have owners. They own you. They own everything. They own all the important land. They own and control the corporations. They've long since bought and paid for the Senate, the Congress, the statehouses, the city halls. They've got the judges in their back pockets. And they own all the big media companies, so that they control just about all of the news and information you hear. They've got you by the balls. They spend billions of dollars every year lobbying  lobbying to get what they want. Well, we know what they want; they want more for themselves and less for everybody else."

"But I'll tell you what they don't want. They don't want a population of citizens capable of critical thinking. They don't want well-informed, well-educated people capable of critical thinking. They're not interested in that. That doesn't help them. That's against their interests. They don't want people who are smart enough to sit around the kitchen table and figure out how badly they're getting fucked by a system that threw them overboard 30 fucking years ago.

"You know what they want? Obedient workers  people who are just smart enough to run the machines and do the paperwork but just dumb enough to passively accept all these increasingly shittier jobs with the lower pay, the longer hours, reduced benefits, the end of overtime and the vanishing pension that disappears the minute you go to collect it. And, now, they're coming for your Social Security. They want your fucking retirement money. They want it back, so they can give it to their criminal friends on Wall Street. And you know something? They'll get it. They'll get it all, sooner or later, because they own this fucking place. It's a big club, and you ain't in it. You and I are not in the big club."

"This country is finished."

short answer: no (4, Informative)

Anonymous Coward | more than 2 years ago | (#38755632)

I'm a work at a somewhat large financial services company that provides customer information to various other large financial institutions (chase, wells fargo, capital one, amex, discover, just to name a few). We receive this customer information from pretty much everywhere - those self same banks, government agencies, credit card companies, universities. Basically, if you've ever had a loan or grant, credit card, bank account, paid a utility bill, child support or been in prison then we have that data. Your address, phone number, social security number, bank account information, etc.
    The majority of this information is stored unencrypted on systems that are accessible to any employee, often with 777 permissions. While the majority of the systems are patched pretty regularly, many aren't. I recently had to convert over an old apache 1.3 server that hadn't been patched since 2006 - there's another similar server that is regularly used by outside contributors to drop off customer information.
    We have customer facing IPlanet servers that haven't been patched since 2004 - the software isn't even under support anymore.
    We have session recording software on our unix servers that is so ridiculously trivial to bypass that the company that sells it (centrify) should be ashamed to sell it.
Yet we've had PCI certification for 3 years, we've passed the SAS70 certification every time - they are rubber-stamps, nothing more.

Re:short answer: no (1)

Lotana (842533) | more than 2 years ago | (#38756736)

Basically, if you've ever had a loan or grant, credit card, bank account, paid a utility bill, child support or been in prison then we have that data. Your address, phone number, social security number, bank account information, etc.
        The majority of this information is stored unencrypted on systems that are accessible to any employee, often with 777 permissions. While the majority of the systems are patched pretty regularly, many aren't. I recently had to convert over an old apache 1.3 server that hadn't been patched since 2006 - there's another similar server that is regularly used by outside contributors to drop off customer information.

Well, looks like I won't be able to sleep ever again.

What country is this in? My only hope is that it is not where I reside and things are much more secure here. Please don't shatter my illusion! Looks like ignorance really is bliss.

Re:short answer: no (2)

skelly33 (891182) | more than 2 years ago | (#38757378)

My understanding is that SAS-70 is entirely self-defined. The point of the certification is to validate that your company has established operational processes and procedures for itself, and that the processes and procedures established are adhered to.

Nothing about SAS-70 requires any measure of quality or completeness. When the business claims, "yes, we have a disaster recovery plan. yes, we have a business continuity plan. yes, we have a backup, fault tolerance, order handling, fulfillment, budgeting, auditing, security process, etc." being SAS-70 compliant implies that those claims were audited by an independent third party and found to be true statements. It does not mean that any one of those plans is worth a hill of beans, but at least they're there.

That said, while I am actively involved in such planning for my company, we do not pay for certification because we are selective about who we give our money to. This is just to say that passing SAS70 certification (AC parent) is thoroughly unrelated to the state of sin that the operational systems are in.

Re:short answer: no (0)

Anonymous Coward | more than 2 years ago | (#38758014)

Apart from mere controls, there is also a control objective that these controls are supposed to satisfy. Any competent auditor would first look at the control objective and only then look at the controls to verify if the controls meet the objective. They would also check the sanity of the control objective before testing the controls. If your control objective does not make sense, then the auditor for whom this report is being prepared can very well decide to terminate their contract with you based on the grounds that your processes for securing their data is unreliable. You could lose clients this way.

Re:short answer: no (0)

Anonymous Coward | more than 2 years ago | (#38759220)

That's absolutely correct. SAS70 is a worthless auditing standard - it doesn't even come anywhere near IS0 2000:2001 or ISO 20000 in its rigour. All they're interested in is you've written down procedures and follow them. Most internal audit teams treat it as a joke - but it persists in US firms for some reason.

Re:short answer: no (1)

itcontrolsfreak (2557520) | more than 2 years ago | (#38766670)

SAS 70 is a great, but ultimately obsolete audit standard. It is NOT a data center security and availability standard. Again, SAS 70 and the new SOC standards are attestation standards. That is, the standard is about how to conduct the review, NOT what should be in place at the data center. This is totally different than ISO and other "certification" bodies. The AICPA is not a certification body. I believe what you intended is that SAS 70 is a worthless vehicle for certifying the security and availability controls of a given data center. And you would be correct. For that was never the intention of SAS 70 and is STILL not the intention of SSAE 16 (SOC 1).

Re:short answer: no (0)

Anonymous Coward | more than 2 years ago | (#38757972)

Who performed the SAS 70 and PCI reviews? I am sure no self-respecting auditor would certify you if you had those kind of gaps in your system.

Re:short answer: no (0)

Anonymous Coward | more than 2 years ago | (#38762290)

Report it to your proper chain of command/org/etc or gtfo. Seriously.

Re:short answer: no (1)

hugetoon (766694) | more than 2 years ago | (#38769536)

And Your QSA is ?

Re:short answer: no (1)

NetNinja (469346) | more than 2 years ago | (#38771278)

Yes your QSA should be fired and your company should be fined for running antiquated software that has security holes the size of the grand canyon.

Of course it matters (4, Informative)

ZouPrime (460611) | more than 2 years ago | (#38755672)

Well, it certainly matter for regulation purpose. If you handle data that need to be covered under a specific standard (say, PCI), you'll seek out a certified data center. In this context, the certification isn't about security, it's about risk transfer. It's the provider who become liable if there's a breach if it can't show to have respected the standard properly.

Now as security references, they certainly have their problems. We can take solace in the thought that they help enforce the bare minimum at the very least. As a security professional, I would say their best benefit is how well they can be used as a big stick, "encouraging" management to perform necessary changes. It's a hard sell to convince an average manager to invest in security for the sake of security. But if there's a legal penalty associated with whatever standard must be put in place, as well as a big dollar sign attached to it, they'll suddenly start to listen. That's a language they understand.

Need to see the criteria (2)

mclearn (86140) | more than 2 years ago | (#38755700)

I've always been amazed at things like SAS 70 which, as the poster states, is based on self-defined criteria. The most shocking part, if I recall correctly, is that the criteria are not publicly consumable! This is the worst part of it all and the key part which needs to change.

Makes the customer happy. (0)

Anonymous Coward | more than 2 years ago | (#38755732)

'nuff said.

Better Measurement of Success (2)

Herkum01 (592704) | more than 2 years ago | (#38755758)

The problem is that they are trying to get certifications when what they really need are Achievements! Just ask Microsoft, it worked for the XBOX, they are throwing it into Visual Studio, it will work for data centers.

Include a Facebook Like button and a Twitter link and your done!

Would you put your money in a non-FDIC bank? (2)

Hadlock (143607) | more than 2 years ago | (#38755760)

Just like when hiring a new employee, you look for certifications and credentials. When trying to separate the legitimate companies from the fly-by-nights, seeing audits every year going back 2, 3, 4 years can help verify that they've been around for a while. Datacenter space isn't cheap, and if you find a good deal you want to make sure that your server with all the company data on it is still there on monday morning, and not on a plane to China.
 
You put your money in an FDIC-insured bank account because it's registered with someone who's taken the task of keeping an eye on their registrants and staked their reputation on it. If you don't agree with that, I have an offshore bank account you can transfer some money in to for the International Bank Of Hadlock, we just opened yesterday, but we offer 3000% interest daily and don't keep money laundering records.

Re:Would you put your money in a non-FDIC bank? (1)

SuperQ (431) | more than 2 years ago | (#38755888)

I have some of my money go through a non-FDIC credit union. They have a non-federal deposit share insurance provided by http://www.americanshare.com./ [www.americanshare.com]

Re:Would you put your money in a non-FDIC bank? (0)

Anonymous Coward | more than 2 years ago | (#38756192)

What's their relationship to the NCUA?

Only as good as the auditor (2)

hawguy (1600213) | more than 2 years ago | (#38755772)

I'd say that data center audits, just like financial audits, are only as good as the auditor. If you're a big enough client, the auditor will say pretty much any thing you want:

http://en.wikipedia.org/wiki/Arthur_Andersen#Demise [wikipedia.org]

But still, I look for the certifications to cover my butt. Of course, that's what all of these standards are about - just saying that you've implemented procedures to cover your butt. It doesn't matter whether or not the procedure actually does anything worthwhile or even if there are big gaping holes elsewhere. As long as you can say you've implemented it fully, then you're covered.

These are NOT Certifications! Nobody Gets This! (1)

Anonymous Coward | more than 2 years ago | (#38756016)

There is a number of problems with how data centers make these statements and what people interpret.

The main problem is that people say things like "SAS70 Certified". That is terribly bad wording. There is no such thing. The SAS70 (now SSAE16 or SOC1 report) is not a certification. There is no preset/predetermined criteria that is universal to all companies that receive such a report. Each report is specific to that particular company/data center. It's almost like saying I have a diploma as an independent study major.

The next thing is that these reports are not intended for public use. These are auditor-to-auditor reports. They are meant for the auditor for a company that uses said data center (or other service provider) to rely on and not need to audit the data center itself. That is why auditors review these reports to make sure it contains the provisions it's looking for. Otherwise, they're going to go in and audit the data center.

Companies that get such reports tend to use it as a marketing tool to show potential customers, when that isn't the purpose. To reduce some blame, I've known auditors guilty of telling data centers that they can do that so that they could convince the data center to pay for the service.

Also, SAS70 was designed to reflect controls at a service provider that impact or relate to the processing of financial data, which would have an effect on the financial statements that the auditor is reviewing. Most data centers don't process data (the customers that host stuff there do and they need the SAS70). However, over the years, people have convinced themselves that because the data physically resides at the data center, they impact the financial statements and so they should get a SAS70. This is however, not really true, since with good security controls around the data, the physical hosting of it won't materially misstate the financials. It was for this reason that the AICPA split the old SAS70 into 3 separate services: SOC 1 (SSAE16) which is what the old SAS70 was meant to be, SOC 2, SOC 3. The latter 2 are geared more toward data centers and technology firms that don't impact financial data.

The seals that are issued by the AICPA just state that you've had a report done. They do not speak to the content of the report. I could get a SOC report that just says "All employees are entitled to free breakfast". The auditor I hire will come in and test/verify that and then will sign-off saying that they agree. I now have such a report and can boast "SAS70 Certified" everywhere, which doesn't mean squat.

It only matters to the company itself, the company that uses their services (depending on context), and the auditors of the company that uses their services.

These are NOT Certifications! (5, Informative)

DaCurryman (1116593) | more than 2 years ago | (#38756060)

There is a number of problems with how data centers make these statements and what people interpret. The main problem is that people say things like "SAS70 Certified". That is terribly bad wording. There is no such thing. The SAS70 (now SSAE16 or SOC1 report) is not a certification. There is no preset/predetermined criteria that is universal to all companies that receive such a report. Each report is specific to that particular company/data center. It's almost like saying I have a diploma as an independent study major. The next thing is that these reports are not intended for public use. These are auditor-to-auditor reports. They are meant for the auditor for a company that uses said data center (or other service provider) to rely on and not need to audit the data center itself. That is why auditors review these reports to make sure it contains the provisions it's looking for. Otherwise, they're going to go in and audit the data center. Companies that get such reports tend to use it as a marketing tool to show potential customers, when that isn't the purpose. To reduce some blame, I've known auditors guilty of telling data centers that they can do that so that they could convince the data center to pay for the service. Also, SAS70 was designed to reflect controls at a service provider that impact or relate to the processing of financial data, which would have an effect on the financial statements that the auditor is reviewing. Most data centers don't process data (the customers that host stuff there do and they need the SAS70). However, over the years, people have convinced themselves that because the data physically resides at the data center, they impact the financial statements and so they should get a SAS70. This is however, not really true, since with good security controls around the data, the physical hosting of it won't materially misstate the financials. It was for this reason that the AICPA split the old SAS70 into 3 separate services: SOC 1 (SSAE16) which is what the old SAS70 was meant to be, SOC 2, SOC 3. The latter 2 are geared more toward data centers and technology firms that don't impact financial data. The seals that are issued by the AICPA just state that you've had a report done. They do not speak to the content of the report. I could get a SOC report that just says "All employees are entitled to free breakfast". The auditor I hire will come in and test/verify that and then will sign-off saying that they agree. I now have such a report and can boast "SAS70 Certified" everywhere, which doesn't mean squat. It only matters to the company itself, the company that uses their services (depending on context), and the auditors of the company that uses their services.

Re:These are NOT Certifications! (0)

Anonymous Coward | more than 2 years ago | (#38758134)

I agree. SAS 70 and SSAE 16 are general guidelines to be followed by an independent auditor to assess the controls related to financial transactions that are performed by a service organization. Using a SAS 70 report as a certification is a dubious claim by the service provider.

Re:These are NOT Certifications! (1)

itcontrolsfreak (2557520) | more than 2 years ago | (#38766830)

Well said DaCurryman! Data centers have adopted SAS 70 and SSAE 16 as a certification of good security and availability practices, however, that was never the intent. The reason we had "SAS 70 Certified" data centers and we now have "SSAE 16 Certified" data centers is because the customer is always right. The chain of demand for SAS 70 began with financial statement auditors that needed a vehicle to understand the controls at service organizations. Sarbanes Oxley fundamentally changed the requirements for financial statement audits. The auditors now had to have an understanding of the controls that were in place over financial reporting. That included IT general controls like physical and environmental controls that most data centers provide. Rather than send a team of auditors to examine the physical and environmental controls at a third party data center, the audit firm asked the data center to provide a SAS 70 report. Pretty soon, the marketing people said "Hey, we can get more customers if we say we are "SAS 70 Certified" and since SSAE 16 was officially introduced as the replacement for SAS 70, you now have those same marketing people claiming SSAE 16 Certified.

How much is PHB speak that what people who don't (1)

Joe_Dragon (2206452) | more than 2 years ago | (#38756092)

How much is PHB speak that what people who don't know that they are running come up with BS to make it look like they do.

Re:How much is PHB speak that what people who don' (0)

GreyWolf3000 (468618) | more than 2 years ago | (#38756258)

I have no idea what that sentence means.

If you want security and reliability... (4, Insightful)

jafo (11982) | more than 2 years ago | (#38756232)

Security and reliability are processes, they are not something you can do once and then forget about. So, yes, I would say that having regular audits are a useful thing. As far as whether these specific standards are useful, the facility we have most of our servers in we have been in since before their SAS 70 audit, and their procedures were good before, but there's a noticeable improvement after. Things like a man-trap with a live security person comparing you with your on-file photo before you enter the raised floor, 2-factor auth on all doors rather than just on the key doors, maintenance lock-outs displayed more prominently, EPOs installed (not a benefit to me, but they did put alarmed doors around the EPOs to prevent the common problems).

As far as it being "based on self-defined standards", I'm ok with that. I'm ok with the requirement being that they *HAVE* standards for certain things rather than dictating what exactly those standards are. One size does not fit all, but having standards for what you do, I have found in my own business, improves quality.

Re:If you want security and reliability... (0)

Anonymous Coward | more than 2 years ago | (#38756838)

It comes down to the intent and desires of management.

If the data centre management are just looking to tick the boxes, it doesn't matter whether the audits are regular or not - they'll be effectively useless.

If, on the other hand, data centre management are serious about providing a reliable and secure service to their customers, the audits will (or should be) more thorough and detailed, and management will follow through on the recommendations. Things like reviewing who has access to the data centre on a monthly basis, removing dead wood at least that frequently (if not more so); securing the entrances; keeping customers' systems in separate cages.

As you say: it's all about the processes and how they're followed. Audits are ways to improve that, if and only if that's what management want. No audit will fix problems if management doesn't follow through.

And here's the kicker: if the customers are serious about security, they should be reading the audit reports in detail - both before they hire floor space (or rack space) in the data centre, and whenever they come out. If they don't want to do that, they only have themselves to blame when things go wrong. It's not easy. Never has been. Never will be. Taking shortcuts in this sort of case will bite you later on.

sas70 quality (1)

Anonymous Coward | more than 2 years ago | (#38756544)

SAS70 and the new SSAE 16 require that the assessor and the reader both recognize the limitations and the scope of the work. Like many have said, its dependent on the quality of the auditor. This is why there are a "Big 4" in the audit world, their name carries weight to the quality of review. We will exclude the perceptions about the quality that may actually be provided. When I did SAS70 and similar audits, one of the first things i look at is the company that performed the review, the quality of the finished product (format, style, mistakes, sentence construction.) SAS70, HIPAA (HIPPA was when it was in draft) certified or similar "certified" is bs. No such thing exists. The controls selected for testing are decided by the hosting company, not by the reviewer. This requires the secondary auditor to evaluated the the scope of controls and quality of the controls tested against the controls they need to evaluate. Since many that receive a SAS70 are not actually the data processors, the assessment usually only provides some comfort in the effectiveness to the physical access and system availability (UPS, AC, redundancy).

like all audits (1)

BlindRobin (768267) | more than 2 years ago | (#38756822)

rarely

Conversation with an Auditor (1)

thinktandem (1639931) | more than 2 years ago | (#38756978)

Me:"So you don't care if the written procedure is a valid method to accomplish practical outcome, as long as I follow the procedure as written." Auditor: "Yes." That pretty much sums up their real world value.

Re:Conversation with an Auditor (2, Insightful)

Anonymous Coward | more than 2 years ago | (#38757532)

I don't understand why people have such a hard time with this Audit concept. In these cases, an Auditor audits your processes as defined by your management. It doesn't matter what your "process" does in real life. That is NOT what an Auditor is checking. You are not being graded on what you do or how you do it. An IT/Financial/Process Audit is NOT an employee performance review. Something that Engineers and Programmers can't seem to get through their thick heads.

An Auditor is providing a report to the readers that what is documented is being followed. There is also another part of the Audit where what is written is checked, but that has been too subjective for my tastes too.

Anyway, what you should have done is talked to your management that the bloody documentation needs to be updated, cause your stockholders, board, & upper management think you are doing something totally different from what you really are.

I used to have a LOT of conversations over the years similar to the above. From a former auditor point of view (with a C programming & process designer background) this is how that conversation sounds to us:
Me: How hot do your servers get in the DC?
Tech: We got Quad core blades running at X GHz.
Me: That's nice, but I need the info to design the cooling systems.
Tech: Dude, we got multiple quad core systems; we never even come close to capacity. How is that not good enough?!!?
Me: Irrelevant, I just need to know how hot your systems run so I can design the optimum cool solution!
Tech: But nothing is crashing, we are doing great! I don't see the problem.
Me: I don't think you are understanding me. I ....

The answer you need to show your boss (4, Informative)

colonel (4464) | more than 2 years ago | (#38757108)

Right here, pure gold: http://www.gartner.com/it/page.jsp?id=1400813 [gartner.com]

Read that 5 times, carefully, and then get your bosses to do the same. Seriously.

SAS70 is a *questionnaire* that the vendor completes, and then the auditors just go in and confirm that their answers are correct.

So I could say "we don't do backups" in my answer to the questionnaire, the auditors would verify that I didn't do backups, and I'd "complete" the SAS70 process (not a certification!) successfully.

It is the client that is resoponsible for reviewing the questionnaire and ensuring that the audited answers are sufficient for the needs of their business. That's called "vendor management" and is a core practice area in ITIL.

Re:The answer you need to show your boss (1)

Glendale2x (210533) | more than 2 years ago | (#38757514)

Pretty much. If your procedure is that you never do backups, or don't have redundancy, and the auditors verify that as fact: ta-da SAS70 seal of approval.

Re:The answer you need to show your boss (1)

Skapare (16644) | more than 2 years ago | (#38758316)

We are way ahead of you! We did a backup once, and threw it away!

Yes, they do mean something ... (1)

RockDoctor (15477) | more than 2 years ago | (#38758748)

... they mean that the data centre in question definitely does not want customers to come round and inspect the place themselves, and possibly ask awkward questions.

The presence of a certification unaccompanied by an invitation to come and look over the place yourself should be a pretty good warning.

only if it involves testing (1)

tendergluttony (2556588) | more than 2 years ago | (#38759058)

in my humble experience, especially on the continuity area, certs mean something if management assertions are really tested in real life like scenarios... a data center may have adequate UPS and 2 generators but it may not protect the "system" from pure human idiocity. So in a real life case, once, when power failed, UPS kicked in until generators started. However BOTH generators started and due to high power surge current protectors shut both down at the same time... what do you say to that?

Re:only if it involves testing (1)

afidel (530433) | more than 2 years ago | (#38762196)

What I say is someone didn't know what they were doing. Why would you have both generators powering a single feed? It makes MUCH more sense to have completely physically independent A+B paths that only share a common ground/earth. Heck in that scenario they must only have a single ATS which in my experience is the least reliable part in an emergency power system (generators can be finicky but in a well run center they are tested weekly so you should know when they are acting up, most places don't test their ATS regularly and if they do then they're actually making the reliability worse as the contactors wear down).

The bigger picture (1)

hrieke (126185) | more than 2 years ago | (#38760078)

Certainly at the worker bee level we all can agree that most certs are not worth the paper that they're printed on, however when you start looking at what the company needs and requires then, yes, the certs do provide some value.
Imagine signing a multi-million dollar contract with a data center. Part of the contract are clauses for things like QOS, DR, and a whole host of other very tiny details which are so, so important in the contract. Now, lets say that the data center goes down (fire for example), along with it your business. Now you're out major bucks, you will be turning to your lawyers and asking if you can sue for breach of contract. The lawyers are going to review the contract, see what the certs mean, review the certs and the data center's answers, and then tell you if you do or do not have a case.
And that's why the certs are important. Because they spell out exactly what the data center will do in case of [fill in the blank], and gives the data center coverage which is then enforced by your contract.

SAS70/SSAE16/SOC2 Certifications are Fictitious (0)

Anonymous Coward | more than 2 years ago | (#38760294)

Firstly, there is no such thing as a SAS70, SSAE16, or a SOC2 Certification. Please watch this video from the president and CEO of the AICPA here http://bit.ly/yO6bgc

On July 14, 2010, Gartner predicted that "By 2012, No Customers of Cloud Providers Will Accept SAS 70 Alone as Proof of Effective Security and Compliance" 9http://bit.ly/wEt2i5). They were right, but they were also wrong. People are accepting SSAE 16 (SOC1) as proof of security, and SOC1 is essentially the same as SAS70. In the first year of SOC reports, CPA firms rushed out to be the first ones to do one of these new engagements, and they stepped in it big time. They included security related non-ICFR (internal controls over financial reporting) controls in the SOC1 reports they issued in violation of the attestation standards. I feel bad for the datacenters that have announced that they are SSAE16 certified because when the dust settles, they will be extremely embarrassed. The CPA firms that conducted the attestation engagement will be held accountable by the AICPA peer review board.

Now, should user seek assurance about security, availability, processing integrity, confidentiality, and privacy from the datacenters they use? Yes. Most definitely. The question is what is the standard that should be used to measure them against, and what level of assurance should they have to provide. I have a presentation that steps through each of the different categories of standards and attestation standards that are available out there in case anyone is interested.

SAS 70 (0)

Anonymous Coward | more than 2 years ago | (#38760434)

SAS 70 can be integral to a wider (financial) audit as its essentially auditing the process over the controls...which you can then make assumptions based off of.

It may seem like its nothing but the way auditors make their formula and asses risk, it means a lot to them knowing the shit might not stink so to speak.

For What It's Worth, Your Mileage May Vary... (0)

Anonymous Coward | more than 2 years ago | (#38763530)

I am posting as AC to protect myself and the very, very guilty...

I am a technical writer with 25 years experience, starting as a SW engineer and including project management. Some years back, I was contracted by a company that performed financial services for large invenstment banks to work with an external SAS 70 auditing company. I spent months interviewing everybody in all aspects of the company on how they did their jobs (policy, procedure, checks-and-balances), what data they used, how that data flowed (creation, storage, manipulation), about IT infrastructure (policy, procedure, backup and failure plans) and on how decisions were made. The interviews were extensive, detailed, and wideranging; complex issues were carefully and completely laid out in my documents. The person who contracted me was impressed and happy with the data I generated.

The auditor came in, threw away 90% of my data, wrote a report that said everything was wonderful but provided no substantive detail, and presented the company a SAS 70 approval certificate.

I am still of the opinion that either SAS 70 is a cynical ploy or that the auditing company knows just exactly how to f*ck a dog...

NOT A CERTIFICATION!!!! (1)

itcontrolsfreak (2557520) | more than 2 years ago | (#38766404)

DaCurryman has it right. SAS 70 (dead) and the new standards (SOC 1, SOC 2, SOC 3) are not certifications. SOC 1 (SSAE 16) is not intended to provide assurance over security. It is intended to provide financial statement auditors (not management of prospective customers) with an understanding of controls in place at a service organization that impact the financial reporting of their audit clients (the data center customers). None of the AICPA "standards" have anything to do with the "lofty operational standards" mentioned in the original post of this thread. The AICPA standards are standards for conducting the attestations, NOT standards for data center security and operations. If your intended audience is management of existing customers and of prospective customers, then SOC 2 and SOC 3 are infinitely better attestations (NOT CERTIFICATIONS!!!) to request. Many data centers are helpless at this point because existing customers all want SSAE 16 reports. They don't understand why, they just know that their auditors will ask for the report during the next audit cycle just like they asked for a SAS 70 report in years past. It is up to those on this board to educate your management and your customers about the differences and ensure you provide your customers with the correct report.

There are two types of SSAE16 audits (1)

fleetwood (230282) | more than 2 years ago | (#38768070)

In a Type 1 audit, all the auditors look for is whether the company has policies/procedures/controls in effect to obtain the objectives of the company (whatever those may be)

In a Type 2 audit, the auditors will attempt to determine whether the policies and procedures in place are being followed. Whether the controls are effective in achieving the objectives that have been stated.

I work for a software company that recently went through a Type 2 audit. In our case most of what was looked at was our SDLC (software development life cycle) process, version control, etc. They went through our work ticket system & spent a week following more than a few tickets through the entire process: code check out, work produced, QA testing, user testing, peer review, code check in. They spent several weeks over a three month period driving our internal audit & software staff nuts.

Does it mean anything? From our point of view, yes. But, not only does the audit depend on the quality of the auditors, but on the quality & detail of those process & procedure documents that they are auditing.

Check for New Comments
Slashdot Login

Need an Account?

Forgot your password?