Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

DSS/HIPPA/SOX Unalterable Audit Logs?

kdawson posted more than 6 years ago | from the write-once-keep-forever dept.

Businesses 381

analogrithems writes "Recently I was asked by one of the suits in my company to come up with a method to comply with the new PCI DSS policy that requires companies to have write once, read many logs. In short the requirement is for a secure method to make sure that once a log is written it can never be deleted or changed. So far I've only been able to find commercial and hardware-based solutions. I would prefer to use an open source solution. I know this policy is already part of HIPPA and soon to be part of SOX. It seems like there ought to be a way to do this with cryptography and checksums to ensure authenticity. Has anyone seen or developed such a solution? Or how have you made compliance?"

cancel ×

381 comments

Sorry! There are no comments related to the filter you selected.

Write them to a DVD jukebox (3, Interesting)

Anonymous Coward | more than 6 years ago | (#20067393)

Optical media are great for write once, read many.

Re:Write them to a DVD jukebox (4, Informative)

arivanov (12034) | more than 6 years ago | (#20068011)

Not quite.

They are not very good at tasks which involve writing a lot in small increments like a log. The sector size is quite big so if you guarantee that each log entry has finished physically on disc without caching till the sector is full the disc will be eaten in no time.

You probably need a custom writer/reader (most normal ones cannot alter sector size) and custom formatted media along with something different from isofs. Not rocket science really, but definitely beyond the scope of DIY.

Re:Write them to a DVD jukebox (4, Funny)

//rhi (15411) | more than 6 years ago | (#20068077)

I always thought that WORM stood for "Write Once, Read Maybe"
//rhi - Enjoy the American Dream - You have to be asleep to believe it.

Syslog (0, Offtopic)

cerberusss (660701) | more than 6 years ago | (#20067405)

Good old syslog comes to the rescue. Besides logging locally to disk, also add a line to /etc/syslog.conf to log to a remote machine. That's probably enough read-only for you.

Don't thank me until you've seen the bill.

Re:Syslog (4, Insightful)

Whiney Mac Fanboy (963289) | more than 6 years ago | (#20067451)

Besides logging locally to disk, also add a line to /etc/syslog.conf to log to a remote machine.

If syslog can write to a remote machine, then a compromised syslog can overwrite a file on a remote machine. I suspect thats not even remotely close to enough read-only.

As others have suggested, print your logs on a line printer.

Re:Syslog (3, Insightful)

pedestrian crossing (802349) | more than 6 years ago | (#20067529)

As others have suggested, print your logs on a line printer.

But that doesn't really scale very well, and then you have the problem of dealing with retention/storage requirements.

Re:Syslog (1)

a_n_d_e_r_s (136412) | more than 6 years ago | (#20067535)

The remote machine should preferable save the syslog on a WORM device - for example a CD-R.

Its very hard to tamper with a write once CD-ROM.

Also it cheaper than the paper fed into a line printer option some people proposed in other threads.

Re:Syslog (1, Insightful)

Anonymous Coward | more than 6 years ago | (#20067599)

Sure. but what keeps you from making a copy of the CD-ROM with certain info deleted or altered, and putting that in the archives instead of the original?

Re:Syslog (2, Interesting)

Boricle (652297) | more than 6 years ago | (#20067715)

Probably the same thing that stops you from making scanning in the old print out, modifying, printing it out again and putting it in the stack.

i.e. Nothing really.

However, if you have the CD or tapes signed and dated by the ops staff, then shipped to off site security, you've made it harder to falsify.

The interesting issue is that if you are organized enough, what's to stop you from intercepting the messages on the way to the printer / CDR?

The only way I could see around this is some kind of trusted computing style initiative.

Re:Syslog (4, Insightful)

Anonymous Coward | more than 6 years ago | (#20067751)

a compromised syslog can overwrite a file on a remote machine

Not with a properly configured syslog. You're not supposed to just use a remote logfile, but a remote logging daemon (RFC 3164 [faqs.org] ). That way you can add entries to a remote log, but not change or delete any (unless you make the logfile directly accessible over the network, which I wouldn't recommend).

Re:Syslog (5, Informative)

dkf (304284) | more than 6 years ago | (#20067849)

You're not supposed to just use a remote logfile, but a remote logging daemon.
Another thing you can do is to send the logging messages over a non-IP connection (e.g. a serial line) so that even a standard network failure won't disrupt the logging and a hacked machine will continue to generate a track-able log. And the last I heard, you can't unsend bits sent down a serial line.

Re:Syslog (0)

Anonymous Coward | more than 6 years ago | (#20067885)

"And the last I heard, you can't unsend bits sent down a serial line." You can, as long as they haven't been seen at the other side. Just send a masking signal. If a human is listening at the other side, it need not even travel faster than the original signal Backward masking [wikipedia.org] can mask signals with signals send later.

Re:Syslog (1)

arivanov (12034) | more than 6 years ago | (#20068025)

This is actually the standard approach recommended in places like "Building Internet Firewalls" and such. It does nicely with the write-once requirement provided that you have also secured the machine from tampering. Unfortunately it does not do very well with the "read many".

Re:Syslog (2, Insightful)

cerberusss (660701) | more than 6 years ago | (#20067765)

If syslog can write to a remote machine, then a compromised syslog can overwrite a file on a remote machine.
I don't think so; the receiving syslog machine will be "add-only" and won't have rights to overwrite files. Of course, you can print your logs and that would be a good second defense. But searching through printed logs manually is a pain in the butt.

Re:Syslog (1)

ozmanjusri (601766) | more than 6 years ago | (#20067907)

If syslog can write to a remote machine, then a compromised syslog can overwrite a file on a remote machine.

Not if you're using some sort of data escrow service like OpenAccess [openaccess.org]

Re:Syslog (1)

fbartho (840012) | more than 6 years ago | (#20067939)

not if it's in a input stream only capacity... I admittedly don't know the details of syslog.conf and it's capabilities, but once you can send data to another computer, that other computer can control delete access. The uncompromised computer can just act as an append only log storage facility. I don't know quite how syslog.conf ensures that the logging isn't simply stopped or the source falsified. Maybe someone else can weigh in with answers

Re:Syslog (-1)

Anonymous Coward | more than 6 years ago | (#20067469)

No offense, but I'm glad that you are not a developer on my team.

"...never be deleted or changed..." someone could simply remove the line you added to /etc/syslog.conf on the remote machine.

Re:Syslog (0)

Anonymous Coward | more than 6 years ago | (#20067635)

... leaving a clear sign of his breaking in, and the logs up to that point, probably showing his IP and how he got in. Which is what he wanted to get rid of in the first place, I suppose.

Re:Syslog (1)

cerberusss (660701) | more than 6 years ago | (#20067773)

Yes, someone could do that. But of course, all sorts of measures should be taken to mitigate that risk. I'm thinking Nagios or Zabbix installations that guard the fact that logging should come in. Hardened machines, firewalls, Tripwire, et cetera.

But of course, nothing beats a printout on paper.

Re:Syslog (1)

pedestrian crossing (802349) | more than 6 years ago | (#20067493)

Good old syslog comes to the rescue. Besides logging locally to disk, also add a line to /etc/syslog.conf to log to a remote machine. That's probably enough read-only for you.

That actually may not be enough.

The systems I've worked with take it a bit further.

Once the syslog gets to the remote machine, it is then frequently dumped to an enclave machine behind the remote machine where it is dumped into a database for analysis and the raw logs are burned to cd or dvd on an autochanger.

The only proprietary piece of the system I saw was the software used to pipe the Windows systems' logs to the syslog server, but there may be open solutions available.

The rest could be handled by GNU/Linux/BSD and postgres (if you want to do the database part). The system I worked with ran on Solaris.

Re:Syslog (0)

Anonymous Coward | more than 6 years ago | (#20067531)

I'm using that approach, but just for myself - I'm not being bothered by river horse and dirty laundry requirements. I wonder if it will be read-only enough when some external party mandates the read-onlyness of the logging.

I'm not logging to a specific machine BTW: my logs are being broadcast, so I can have my logging machine anywhere on the network segment, or I can have more than one. I suppose this reduces the risk of attempts to break into the logging machine to go change the logs there.

Re:Syslog (1)

pedestrian crossing (802349) | more than 6 years ago | (#20067571)

I'm not logging to a specific machine BTW: my logs are being broadcast, so I can have my logging machine anywhere on the network segment, or I can have more than one. I suppose this reduces the risk of attempts to break into the logging machine to go change the logs there.

Broadcasting your syslogs across a production network is probably not a very good idea.

Ideally, you have a back-side management network that is separate from your front-side production network. All of your logging/IDS management/network management happens on the management network and none of the sensitive traffic is ever visible on the production network.

Re:Syslog (1)

fbartho (840012) | more than 6 years ago | (#20067953)

So... what a special vpn for logging? or a totally separate wired network?

Re:Syslog (1)

the unbeliever (201915) | more than 6 years ago | (#20068053)

Commonly known as a backnet, yes.

Some of my company's bigger customers have their own private backend network for internal data (db queries, backups, etc) and only pass necessary data over their public internet pipe. Not a virtual private network, but a truly private network.

Re:Syslog (0)

Anonymous Coward | more than 6 years ago | (#20068083)

Or a private VLAN.

Re:Syslog (1)

mpe (36238) | more than 6 years ago | (#20067959)

Good old syslog comes to the rescue. Besides logging locally to disk, also add a line to /etc/syslog.conf to log to a remote machine. That's probably enough read-only for you.

A simpler (and older solution) would be to have the syslog file be a printer or paper tape punch. Even have /dev/console be a teletype.

Re:Syslog (2, Interesting)

cerberusss (660701) | more than 6 years ago | (#20068099)

Hmyeah I agree a line printer is good as an addition, however paper is hardly searchable. I bet one of the requirements would be to have an auditing interface searchable by user, date, et cetera.

Question... What's to stop (2, Insightful)

Anonymous Coward | more than 6 years ago | (#20067409)

What's to stop someone from reading in one of the WORM tapes, modifying the log file data and then writing it back to another blank WORM tape and claiming it's the unaltered tape?

Do all logs have to be encrypted and signed with a seperate super sekret key/cert before being recorded?

Re:Question... What's to stop (2, Insightful)

beyondkaoru (1008447) | more than 6 years ago | (#20067547)

i don't know much about the laws/regulations in question here, but yes, there isn't anything stopping someone from making a new 'worm' storage device and claiming it to be new unless there's a third party who will remember identifying information on the data.

if i really wanted to make sure my archives weren't tampered with, i'd bring my data (in whatever medium, the 'worm' thing wouldn't be necessary to ensure non-tampering, though it'd be good for storage purposes) to a trustworthy and hopefully vaguely computer savvy notary. then, i and the notary would hash the data, write up a form that says "i, name here, declare that on this date data with this hash value, some hexadecimal, was filed. signed, signature".

storage aside, this means that for someone to tamper with it they'd have to either bribe/coerce/kill people who saw this form (difficult) or reverse a cryptographic hash (even more difficult). so, pick a good notary (or submit the hash value to the gov maybe?) and a good hash function (like a larger sha or whirlpool) and i think you're tamperproof.

of course, i don't know the regulation so i don't know if this matches the needs of the article.

Re:Question... What's to stop (1)

Barny (103770) | more than 6 years ago | (#20067669)

Hrmm, tfa sounded more like he wanted something that was tamper proof, rather than tamper evident (although evident with knowledge of who changed it would at least give the company a scapegoat).

Best bet imho is dump it all to a large DVD storage machine (pioneer used to make these, not sure if they still do) and replace the load of DVDs when it gets full, choose good quality DVDs, they degrade over time remember :)

Re:Question... What's to stop (2, Insightful)

beyondkaoru (1008447) | more than 6 years ago | (#20067723)

well, by being 'tamper evident' as you say, you are in fact tamper proof, so long as the data is well stored (and tape drives, cd or dvd jukeboxes, etc can do a good job of this). an iron-clad 'tamper-proof' box is not, in fact, tamper proof if one can simply substitute another iron-clad box in its place. this is the reason that only having a dvd jukebox wouldn't be secure (though again, i don't know what the regulation's requirements are). a nefarious company could simply juxtapose dvd's. remember that the ones who would be interested in tampering with the data are also the same folks who are storing and maintaining it.

Re:Question... What's to stop (1)

Propaganda13 (312548) | more than 6 years ago | (#20068027)

WORM devices are fine if you're talking about external compromise. Even paper documents are vulnerable to internal compromise. I've seen falsified paper documents. To guard against internal compromise, you usually look at remote data storage companies that you can only get a copy of the data if you need it back.

use a line printer (4, Insightful)

1u3hr (530656) | more than 6 years ago | (#20067417)

Connect a line printer to mirror the log file as it's created. Use continuous fanfold paper. Get staff to sign and date first and last page.

Lawyers love paper. (A magistate once asked me if a printout I presented in a case was an "original email". I said it was as close as you could get.) In all likelihood, no one will ever refer to it, so don't worry about that it might take 10 minutes to find a page. Once a month, ship it to a secure storage. For real paranoia, have two printers making two simultaneous copies.

Line Printer won't scale (2, Insightful)

jsimon12 (207119) | more than 6 years ago | (#20067489)

If you only had a single machine or maybe even a couple lightly used boxes a printer might work. But even that would be near impossible to go back and sort through and if you ever ran out of paper or ink you would be SOL. And trust me the last thing you want is to to be SOL when it comes to SOX. If you have the money don't half ass an audit solution.

Re:use a line printer (2, Interesting)

Nefarious Wheel (628136) | more than 6 years ago | (#20067521)

Wouldn't work in Australia, compliance penalties apply if you can't dredge up the data within a specified period of time. YMMV but it'd be worth checking what the regs actually require. A good reference is this little PDF I found http://www.ironport.com/pdf/ironport_email_complia nce_guide.pdf/ [ironport.com]

Personally I'd think about a hardware solution, block replication off-site to a third party registry. When you're talking compliance (especially fiduciary compliance) it's usually easy to come up with the bucks, so dream up something right and propose it.

Re:use a line printer (1)

1u3hr (530656) | more than 6 years ago | (#20067555)

Wouldn't work in Australia, compliance penalties apply if you can't dredge up the data within a specified period of time.

Within a day at most, I'd think. Anyway, you'd have your "ordinary" file copies to make reference to immediately, you'd only have to dig up the papr record to verify it.

How often would that actually occur? Only in case of company meltdown, a la Enron?

Re:use a line printer (1)

Zadaz (950521) | more than 6 years ago | (#20067737)

How often would that actually occur?


Once would be enough.

Re:use a line printer (1)

1u3hr (530656) | more than 6 years ago | (#20068127)

Once would be enough.

So if it's only likely to be needed once, why is it a problem to take a few hours to rerieve it? We're talking about accouting records of years gone by, not Jack-Bauer-lives-at-stake urgency.

Re:use a line printer (0)

Anonymous Coward | more than 6 years ago | (#20067785)

I live in Australia, and I was once asked to perform a task on some data for a court case. The data was to be converted from a Microsoft Access database into comma separated values, and then was to have all the linefeeds stripped out and then printed out onto hardcopy, and that was how the data was to be presented.

I wondered about this for a while, if the court has asked for the data, can you be considered to have provided it if you show up with a shipping container full of hardcopy data and say "there it is", or are you legally required to be a bit less petty than that? I guess SOX has its own set of rules again so there probably isn't a comparison here.

Re:use a line printer (1)

lukas84 (912874) | more than 6 years ago | (#20068043)

I did that once for a court case - not SOX etc. related.

I sent about 5000 pages through a courier with cash on delivery (Paper, Toner, etc. isn't cheap). I got my money and they got their data, in unusuable form.

Re:use a line printer (1)

KlaymenDK (713149) | more than 6 years ago | (#20067771)

I was going to post exactly this, the use of continuous paper.

But please, do make sure to select as small a typeface as technically possible.

This not only "saves trees", but also costs associated with handling and storing the logs after the fact. Paper piles quickly get both large and heavy! If you can save 15% by running 7pt instead of 10pt, it pays off in terms of square footage in the archive.

Re:use a line printer (1)

JohnFluxx (413620) | more than 6 years ago | (#20067923)

Instead, save the files to disk but get the md5 sum of the file and print out the md5 sum. Then you can prove that the logs haven't been tampered with.

Re:use a line printer (1)

ovideon (634144) | more than 6 years ago | (#20068005)

Why is this funny? We used this at my old work, and it worked great.

The only real concerns are ease of sabotage (although normally the printer will be behind a locked door), equipment failure and fire - and the last ones are relevant for any logging system.

Re:use a line printer (1)

Eivind (15695) | more than 6 years ago | (#20068073)

Better yet. Don't print the logs. Print every (hour|day) a single line:

At (time) the sha1sum of [file] was [sha1sum] signed: .....

Then proceed to store [file] however you please, make sure to have good backups. There's a problem offcourse, there's nothing stopping you from replacing the paper in the future. Since you produced it once, you can certainly produce a valid-looking similar copy.

This problem can be solved by having an external, trustworthy, keep or publish a fact. For example, if you published the sha1sum of your logfile as a ad in the NYT, you'd never be able to change it after-the-fact.

But that's overkill. A (digitally) signed statement would suffice: "We, the [trustworthy-institution] attest to having been presented with the string [sha1sum] by [your-company] at [date] [our-signature]"

Such a scheme would make it impossible for you to tamper with the logs unless you had either subverted sha1, broken the digital-signature algorithm, or somehow gotten hold of the secret-key of trustworthy-institution. All of which should be significantly more difficult than exchanging one piece of paper for another.

As it happens, Thawte and others offers such "timestamp signing" where you send them a fingerprint of a file, and they send you back a digitally signed copy of the fingerprint, with a timetstamp for their reception added. (in effect, "we Thawte received [fingerprint] on [date:time] [signature]")

This is a solved problem. (0)

Anonymous Coward | more than 6 years ago | (#20067419)

Write them out to a dot matrix printer.

No cryptography necessary!

USB Card punch (2, Funny)

www.sorehands.com (142825) | more than 6 years ago | (#20067423)

And you thought there was no use for a USB card punch.

Hard to change punched cards. Just don't trip with your box of cards.

Tattoos (2, Funny)

Anonymous Coward | more than 6 years ago | (#20067425)

Start tattooing everyone in the office with data. Encode it in some nice optical way (a la barcodes) for easy reading later.

Ontop of the obvious benefits, it provides a good deal of job security, if they get fired, they take away some important data, your employees will be thrilled with their newfound sense of security.

Go with commercial hardware solution (5, Informative)

jsimon12 (207119) | more than 6 years ago | (#20067431)

I preface this by saying I know I will get flamed for not recommending Open Source but SOX is a Federal mandate (Federal equals PMITA)

EMC's Centera [emc.com] is my personal favorite, it isn't cheap but it does exactly what you need and is auditable and recognized by all the third party audit compmaies as well as the Federal government.

I have worked in IT for 15 years and 5 of those have been for a LARGE financial institution. When it comes to audit and SOX go with something standard, tested and commercial, unless you want to spend the next 6 months explaining to your auditors how your homegrown solution works and then the next 6 months building something new that your auditors do understand (or worse, like losing your job).

Re:Go with commercial hardware solution (5, Funny)

feepness (543479) | more than 6 years ago | (#20067695)

unless you want to spend the next 6 months explaining to your auditors how your homegrown solution works and then the next 6 months building something new that your auditors do understand (or worse, like losing your job).
I dunno, I can lose my job WAY faster than 6 months.

How odd (2, Insightful)

Anonymous Coward | more than 6 years ago | (#20067433)

It looks like that commercial offering is a piece of hardware with a network API (web service) that you write to which doesn't provide any network APIs for deleting or modifying records. Presumably it has a read-only view of the data.

Now, assuming that they use harddrives, we all know that someone could extract mount the file system and change records. They could hide their tracks by recalculating cryptographic hashes. So it's simply a lie to say that the only way to modify or indeed delete the data the data is through physically destroying the hardware.

However having a dedicated hardware for a write-only, read-many system is a good idea, but I can't imagine that this would necessarily be more complex than a locked down box running some web scripting language that appends http post data to a log, or a database.

If there is more to it then please could someone elabourate on what exactly one must do?

Re:How odd (2, Informative)

arivanov (12034) | more than 6 years ago | (#20068101)

Now, assuming that they use harddrives, we all know that someone could extract mount the file system and change records.

Not if they have done it properly. If it is designed as an audit solution it is likely to have a hardware crypto module, a device specific key and have all data written out to disks at least signed with it. More likely - encrypted with it. In either case even if the fs is standard you cannot do jack sh*** with it after taking the drives out.

By the way - implementing the above using OSS is trivial as all free OS-es nowdays provide a TPM API so you can have unique machine keys. In fact you can implement this on top of any Free OS and integrate it with any standard MTA and most applications with minimal effort. The implementation would also most likely pass audit scrutiny as it is trivial. The only sticking point will be the crypto procedures and especially escrow. While proving that the app and the design is compliant is not hard, proving that your CA procedures are solid is a phenomenal pain in the a***. Also, you need to prove that you have an effective escrow and taking a hammer to the log machine does not prevent reading the compliance logs later on. The vendor has already done that and the auditors are happy. Compared to that it will take you on average 4-6 months to get this done with the help of external consluttants. Now, if you have done it anyway for a different project that is an entirely differnet ball game. You always have to prove to auditors that your app does what it says on the tin anyway and the apps are often internal. So one more or one less item is not going to turn the boat if the main sticking points (the CA and the escrow) have already been done.

Re:How odd (3, Insightful)

Sobrique (543255) | more than 6 years ago | (#20068107)

Sort of, but not quite. A Centerra is a Content Addressed Storage thingy. Which basically means it's file identifiers are md5 sums. It's a multi node thingummy too, which replicates stuff about. Is it impossible to tamper with? Well, no, nothing is. But it's pretty hard. Simply because it has implicit 'tamper detection'.

The API is also geared up so you can choose what 'mode' you want it to operate in. In the most secure mode, the API and OS built in (it's Suse based) won't let you delete anything. Which, basically means you have to pull out the individual drives that 'clip' is stored on, to trash it. Data will be gone, which isn't great, but ... well, pretty much impossible to prevent for any system. Modifying data retroactively though, is much much harder - recreating the right md5sum is a non trivial task. Impossible? Perhaps not, but ... well, EMC have done quite well with 'selling' this product in a 'it will meet your compliance needs' which is considered good enough for our auditors.

We have 'financial organisation' regulations, for retention of emails, and a Centerra is what we settled on as the solution.

Is modification detection enough? (1)

gweihir (88907) | more than 6 years ago | (#20067439)

If so, you could use a hardware solution, e.g. a WORM MOD disk rto store periodic crypto-hashes of your logs. Then any changed would be reasily obvious, even if it would not be obvious what was changed.

How about (0, Redundant)

phagstrom (451510) | more than 6 years ago | (#20067445)

/dev/null?

No altering of the data here.

Sometimes, the old ideas are the best (2, Insightful)

polymath69 (94161) | more than 6 years ago | (#20067457)

Nothing is an unalterable as a line printer which lacks reverse-vertical-paging capability. Just make sure it doesn't run out of ink or paper.

Re:Sometimes, the old ideas are the best (1, Funny)

Anonymous Coward | more than 6 years ago | (#20067505)

Paper is easy to delete. They sell the tools at every drug store in I've ever seen. It's called a lighter.

WORM Device (2, Informative)

passthecrackpipe (598773) | more than 6 years ago | (#20067459)

What you need is a Write Once Read Many (WORM) device. Unless EMC started shipping Open Source hardware (hahaha) I don't think you will be able to find this as Open Source. There may be some software solution, but you would most likely need some certification for it anyway ("no, officer, it _really_ is unalterable, trust me....."). Granted, most "hardware" solutions implement WORM through software, but I know from experience that it is impossible to change the data on WORM.

Technically, a CD-R with some checksumming would work to be compliant - these guys [am-utils.org] have some more info, but if you need it for formal compliance use, you are better off talking to your friendly neighbourhood storage vendor to save you lots of legal hassle should you ever need the WORM thing for evidence. It is the difference between a lengthy legal process where you have to explain exactly why your homebrew solution is legal and simply saying "talk to NetApp"

Re:WORM Device (0)

Anonymous Coward | more than 6 years ago | (#20068039)

It is the difference between a lengthy legal process where you have to explain exactly why your homebrew solution is legal and simply saying "talk to NetApp"
Buying any premade devices may shield you from some legal problems, but certainly does not absolve you from everything. If you're very unlucky, you will get sued, and can in turn try to sue the manufacturer, who most certainly has excluded any liability exceeding the purchase price in the clickthrough and the shrinkwrap license of the device.

How sure do you need to be? (3, Interesting)

edashofy (265252) | more than 6 years ago | (#20067473)

Cryptography, digital signatures, and checksums can only take you so far. They can detect tampering pretty easily. However, crypto can't prevent someone from deleting a file, although by checksumming or signing a whole bunch of files you could at least detect deletion of one of them. Ultimately, if you really want permanence, you need to write it out (as an above poster suggested) to some sort of write-once media. CD-Rs or DVD-Rs would obviously fit the bill here, although one can indeed delete a CD-R by simply throwing it out, of course.

Another cheap write-only medium is paper; I suppose you could purchase a laser printer (or even a line printer), and have it spit out the logs as they occur. If you kept the printer in a locked transparent box, nobody but people with the keys would have access to the output.

You could burn the logs onto PROMs as well, that's pretty permanent :)

Anything on magnetic or flash media can be erased or tampered with somehow, unless the drive controller hardware itself prohibited overwriting existing data. Even then you're relying on someone not being able to replace the drive controller or take the drive apart and diddle the platters/flash chips directly (although I suppose a decent amount of epoxy could thwart this). Any software-based solution can be tampered with in theory. One hacker favorite (which may be a legend or not) is that people used to get root on other people's boxes and then replace their copy of PGP with an instrumented copy. Thus, even the encryption software became compromised.

For compliance, though, I'm not sure what kind of oversight you have to have. At the end of the day, somebody has to be trusted with these logs, and that person would almost assuredly have the power to destroy them, or at least portions of them.

Re:How sure do you need to be? (1)

JoshHeitzman (1122379) | more than 6 years ago | (#20067803)

Anything can be destroyed or altered and as with any security issue this a matter of making the cost of doing so more then anyone is willing to pay.

All you need to determine if data has been tampered with is a currently unbroken cryptographic hash for the data that has been timestamped and signed by a timestamp authority (or more then one depending on how much you want to tampering to cost).

Preventing deletion (i.e. total destruction rather then simply deleting entry from a larger file which is merely tampering) of data is matter of having seperate and secure back-ups and the ability to tell when a file has been deleted. This can be handled with cryptographic hash of all of the current signed timestamps for the files so if one goes missing you can tell immediately and retrieve it from the archive.

Re:How sure do you need to be? (1)

JohnFluxx (413620) | more than 6 years ago | (#20067927)

Save data to a hard disk. Put in envelope. Have lawyer time and date envelope. Stick in bank vault with bank time and dating it and not allowing any future access to the device until needed.

If you use Linux... (0)

TerminaMorte (729622) | more than 6 years ago | (#20067487)

If a linux machine is storing these logs, just use 'chattr +a filename'. This is append-only, so you can still write to the logs but never alter/delete them.

Re:If you use Linux... (0)

Anonymous Coward | more than 6 years ago | (#20068095)

Never? As in 'chattr -a filename' never?

Syslog + chattr (5, Insightful)

ethzer0 (603146) | more than 6 years ago | (#20067491)

I use syslog-ng to relay information from several different datacenters to a centralized and secure location hosting all of the syslog information. Each DC has its own syslog-ng system acting as the local relay, transporting syslog information from local clients using TCP over a VPN to the centralized host. The logs are written on the central syslog sever organized by on date and hostname, and each file that is created is then assigned an 'append-only' bit using chattr. It works really well.

The Linux way (2, Insightful)

professorfalcon (713985) | more than 6 years ago | (#20067495)

tail -f thelog.txt >> /mnt/cdrom/thelog.txt

FreeBSD to the rescue (3, Insightful)

stox (131684) | more than 6 years ago | (#20067501)

FeeBSD supports append only files via the chflags command.

Re:FreeBSD to the rescue (2, Informative)

moosesocks (264553) | more than 6 years ago | (#20067895)

Yes, but the tricky thing about this situation is that it's a "who will guard the guards" type of deal.

If the root user can set that attribute, he can just as easily unset it, modify the data, and clean up after himself before re-setting it.

Remotely spitting your logs out to a line printer managed by a trusted 3rd party would seem to be a reasonable solution.

write checksums only to RO media (1)

DanMc (623041) | more than 6 years ago | (#20067507)

These ideas are still a hardware based solutions, but cheap and easy. Once every X seconds, write a checksum to a CD-R or DVD+R media. If the logs on your traditional magnetic disk are altered, you'll know. Depending on how much you can afford to lose, make X smaller. (I haven't read the new requirements, but I assume there's a tolerance of some kind) Another very certain method would be to write to a 1-way firewalled magnetic disk. e.g. copies of the magnetically stored logs are shipped (or just checksum packets) to a firewalled server with a hard disk. They're saved, but no traffic whatsoever is allowed out from that log box. Reading the logs requires physical access, and some kind of human oversight. (like launch codes that need 2 or more keys to access).

Your day to day log checks, like "any errors in last night's backup?" could be done live against the non-firewalled copies. When lawsuits, audits, or questions hit (and at some frequent interval), compare the live logs to the 1-way physically secured ones to make sure your live logs aren't being altered.

One-way data cable (4, Interesting)

rjh (40933) | more than 6 years ago | (#20067509)

At USENIX/EVT06 last year a team from the University of Iowa presented a cheap one-way data cable you could make with off-the-shelf parts from Radio Shack. Total cost is about $5 (for bulk, maybe $10 if you're buying single units) and it is provably, auditably, one-way. It was originally developed for electronic voting, to allow for counting computers to communicate with webservers that post election results. An attacker compromising the webserver cannot attack the counting computer, because there is literally no return path.

It works with very high reliability up to about 9600 baud.

You may be able to use this to your benefit. Have an isolated system air-gapped from the rest of the network which listens for log events on a one-way data cable. While you're no longer guaranteed to be safe (since if a logging PC is compromised, an attacker could send compromised data to the syslog PC and perhaps cause some sort of mayhem), but the lack of a return path makes interactive attacks infeasible.

ObDisclosure: I am a graduate student at UI and know the guy who invented the data cable, although I am not associated with the gadget.

Re:One-way data cable (1)

PeterBrett (780946) | more than 6 years ago | (#20067583)

At USENIX/EVT06 last year a team from the University of Iowa presented a cheap one-way data cable you could make with off-the-shelf parts from Radio Shack. Total cost is about $5 (for bulk, maybe $10 if you're buying single units) and it is provably, auditably, one-way. It was originally developed for electronic voting, to allow for counting computers to communicate with webservers that post election results. An attacker compromising the webserver cannot attack the counting computer, because there is literally no return path.

It works with very high reliability up to about 9600 baud.

Ah, the old "RS232 cable with the RX conductor snipped" trick. So, how does this help if you've got 20 servers?

Re:One-way data cable (1)

rjh (40933) | more than 6 years ago | (#20067663)

(a) snipping RX doesn't make the cable auditable or provable.
(b) serial cards are cheap.

Re:One-way data cable (1)

befletch (42204) | more than 6 years ago | (#20067809)

Don't know about any of this HIPPA stuff, but Bellovin mentions snipping the TX line on a network cable (presumably ethernet) in his 1992 paper about attacks on att.com entitled, "There Be Dragons." You can find a PDF off of this page: http://www.deter.com/unix/ [deter.com] . Top of page 6. Presumably ethernet would get you a little more bandwidth than serial.

I also remember reading about some NSA (or similar) machine that someone had to feed data to. It did reply when data was sent, but the only reply it ever gave was some sort of OK packet. Maybe someone else has more details.

If I had to do some legally mandated logging, I would definitely use something certified for the job. But for just hacking around, there are some interesting ideas out there.

Write-Only Memory (4, Funny)

jmv (93421) | more than 6 years ago | (#20067549)

That's all you need [wikipedia.org]

TimeStamps (0)

Anonymous Coward | more than 6 years ago | (#20067609)

Just my 2 cents ....
Couldn't you add a timestamp to each log entry issued by a certified (and preferably external) time stamp server?

Tamper evidence not tamper proof (0)

Anonymous Coward | more than 6 years ago | (#20067643)

At a company I once worked for, we used a scheme whereby every entry in the audit log (database) had a hash of the entry's contents, which also included the previous entry's hash. This made tampering with the audit log evident when the hashes are recalculated--a change in one entry would throw all subsequent entries' hashes out of whack.

Note that this made the log tamper evident, not tamper proof which is a different ball game entirely.

Why not store it in a version control system (3, Interesting)

Ptur (866963) | more than 6 years ago | (#20067653)

I would dump it in GIT or the likes.... any change of it will be recorded ;) Seriously, many version controlling systems already contain the data integrity and authenticity checks that you need

Filesystem level ? (1)

bytesex (112972) | more than 6 years ago | (#20067671)

Are there no modifications possible of standard filesystem code, say ext2, to create an append-only filesystem ? You can pass O_CREAT but not O_TRUNC to a open(2) call, and you cannot do lseek(2) (or you can do lseek, but only to EOF, or you can do lseek, but only to read - as soon as you write, your pointer moves to EOF). It could also be a mount option next to read-only and read-write. Then, once a month, under four eyes only, you copy the lot to a tape/DVD/paper/your mom's forehead, remount in read/write mode and wipe, and remount in append mode.

SELinux (1)

amorsen (7485) | more than 6 years ago | (#20067683)

People have only mentioned hardware solutions so far (apart from a few useless ones like chattr). You can do this with SELinux, in a way so that it requires a reboot with console access to do anything about it. It may not be enough for you, but it's enough for some people.

This request is impossible. (3, Insightful)

The Master Control P (655590) | more than 6 years ago | (#20067691)

Given sufficient resources, time, and dedication, ANY log can be altered.

If the "unalterable" log is maintained in software, it's a comparatively simple matter of hoisting it up on a VM. Since we're presumably talking about white-collar crime, it's a fair bet they have or can get root access to the machine to install the VM and rootkit to hide it. At that point, the CEO can do anything and the system can't fight back. Capture passwords of people logging in, alter data, you name it.

A hardware system would be more robust, but still vulnerable. I imagine the most likely attack vector would be Man in the Middle - Just take over the box that guards/drives the logger machine.

Re:This request is impossible. (1)

smallpaul (65919) | more than 6 years ago | (#20067911)

A hardware system would be more robust, but still vulnerable. I imagine the most likely attack vector would be Man in the Middle - Just take over the box that guards/drives the logger machine.

Why is SSL not sufficient to overcome a man in the middle attack?

Re:This request is impossible. (1)

pilot1 (610480) | more than 6 years ago | (#20068135)

An SSL certificate has to be signed by a CA, Verisign for example. There's nothing stopping someone from telling the system creating the logs to accept certificates from another CA (yourself), and using this CA to sign the man in the middle's SSL certificate. SSL won't provide any protection when the attacker has access to the software involved.

Re:This request is impossible. (2, Insightful)

hazem (472289) | more than 6 years ago | (#20068061)

Given sufficient resources, time, and dedication, ANY log can be altered.

What really matters is if there is any case law that actually interprets the laws and provides standards for due diligence.

The law might say "unalterable" or "lasting indefinitely" we all know there are practical physical limits - given enough time, anything is alterable and nothing lasts forever. We could come up with outrageous methods like using satellites with lasers to etch logs on the surface of the moon, but there's not much point in going to such expense until the case law suggests that is what is needed to be "due diligence".

Until you have that case law each organization will simply have to do a cost x risk analysis and determine how much they're willing to risk in order to "do enough" to keep out of trouble with the auditors. Something like:

chance of being audited x cost of failing an audit vs. cost of maintaining an "unalterable" logging system

Then you just wait to see what organizations get skewered and adjust your analysis and practices accordingly... and just hope you're not the first to get audited. OR... we can work on a way to etch logs onto spheres of aluminum that are then launched into orbit so they can be read with a telescope. Though that will probably lead to an increase in insurance premiums.

Guy Fawkes Protocol (5, Interesting)

LilBlackKittie (179799) | more than 6 years ago | (#20067693)

Some of the work I do may require something like this, so I'm considering implementing Guy Fawkes over syslog.

http://www.cl.cam.ac.uk/~rja14/Papers/fawkes.pdf [cam.ac.uk]

From the paper:

6.2 Tamper-evident audit trails

It is a well known problem that an intruder can often acquire root status by using well known operating system weaknesses, and then alter the audit and log information to remove the evidence of the intrusion. In order to prevent this, some Unix systems require that operations on log and audit data other than reads and appends be carried out from the system console. Others do not, and it could be of value to arrange alternative tamper-evidence mechanisms.

A first idea might be to simply sign and timestamp the audit trail at regular intervals, but this is not sufficient as a root intruder will be able to obtain the private signing key and retrospectively forge audit records. In addition, the intervals would have to be small (of the order of a second, or even less) and the computation of RSA or DSA signatures at this frequency could impose a noticeable system overhead.

In this application, the Guy Fawkes protocol appears well suited because of the low computational overhead (two hash function computations per signature) and the fact that all secrets are transient; this second's secret codeword is no use in forging a signature of a second ago.

Use a GIT repository (1)

HRogge (973545) | more than 6 years ago | (#20067719)

GIT is the sourcecode management system for the linux kernel and it has a number of advantages for you:
- GIT use a SHA1 hash for checking the integrity of files, so you can be sure that you got the right file into your repository
- a GIT "version tag" is a SHA1 has over the whole history of the repository. Noone will be able to change anything below the "version tag" without changing the hash
- GIT supports signing checkins with PGP signatures, so it's clear who inserted new files into the archive.

Just burn the GIT repository on a CD or tape every week and keep the SHA1 hash tag for every day at a secure site on paper.

http://en.wikipedia.org/wiki/Git_(software) [wikipedia.org]

Clicky (2, Informative)

mritunjai (518932) | more than 6 years ago | (#20067767)

WORM media with HIPPA compliance in mind...

WORM on wiki [wikipedia.org]

From Experience (5, Informative)

Evets (629327) | more than 6 years ago | (#20067805)

I honestly don't know about DSS or SOX, but I have had plenty of fun with HIPAA.

Unalterable logs as a matter of compliance does not mean "absolutely unalterable under any circumstances". There should be no way for an end user to modify audit trails. There should be no preconceived way for an administrator to alter audit trails - i.e. no utilities for doing so. That does not mean that an admin can't go directly into the DB and alter the data from behind the application layer.

Under every circumstance when I have run into audit logs involving HIPAA compliance, they have been written by an application directly into a SQL database (oracle, ms sql, informix, and one time db2). It used to be that they were written in a fairly easy to decipher format within a single text column on a per record basis - which made for a fairly-difficult-to-alter audit trail because within that easy to decipher format were non-printable characters that you would at least have to know to look for them. With current implimentations, however, the records are stored in a separate table with a many-to-one relationship with the audit-required records, in varchar fields, as plain text - much easier to alter or get rid of single entries. There is still a level of obfuscation as far as table names and column names but thats really a side effect of other things that are going on.

These systems have been reviewed by auditors and certified as compliant. In the older system, there was no application interface to delete audit records. In the newer system, there is an application interface to delete records in any given application table - and therefore there is one for the audit tables as well. Admin level access is required to delete or alter the records, though.

Personally, I would expect more as far as HIPAA compliance goes - from both a customer standpoint and an auditor standpoint. My experience (and it is pretty extensive across several high profile enterprises) - is that the customer will demand a better system only when the auditors demand a better system. I haven't run into an auditor yet who has even given more than a casual glance at the 'back door' scenario. I suppose it's because there is no true way to keep things absolutely secure and application level audit log security is only one layer of the onion.

Before you get too far into an overly complex and potentially expensive solution, talk with your auditors about the requirements for your specific scenarios. They've seen it before and can tell you exactly what they are looking for from an audit compliance standpoint. They are usually pretty easy to work with and open with their knowledge.

Re:From Experience (2, Informative)

georgewilliamherbert (211790) | more than 6 years ago | (#20067913)

Second the "ask the auditors what they are looking for"... not everyone gets audited the same.

Financial company I know passed audit fine with syslog -> a secure system which the normal sysadmins didn't have access to. The people whose actions were being logged couldn't get to the logs (well, presumably someone could break the system, but it was well secured and had non-overlapping sysadmin staff).

That was good enough. As long as it took two compromised people to hide any given event, that passed audit.

Physical Destruction (2, Insightful)

unsubscribe (1135687) | more than 6 years ago | (#20067807)

In short the requirement is for a secure method to make sure that once a log is written it can never be deleted or changed.

Although it is possible to prevent logs from being modified (using write-once media) or undetectably tampered with (using crypto, possibly with a TPM module for the ultra-paranoid), any log can be 'deleted' by physically destroying the device/media on which it is stored.

Dont skimp... (3, Insightful)

pjr.cc (760528) | more than 6 years ago | (#20067827)

Seriously, when it comes to legal requirements, do not skimp!

Go for something that is guarentee'd to fulfill your legal compliance requirements.

Yeah, optical media is great for WORM, but you dont want something your going to have to manage day to day. The legal req's of sox and so forth are beyond that of traditional optical drives in terms of life span in any case. Do not go with optical for compliance unless its something specifically designed for compliance (Again, thats $$$).

As someone suggested, centera is a good option - but all the storage vendors have good options (from emc, netapp, hds, sun, falconstor, mimosa the list is endless) and they'll all tell you how theirs is better than anyone else (and why). At the end of the day, you want a compliance solution with someone's stamp on it, and a throat you can cut when it goes wrong.

If your absolutely determined to go the compliance route on OSS - go with ext3cow (www.ext3cow.com) IMHO, a fully versioning COW fs with a non-erasable past and the best OSS solution for the job - backup on to optical if you like, but dont make optical your only option. If it only had policy-based management (i.e. snapshot whenever user X or group y writes a file) rather then crontab'ing its snapshot agent it would almost be perfect for a start-point solution for compliance. It has a big benifit along with it though, you can show users how to get files "from yesterday".

Keep in mind, WORM means policy-based write-once, not necessarily immutable storage! And almost every compliance worm product out there depends on that fact.

Re:Dont skimp... two other things. (3, Informative)

pjr.cc (760528) | more than 6 years ago | (#20067925)

ext3cow was written with compliance in mind (i.e. with an untouchable past), and so its AFAIK the ONLY solution that can fit in compliance (keeping in mind that this only covers part of compliance). svn, git, cvs - im sorry, but thats just a non-solution for compliance. It also gives you no-mess management with a very easy interface to make sure you are being compliant (this is important, and its something YOU dont have to be involved in, your lawyers can "look at the past" to make sure "discovery" is going to be consistent).

The second thing is, compliance is (ridiculously) complex - the compliance vendors have spent many hours with lawyers getting it together, they know the requirements and they know they fullfill them - this is important. It also means their solutions come with an implicit warranty - "hey, your using netapp worm, we know it works" as apposed to "what software is that? how do you know it works?". At the end of the day a lawyer is going to either go "well i cant argue with the compliance solution" when your with a well-known or "your honor, the defendant is using ... which has never been proven or certified by anyone".

Compliance is the only time i will say to someone - "get a throat to cut", get a solution you know works, written by people who know what they are doing and its all because compliance req's were written by lawyers for lawyers (i.e. scum) and so their scum is going to make you have to act like scum.

fast search as well (1)

martin (1336) | more than 6 years ago | (#20067841)

It's already been said in other posts but to summarise:

It's not just Read Only (which can be achieved easily with syslog), it's the ability to return requested data quickly that the critical thing with SOX et al.

If someone says they want to see information on item X, you HAVE to return this imformation in hours not days or weeks.

This is where the commercial products have spent their time, developing fast search capabilities over the data they collect.

Certification is also important. Ie product X has been shown to work and works like this..

It's HIPAA (0)

Anonymous Coward | more than 6 years ago | (#20067901)

Why can no one spell HIPAA properly?

Don't Build Your Own Device (3, Insightful)

Interfacer (560564) | more than 6 years ago | (#20067905)

I work for a big pharma company as a sysadmin, and we have to abide by similar rules and laws. Our data recording and data logging has to be proven to be unalterable.

Go with a commercial solution unless you want to battle with the QA and Validation departments for haf a year. And even if you would get the go-ahead (unlikely) you'd get in a hell of a lot of trouble during an audit because auditors a) don't know your solution and b) they will quickly see that it is not certified.

There are specified requirements (don't know the names and numbers by heart) that your solution has to proven to fulfill, and certified by some external party.
Just saying 'Yeah but I know it cannot be altered because it is syslog / ' will not cut it.

And non-compliance can eend up costing your company millions if not hundreds of millions.
Open source or home grown has it's place, but in a regulated environment you go with commercial for certain things because that is the only option where you get certification with your device / software.

Re:Don't Build Your Own Device (3, Informative)

timmarhy (659436) | more than 6 years ago | (#20067973)

"Our data recording and data logging has to be proven to be unalterable."

no such thing exists. given enough time and a mediocure amount of money, i'm 100% certain i could alter anything your storing your information on and make it look real.

the toughest system i've ever seen as far as audit trails goes is using cdr's in a machine that makes a hash of the data on the cdr AND reads the serial number on the cd and stores that on a geographically seperate cdr system. it's similar to those automated cd turnstyle things you can buy, only beefy with steel casing and alarms on it and what not.

Paper and line printers (1)

jsse (254124) | more than 6 years ago | (#20067999)

A couple of the posts above has already bought up the use of paper printing logs for the purpose. That's really practically used by many companies.

The solution, however, is hardly cheap. You need an expensive 4000-line per minutes line printer with stackers to re-fold and stack the fan-fold papers as they emerged from the printer. Also, you need monitoring tools to monitor the status of the printer and a room with security staff to prevent physical tampering.

(Don't bother using a laser printer, the cost of the ink along would drive your company into bankruptcy :)

I saw one of those system in IBM before, may be you could approach them for advise.

An idea -probably deeply flawed (1)

asifyoucare (302582) | more than 6 years ago | (#20068055)

A secure 'logging computer' is sent lines of text like "20070801:140537,The pearl is in the river".

md5 = 0 // md5 of previous line
When such a line is received:
    1. augmentedline = line & md5
    2. crypotline = encrypt(augmentedline,privatekey)
    3. md5 = md5(cryptoline)
    4. output = line & md5
    5. write output
Wend

(6. profit)

An attacker cannot append lines to the output (without detection) because they do not know the private key.

They cannot delete lines without detection because the previous line affects the calculation of the md5 for the next line. Similarly they cannot insert duplicates of previous lines.

Of course the attacker is more likely to attack the feed of data into the secure logger, or just delete the logs.

How come nobody mentioned this ? (1, Funny)

Anonymous Coward | more than 6 years ago | (#20068085)

Of all the solutions I have the one that would work, and I can't beleave the /. crowd have not mantioned it yet. Email your logs to gmail Have gmail automaticaly fwd your logs to hotmail Have Hotmail automaticaly fwd your logs to Yahoo You want to put more stuff into the mix, ensure your chain of emailing includes anonymous fwding or whatever other features you want. Every account has a password. If any email is disputed, I can log you in another account to prove its correctness. You get them time stamped What else do you want? G

Technology coming full circle... (1)

MFHFozzy (525991) | more than 6 years ago | (#20068141)

This functionality was built into Netware's NFS, all the way back to 3.x (maybe even 2.x) There was a bit for "Add" and "Edit". Worked just as the article reads.
Load More Comments
Slashdot Login

Need an Account?

Forgot your password?