More PDF Blackout Follies

georgewilliamherbert writes "The latest installment of "As the PDF Blackouts Turn" hit today, with a U.S. government apparently releasing a redacted version of their court filing in the Balco grand jury leak case which merely stuck a black line over the text, which remains available in the document. As with prior documents, entering text cut/paste mode in a normal PDF browser such as Acrobat allows a reader to access the concealed text. Previous incidents include an AT&T filing in the NSA case." This works with Xpdf and KPDF, too; for KPDF, use the selection tool (under the Tools menu) around the redacted section, copy to clipboard, then paste into the text-manipulator of your choice.

Maybe

[GmAz]

Perhaps the people making these "blacked out documents" should be taught a little about Vector Graphics and that a black box is not the same as a sharpie. One word for them 'n00b'!!

Re:Maybe

[gEvil (beta)]

You don't even need to go into vector graphics with these people. All you need to do is attempt to convince them that white text is still text, or that black text on a black background is still text. Either way, the text is still there. The only way to ensure that it's gone is to ACTUALLY GET RID OF THE TEXT.

Re:Maybe

[Doctor Faustus]

All you need to do is attempt to convince them that white text is still text, or that black text on a black background is still text. Either way, the text is still there.

This is a confusion over the way the Adobe Imaging Model works, not white-on-white or black-on-black. In Adobe's model, you start with a blank page, and you essentially paint on it; newly drawn things cover previously drawn things. Basically, despite what the previous commenter said, it really is like a Sharpie.

When you physically draw over something with a black marker, the previous text may be impossible to see, but it's still there. In the PDF, you'd only have to skip the instruction drawing the box to get the text out. Even if Acrobat didn't let you get at the text by cutting-and-pasting, someone familiar with the PDF format could still get to it with some work.

Re:Maybe

[massysett]

"may be impossible to see" are the operative words there. Ever used a Sharpie to black out the routing number on the bottom of a check? You can still make out the numbers. One way I've found to really black them out is to Sharpie the numbers, and then Xerox that check. Even Sharpies don't work as they might at first appear to.

Real redactors use razors. You hold up one of those redacted documents and it looks like a punch card.

Re:Maybe

[IAmTheDave]

Reminds me of one of my favorite Onion articles: CIA Realizes It's Been Using Black Highlighters All These Years [theonion.com]

Re:Maybe

[TubeSteak]

I un-redacted the PDF file, as an example for others. Instead of stripping out the black mask, I turned it red & lowered the opacity. It now highlights the 'redacted' portions quite nicely (and any underlining they used).

Skip to pages 6-16 of the PDF for the not-so-hidden goods
http://www.easy-sharing.com/528126/BALCO_quash_sub poena_sfchronicle_unredacted.pdf.html [easy-sharing.com]

P.S. I did it in FoxIt PDF Editor Pro, which I wouldn't really recommend to anyone

Re:Maybe

[statusbar]

No. no, all you need to do is to sell them 'magic-cyber markers' for $1000.00 that they can apply directly to their screen!

--jeffk++

(... a fool and his money are best parted... )

Re:Maybe

[Mirlas]

Maybe we need to go back to good-old fashioned text files.
It was good enough back in the days of wood-burning computers;
it should be good enough now.

Re:Maybe

[morgan_greywolf]

Maybe we need to go back to good-old fashioned text files.
It was good enough back in the days of wood-burning computers;
it should be good enough now.

Definitely! Then we can redact things with fancy ANSI terminal codes ^[[30;40mlike this super secret hidden message[[m!

w00t! No one will EVER figure how to defeat that!

Re:Maybe

[OldManAndTheC++]

It was good enough back in the days of wood-burning computers

Oh man, that brings back some memories! Late nights cranking out code on my Bunyan 2500 - that puppy went through three cords of oak a week, and it kept the place warm to boot. And we didn't need any of that fancy book learnin' to make it work either; if you were a good hand at whittling, you could be a programmer. Never had a lick of trouble with the Bunyan, except for the occasional splinter. Oh sure, you had to keep some kindling around to get her started, but once she got goin' she could do anything - add, multiply, and of course, branch.

Internet? Pfft. We modulated the smoke exhaust by opening and closing the flue - you could see it for miles, unless it was raining, or windy. Hell, we had peer to peer networks back before most of you guys were even a swimmer in your dad's testicals.

There's still a few Bunyans around, if you know where to look. Auditors like them, since they're so good at logging, and keeping a paper trail. I think the Vatican still has one, though they only fire it up when they elect a new Pope. Ah, the good old days...

Re:Maybe

[scribblej]

I had one of them wooden computers once. Wooden keyboard, wooden monitor, wooden CPU... only one problem: it wooden work. :(

Re:Maybe

[Uncle Kadigan]

Yours could multiply? Ours could only add. In order to multiply, we needed to build a log table.

Re:Maybe

[recursiv]

I can't remember the last time I saw rot13 mentioned without without someone cleverly pointing out their habit of double or triple rot13. That would be a novel thing.

Re:Maybe

[frdmfghtr]

Perhaps the people making these "blacked out documents" should be taught a little about Vector Graphics and that a black box is not the same as a sharpie. One word for them 'n00b'!!

Sometimes I wonder if these incidents are really "accidents" or somebody's way of feigning ignorance of technology to get the facts out to the public.

Re:Maybe

[Nutria]

these fellows at the NSA

NSA? Since when does the NSA redact subpoenas for the District Attorney?

Re:Maybe

[HumanisticJones]

Indeed it does seem I have allowed my mind to wander while writing a post. Anyone want to try out my new book, "Not Hitting Preview First: Making an Ass of Yourself in a Public Forum"?

Re:Maybe

[indifferent children]

Just because you're releasing the 12th printing of the 4th edition, does not make this a 'new book'.

Re:Maybe

[Gilmoure]

Sharpie def [wikipedia.org]

This is why...

[dubmun]

we should all just write everything down in pencil. Boo to technology.

Or... they could just find a better technological solution. Seems like a no brainer to me.

Re:This is why...

[Analog Squirrel]

<oblig>

Are you saying "boo" or "Boo urns"?

</oblig>

History repeats itself

[alshithead]

Perhaps after another dozen or so incidents they'll decide a little training is appropriate for the folks who are doing the redacting.

Re:History repeats itself

[cavtroop]

No, more than likely they will just pass a new law, stating that "Copying and pasting of blacked out (redacted) lines is a felony" or somesuch...

Re:History repeats itself

[Anonymous Coward]

I think that's called the DMCA

Circumvention

[Mateo_LeFou]

If black squares count as a "technical measure" protecting access to a work... ? Someone actually should go ahead and launch this suit, to draw attention to the DMCA's shittiness.

Re:Circumvention

[jZnat]

I don't think that court documents like these are copyrighted, so you can't even apply the DMCA to it. The leading source of public domain material these days seems to be the government itself...

Re:History repeats itself

[geobeck]

Perhaps after another dozen or so incidents they'll decide a little training is appropriate for the folks who are doing the redacting.

No, more than likely they will just pass a new law, stating that "Copying and pasting of blacked out (redacted) lines is a felony" or somesuch...

Train them to use the blackout method, but to replace the redacted text with "If you can read this, you're under arrest!"

Re:History repeats itself

[richg74]

This is in principle a good idea. However, the implementation may suffer from a fundamental problem.

My grandfather used to say that there is one irreducible requirement for training a dog: you have to be smarter than the dog.

Re:History repeats itself

[DarkSarin]

Fortunately this does not apply to humans--not directly.

I can easily train people that are smarter than myself, if the conditions are right. For instance, I know a fair bit about statistics and data analysis, and would be perfectly comfortable training certain folks in the field, as long as they didn't know more than I do. Even then, it perfectly possible for me to come up with a unique idea that someone smarter than myself hasn't (note that I didn't say couldn't) considered.

In the public schools there are frequent cases of a teacher training a student more intelligent than themself. It is unavoidable, although it could be reduced by making sure only the smartest teachers were highered.

Smarter? Not a requirement. More experienced? Having unique knowledge? Yes, that is required, but maybe not irreducibly.

HAND

Re:History repeats itself

[GWTPict]

Nah, he just can't spell.

Comment removed

[account_deleted]

Comment removed based on user account deletion

Re:History repeats itself

[gEvil (beta)]

What happens when I actually want to print white text on a black background? Will I have to go through some convoluted process because setting the background as black doesn't actually change the background to black, but rather also eliminates any text contained within it?

Comment removed

[account_deleted]

Comment removed based on user account deletion

Re:History repeats itself

[DarkVader]

You know, considering the state our government is in, I would much prefer that someone would build into all software going to the government an "unredact" feature to make it even easier to recover government coverups.

Barring that, PLEASE don't educate them, or make it easier for them to really redact anything.

Re:History repeats itself

[Eadwacer]

Sometimes history doesn't repeat itself. Sometimes it picks up a big club and says "Weren't you listening the first time?" - Terry Pratchett.

Re:History repeats itself

[indifferent children]

Maybe Adobe Acrobat needs a new menu item: Edit->Redact Then you only have to train people to use that feature rather than the backgound-color feature.

People...learn...?

[Elros]

You would think that people would have learned after the first time around. Apparently not.

--
"And the geek shall inherit the earth."

Re:People...learn...?

[Billosaur]

You would think that people would have learned after the first time around. Apparently not.

You're giving people too much credit; as has been noted in this forum many times, the average computer user is not exactly bright and doesn't read Slashdot, so they would have no idea that this is a problem. People just assume that if something appears to work a certain way, it in fact works that way.

Re:People...learn...?

[jimktrains]

"Human beings, who are almost unique in having the ability to learn from the experience of others, are also remarkable for their apparent disinclination to do so." - Douglas Adams

Re:People...learn...?

[gstoddart]

You're giving people too much credit; as has been noted in this forum many times, the average computer user is not exactly bright and doesn't read Slashdot

You're giving people too little credit. Most people who use computers are probably fairly bright -- they're lawyers, doctors, accountants, and all sorts of things most people on Slashdot can't do. Reading Slashdot doesn't make you bright (in fact, given much of hte drivel, just the opposite.)

But, they expect computers to work like a friggin' toaster, and to them, if the text it blanked out, it's not readable. They're not going to realize the 'black' is a representation of a rectangle in a different document layer, and that the actual internal tree of the PDF still contains the actual text. Really, how could they?

They understand computers by metaphor and analog to the real world. They don't know or care about the actual internal stuff. Since the paradigms have been done to look like the real-world, these people assume that the rest of the things also apply.

Many people use computers who don't have a full grasp on all of their intricacies. However, I haven't looked inside of a TV in 20+ years, but I'm comfortable using one.

Cheers

Acceptance of Risk

[Kadin2048]

While you make a good point, the people who have to use computers to accomplish their jobs, but do not make an attempt to understand how they work (and just treat them like "black boxes") are taking an enormous risk. They are hitching the metaphorical wagon of their livelihood to a team of horses that they don't know shit about.

If you were somebody who made your living in television, but didn't understand anything about it, you would likewise be taking a great risk. You might, for instance, look like a big idiot when you show up to work at your anchor desk wearing a horizontally pinstriped shirt (which looks like ass on TV because of the Moire effect between the lines on the shirt and the TV scanlines). If you had understood the technology a little better, you might not have done that. That's a trivial example -- undoubtedly if you were a TV anchor, you'd learn or be told at some point not to wear a shirt like that without having to learn about scanlines -- but I hope you see my point.

Whenever you use a technology without learning about it, you accept a certain amount of risk. Sometimes, you gamble and win: you just use the technology, get your job done, and nobody's the wiser. You're faster, more efficient, more competitive, you look like a hero to your boss, whatever. But if the technology doesn't work, then you're SOL -- but that's the price you pay for not understanding it. That's the risk you accepted when you said to yourself "eh, I don't really care what goes on inside there."

In the case of PDF, we have a lot of people using a certain technology without knowing anything about how it works, and thus -- like the TV anchor in his pinstriped shirt (or a weatherman wearing chroma-key blue or green) -- you get these gaffes.

I'm not saying that everybody needs to learn about how everything they use all day works, down to the bare metal. Virtually nobody needs to know that, except perhaps people who are doing things that are so dangerous that they can't afford to fuck up. However, people should be aware of the tradeoff they're making and the risk they're accepting when they forgo figuring out the internal details of a system and simply accept it as a whole, on faith that it will always work a certain way. As long as people are aware of that decision, and make it consiously, and accept the results, you can't ask for more.

Generally speaking: faith is a fine thing, as long as you know when you're relying on it. It's when you thought you were relying on something else, and find out that you had nothing but faith, that a problem has occured.

Re:Acceptance of Risk

[SydShamino]

They are hitching the metaphorical wagon of their livelihood to a team of horses that they don't know shit about.

Millions of Americans hitch the physical "wagon" (or SUV, or sedan, or minivan) of their livelihood to a bundle of "horsepower" that they don't know shit about every single day, and then they drive that wagon at 75 MPH.*

In the case of their cars, the consequences for misuse are serious injury or death. In comparison, the consequences for learning next to nothing about their computers seem slight

Re:Acceptance of Risk

[gstoddart]

I'm commuting a lot these days post Katrina...and it seems very few people understand the left lane is the passing lane...get the fsck out of the way of a driver coming up behind you faster than you're travelling. If possible, you should do the majority of your driving on the rt. lane (US).

And, I'm equally amazed at how many people are too damned ignorant and intent at driving at Max 0.6 to realize I'm in the middle of fscking passing this guy (as evidenced by the fact that I'm going faster than him), and t

Re:People...learn...?

[Nef]

Here's my problem with this scenario though. You could probably describe most of the inner workings of said TV, be it SD or HD, LCD /CRT/Plasma, without much difficulty. I'll grant you, you probably couldn't recreate the thing (unless you were an EE or something similar) but you have a basic understanding of how things work.

This represents a fundamental difference between how geeks/nerds think, and how the population at large thinks. Those technically inclined, whether they're gear-heads, pencil-pushe

which?

[Anonymous Coward]

with a U.S. government apparently releasing a redacted version of their court filing

Which U.S. government?

Re:which?

[TwilightSentry]

The RIAA

Works in Safari directly

[Deep Fried Geekboy]

You can open them directly in Safari and cut/paste into TextEdit too.

Re:Works in Safari directly

[Nutria]

You can open them directly in Safari and cut/paste into TextEdit too.

It will paste into any text editor, even vi-inside-an-xterm-window.

works in older acroread too

[FudRucker]

i keep an older version of adobe's acrobat reader for Linux version 5.0 and copy & paste in to a text editor works in it too...

i hate the new acrobat reader. some claim it calls home to the mothership(Adobe) which i dont approve of either (spyware)...

Re:works in older acroread too

[Sir Codelot]

i hate the new acrobat reader. some claim it calls home to the mothership(Adobe) which i dont approve of either (spyware)...

Then you should try Foxit Reader [foxitsoftware.com]. Apart from being free, light-weight and best for everyday use, it also has got a 'Fox' in its name. :)

Even more shocking

[$RANDOMLUSER]

What's this in TFA about Barry Bonds and steroids? I had no idea.

Redacting right is HARD

[nweaver]

Redacting electronic documents right is HARD. See, for example, The NSA's guide to redacting word documents as PDF [fas.org].

Re:Redacting right is HARD

[fistfullast33l]

From the document:

This page intentionally left blank.

I was going to say, those guys are goooood.

Re:Redacting right is HARD

[gEvil (beta)]

Yeah, except the page isn't blank. Hmmm, sounds like more of that NSA doublespeak to me...

Re:Redacting right is HARD

[Rob T Firefly]

Turning leaked information back into a secret, that's the HARD bit.

Funniest post ever!!!

[technoextreme]

Redacting electronic documents right is HARD. See, for example, The NSA's guide to redacting word documents as PDF.

Im reading the instructions and skimming through them and what do I see?? A bretheren of clippy. At one point it seems like she/he is writing down all the secrets. Either one of two things is going on. The document is a fake or I should join the government because they need all the help they could get.

Re:Redacting right is HARD

[quokkapox]

Two things: 1) Why not have a handy context menu option, "Redact selection", available with a right click on the selected object? 2) Awwww, the NSA uses the little kitty cat assistant instead of Clippy. Just like my mom. Until I gave her openoffice.

Re:Redacting right is HARD

[gEvil (beta)]

Why not have a handy context menu option, "Redact selection"

Because management and clueless users will demand that there be an "unredact selection" menu option, also. I'll let you sort out the implications of that. Either that or original copies of documents everywhere will have text permanently blocked out by the above-mentioned clueless users and management types.

Re:Redacting right is HARD

[More Trouble]

Looks like you're redacting that document. NSA Office Kitty can help! First, tell us what you're trying to hide:

• gov'ment impropriety
• financial cheating
• illicit sex
• other

Yeah, cut them some slack

[shrubya]

You gotta give those poor cubicle folk some credit for trying ... at least they aren't putting Wite-Out on the screen, right?

Seriously though, if the government gets TOO embarrassed about this sort of thing, they'll do something even more stupid, and mandate all official documents to use some proprietary DRM/TPM/HDCP/BVD format that only Windows Vista can display.

Re:Redacting right is HARD

[RobertB-DC]

Redacting electronic documents right is HARD. See, for example, The NSA's guide to redacting word documents as PDF [fas.org]

At least it's obvious that the folks who know what they're doing, know that MS products aren't the best solution. From the doc:

Microsoft Word XP/2003: Microsoft has attempted to remedy certain issues with Metadata in Office XP and up by including a menu option to remove personal information (metadata). There
is also a tool available for free from MS, Remove Hidden Data 1.0 (for XP) and 1.1 (for

Re:Redacting right is HARD

[Pendersempai]

It's not hard; people just have to manually delete (not obscure) data they want redacted. Then all outgoing Word files should be scrubbed of metadata. There are commercial packages, included in many groupware suites, that do this automatically. At the law firm where I work, every single Word file that gets emailed to an address outside the firm is automatically scrubbed of metadata by the server. If you try to save a document with Track Changes enabled, a dialog box warns you. If you try to email a document

Re:Redacting right is HARD

[MrCopilot]

17 Pages. Note to NSA.

There is a much Simpler Solution.

1.)Print Document.
2.)Locate and uncap Sharpie.
3.) Blackout Text.
4.) Scan to DocRedacted.pdf
Wow less than the average government paragraph. Seems like the way they have been doing it for years why change now?

Cache

[Rob T Firefly]

Coral cache of the PDF [nyud.net]

Anyone into mirroring it?

PDF Redaction

[Fedallah]

This is pretty ridiculous. Products have existed for years to take care of this sort of thing, such as http://www.appligent.com/products/product_families /redaction.php [appligent.com].

How does this keep happening?

I wonder how long it'll be...

[TWX]

...before they are told to just take a print-screen of the document, page by page, then use a graphics program to install the black boxes over words, then import each image as a page into their PDF creator...

Re:I wonder how long it'll be...

[gEvil (beta)]

...then import each image as a page into their PDF creator...

While not a bad suggestion, there is a major problem with it. Many offices will use Paint for this process, with the final image saved as a bitmap. Ever tried making a PDF file out of 8.5x11 inch bitmap images? The resulting filesize tends to be pretty nasty. Of course, there are ways around this, but the requisite knowledge of graphics is far beyond the knowledge necessary to understand that white text is still text--e.g., if you can properly

Disability guidelines prohibit rasterized docs.

[Kadin2048]

I am pretty sure that rasterized PDF documents violate government disability-access guidelines, since they can't be read with screenreaders, braille terminals, or basically anything other than a set of human eyes (or a good OCR program).

They would be a lot better off going through the document in Word (or Notepad/Textedit/vi/EMACS/whatever) and just selecting the regions of text that they want to remove, and replacing it with [-- TEXT REMOVED --] or even [REDACTED]. If they were really slick, I'm sure somebody could write a little macro to replace the text with an equivalent number of characters of whitespace or random text or dashes, to preserve formatting. (Okay, so to really preserve the formatting it would have to be replaced with characters that have the same amount width as the deleted characters; maybe there's a font-set containing various widths of whitespace characters that they could use? In TeX it would be trivial.)

The results would be ugly (but really, were black bars ever very beautiful?) but at least it would actually remove the information, and wouldn't result in an inaccessible, rasterized document.

Copy? Paste?

[TheCarp]

Shit... use the default pdf viewer in ubuntu evince 0.5.2

Just select the text and bang,.... there it is for reading.

-Steve

Nice and secure. Riiiiggght...

[blcamp]

Really nice to know that these folks has taken an apparent cue on safe and secure documents from the folks in Redmond.

On a serious note... this is seriously scary. Imagine if the NSA and other agencies are redacting all of their documents this way an passing them around the world to field offices, embassies and elsewhere.

Imagine the implications during legal proceedings here in the States. Yikes.

Re:Nice and secure. Riiiiggght...

[DarkVader]

This is NOT scary. This is refreshing.

I would much prefer my government be unable to successfully keep secrets from me.

blonde joke

[k4_pacific]

Q: How can you tell when a blonde NSA agent has been redacting PDFs?

A: There is magic marker ink all over the screen!

The New Way for Gov't Transparency

[thatguywhoiam]

I love this idea.

Leave PDF the way it is. In fact, make it really hard to actually redact something, but put a tool front-and-center that looks like its redacting something.

Then - remove any delete capability from Outlook. Trash is fine, but not delete.

Then - configure all Windows machines to be inherently wide open, so that we may all peer into gov't computers. Oh wait, this is already true.

Sometimes I think those in positions of high gov't power should forfeit practically all privacy for the duration of their term. Put a webcam on these fuckers 24/7. Does that sound... draconian? Unreasonable? Maybe. But after losing billions of dollars in things like Iraq military contract debacles, I don't trust any of these people. They certainly don't trust us.

Someone missed the memo

[Tozog]

Here's how the NSA recommends redacting files:

http://www.nsa.gov/snac/vtechrep/I333-TR-015R-2005 .PDF [nsa.gov]

Re:Someone missed the memo

[Neil Watson]

The animated moggie Word assitant really adds a professional touch ;)

Re:NSA procedure sucks!

[wiredlogic]

FWIW you could very easily write up some VBA code that converts highlighted text (maybe just one specific color like red) into Xs. Then you would just have to highlight the redacted sections and run the macro when you're finished. The highlighting could be optionally kept in place to make it more visible in the PDF. It also would be useful for an actively changing document to make the author more aware of where the sensitive bits are.

Pretension

[GonzoTech]

http://www.sfgate.com/cgi-bin/object/article?f=/c/ a/2006/06/21/MNGUAJI4B85.DTL&o=0 [sfgate.com] The two reporters "are the only individuals, other than the leaker himself, who would have personal knowledge of the leaker's identity," Hershman and Raphael said.

Is it just me or do they look a little pretensious?

That's the problem with these powerful formats

[hey!]

Like .doc, .pdf, and AFAIK the opendoc format.

It's the same old story as with operating systems or anything else: features are usually either a plus or a "don't matter", except when serious security issues are involved, in which case you can't always predict what is benign, whether in and of itself or in combination with other features. Adobe tried to position PDF for all kinds of other things like portable forms and collaboration, but obviously their users are running into the same problems ad MS Word users have with leaking sensitive information.

What there should be is a standard document format for outside release of legal or sensitive documents, that doesn't have any features that could be inadvertantly used. Maybe it is RFT or a stripped down PDF; but something where you can tell the intern to release this press release, and not count on him being smart enough to check for hidden comments and workflow information. It sould be WYSIAYG -- what you see is ALL you get -- and any additional features, other than possibly a small and well defined set of metadata, should parse as an error.

Re:That's the problem with these powerful formats

[mcmonkey]

Maybe it is RFT or a stripped down PDF; but something where you can tell the intern to release this press release, and not count on him being smart enough to check for hidden comments and workflow information. It sould be WYSIAYG -- what you see is ALL you get -- and any additional features, other than possibly a small and well defined set of metadata, should parse as an error.

You mean like plain text (txt)?

Adobe can come out of this smelling like a rose!

[Laura_DilDio]

Add a "redact" tool to the existing toolbar!

Re:Adobe can come out of this smelling like a rose

[ray-auch]

Kind of kills the market for the third party vendors who already provide tools which do just that.

Maybe those vendors would have an anti-trust case against Adobe for doing it.

Would be ironic given Adobe's anti-trust allegations against MS for essentially doing the same thing (adding a "Save as PDF" tool to the MS office toolbar).

Hush! Hush!

[Anonymous Coward]

Why are we publicizing this flaw? We have a US Government in power that increasingly wants to peer into the lives of innocent citizens, while becoming less transparent itself in order to cover up deceit, fraud, abuse, and just plain bumbling incompetence. If these Keystone Kops want to believe that they are criminal masterminds, let them, but don't help them actually cover stuff up!

This frightens me!!!!!

[waif69]

Having worked for the gov't and knowing that some documents that I have signed and worked on should be redacted, this scares the crap out of me. It's not that I did anything that was illegal or "evil" as google would put it, I just don't want the "bad guys" (terrorists, etc.) knowing my name is attached to anything that resulted in their cohorts arrested or killed on the battlefield (also includes CONUS since 9/11).

Normal average government workers should NOT be redacting, the people who redact should be those who know that if they screw-up, they may be screwing themselves or good friends in the process. Have people do it(redact) who have something to lose.

Just my 2 cents.

Re:This frightens me!!!!!

[dinsdale3]

I just don't want the "bad guys" (terrorists, etc.) knowing my name is attached to anything that resulted in their cohorts arrested or killed on the battlefield

Its a good thing you haven't told anyone, then.

That's just what's called

[alewar]

"Security by obscurity" :)

We have to act!

[Waffle Iron]

Clearly, these information leaks are a major security threat that is aided and abetted by these renegade PDF viewers. I'm encouraging my representatives in Congress to introduce a "Digital Millennium Redaction Act" that will prohibit the manufacture, sale, discussion or hyperlinks to any PDF viewers which enable the illicit extraction of redacted data from PDF documents. Such viewers are little more than the preferred tools for information thieves, hiding in the guise of "productivity applications". It's despicable.

This law would instruct the FCC to create a program to certify approved PDF viewers; such viewers must make it impossible for users to steal the redacted data in a file, along with technical measures to prevent tampering with the viewers by hackers. Certified viewers will be made available to the public by software companies on a list of government-approved PDF vendors. After it becomes illegal to own a non-certified pirate PDF viewer, these dangerous information leaks will thankfully become a thing of the past.

Seems to be a common occurrence

[milgr]

I googled for redacted doctuments, chose some pdfs at random, and found that the text is behind the black bars.

When I started searching, I googled for redact. There were two ads for products that remove the text from the pdf as well as create the black bar. One made it clear that the text would be inaccessible from hackers.

So, why aren't these types of tools being used for all redactions?

Congratulaitons.

[sammy baby]

Congratulations, Slashdot! The FBI will be along shortly to raid your offices on suspicion of violating the DMCA, the Patriot Act, and probably some other bullshit piece of legislation we don't even know about.

Oh, yeah - it's a no-knock warrant, so put your pants on now.

Re:Congratulaitons.

[BandwidthHog]

/. doesn't host with AT&T, so no worries.

Doesn’t necessarily matter. Just because I don’t purchase services directly from NSAT&T, that doesn’t mean that my data isn’t flowing through their network at some point on its journey. So while I am immune (for now?) from NSAT&T’s content ownership bullshit, I can’t count on not having them dump my packets into Cheney’s inbox.

Clear as Mud

[Doc Ruby]

Why doesn't Adobe upgrade their PDF generators to include a "Real Redact" button that actually deletes the redacted data? They could sell it to governments at the usual 1000x government markup rate, and the government would probably still save money vs the fallout from these illusory blackout follies. Neither the government nor Adobe is in the "freedom of information" camp. Maybe the government just refuses to buy an upgrade because that would save money overall.

Command Line Programs; evince

[Noksagt]

CLI programs are REALLY useful to look at "hidden" content.

'pdftotext' comes with xpdf & is even available natively on windows.

Similarly, for MS Word documents, you may use 'antiword' [demon.nl], 'catdoc' [free.net], and 'wv' [sourceforge.net].

These programs are quite nice in that they can easily batch-process a lot of documents & then you can go grepping through them for interesting tidbits.

(On the GUI front, evince [gnome.org] deserves a plug. It uses the same poppler [freedesktop.org] backend as xpdf and kpdf. I used to use tiny & fast xpdf for most of my pdf viewing, but evince has a few nice features which xpdf lacks & has become my personal favorite pdf viewer.)

Common problem with today's UIs

[Namlak]

The industry at large (Microsoft being a big offender) has been trying to get us to a this magical place where everything is system and location independent and this is where we end up:

1) FTP sites in Windows Explorer look like regular Windows folders. People expect them to work like regular folders. I had a field sales force try to "share" an Excel spreadsheet expecting the others to get a "Read Only" copy just like would happen on a local network share. Overwriting madness ensued. You can't blame them, there was no indication that it would work differently. Asking them to understand FTP is like accounting expecting me to fully understand the accounting rules behind my IT purchases.

2) A manager where I used to work had an Excel spreadsheet with payroll data for the entire company. He wanted to send each department their subset of the data. So he filtered his spreadsheet and sent the filtered lists to each department not knowing that he was sending each department the whole list under teh covers. Luckily, the file was 30MB and choked in the mail server and I was able to bail him out of that huge mistake. But you really can't blame him - he saw something on the screen and sent "it". There should be an indication of underlying data. BTW, doing a cut and paste special made each file about 25k or so.

Same thing with this PDF error. If your file shows certain information, it should contain that information only or indicate (or warn) otherwise.

By "simplifying" everything, nobody knows what's really going on. A couple times per week I have to explain some type of issue to some user about how "It's really more complicated than that, see Windows (or an app) hides this from you." User roll eyes as their simple task has become obscurely complicated - all in the name of making things "easier" to understand, ironically.

If something works different, it should be displayed different - that at least gives the user a chance to question what they are doing.

Re:Common problem with today's UIs

[BandwidthHog]

So he filtered his spreadsheet and sent the filtered lists to each department not knowing that he was sending each department the whole list under teh covers.

I use that feature quite often and it was only yesterday that I noticed that the little triangle turns from black to dark blue when you’re viewing a filtered set. All this time I was thinking there really ought to be some sort of visual indication (other than the wonky row numbers).

MS Word Redaction Tool

[blackstripe]

Assuming the original document was in Word format, I'm surprised they didn't use Microsoft's freely available redaction add-in [microsoft.com].

Er... pdftotext...?

[digital photo]

Okay... this is what is considered secured??

Using a STANDARD pdf handling tool:
% pdftotext BALCO_quash_subpoena_sfchronicle.pdf

From the PDF->TXT file:

[snipped to first line before the "blacked out section"]

  C. Movants' Efforts to Obtain the Secret Grand Jury Transcripts

[beginning of first blacked out section]

Prior to the return of the Balco indictments, the lead defendant, Victor Conte ("Conte"), began to correspond via e-mail with Movants. (See Ex. 1 to Donnelan Aff.). Neither Movants nor Conte attempted to keep their relationship confidential, as the e-mail correspondence routinely was reported by Movants.2 (Exs. 1, 2, 3, and 11 to Donnelan 1

[... snipped for berevity ...]

On June 23, 2004, Fainaru-Wada sent an e-mail to Conte indicating that he (Fainaru-Wada) was busy working on some stories that may be "up on the web soon. Hope you like t
hem." (Ex. T to Hershman Decl.). Conte responded that he was looking forward to seeing the article and that his lawyer would be available for comment. (Id.).

[end of first blacked out section]

D. Disclosure of the Montgomery Grand Jury Transcript On June 24, 2004

[more, but why post it when you can read it yourself!?]

Okay... WTF!? Doesn't ANYONE check this stuff before it goes out the door!?

OMG! Wonder if this is how our private documents are "made safe"....

Re:A redacted document? Say it ain't so!

[op12]

The document was edited (by the second definition you listed) to put a black bar over some text. What's wrong with that? Replace the word redacted with edited and it makes sense: "...U.S. government apparently releasing a redacted version of their court filing in the Balco grand jury leak case which merely stuck a black line over the text"

Re:A redacted document? Say it ain't so!

[The Only Druid]

"Redacted" is a legal term of art (i.e. it has a special meaning in the legal context).

For lawyers/courts/etc., redacted (Per Black's Legal Dictionary) means:

n), n. 1. The careful editing of a document, esp. to remove confidential references or offensive material. (Cases: Criminal Law 663; Federal Civil Procedure 2011; Trial 39. C.J.S. Criminal Law 1210-1211; Trial 148-153.) 2. A revised or edited document. -- redactional, adj. -- redact, vb.>

The lesson here is this: if you see a word used in a legal context (or any professional context) and it sounds entirely wrong...ask yourself first whether it might have a special meaning before complaining.

They're correct.

[Kadin2048]

Their use of redact is completely correct.

If I am releasing a document for publication and decide to remove information from it, this is redaction. It's editing for publication, which can include the removal of information. It could also include the addition of new information, but that's not what typically happens. Redaction can be a form of self-censorship, but it's not always the same.

Censorship is when a third party, generally a person in authority, suppresses information which is considered objectionable. The 'authority' can be the same as the author (e.g. 'self-censorship'), or the suppression can be indirect -- it need not be editing per se.

It's my understanding that "redact" is used only in reference to written documents that are being edited, while 'censor' is more general and can refer to anything. The terms are closely related, especially in their typical use, but they're not exactly the same. "Redact" is actually a more specific and precise word for what's going on in this instance. We can argue about whether censorship is also going on, but redaction definitely is.

Anyway, arguing about definitions by citing dictionaries is always a bit pedantic, since dictionaries are not authoritative except as a historical reference: they can tell you what a word meant at the time the dictionary was written, but not what it means right now, since a word's definition is determined by its usage. All language is inherently arbitrary: they're just sounds we make or things we write down in order to convey ideas, and the relationship between the sounds/characters and ideas is not fixed, but infinitely variable. If everyone were to decide tomorrow that 'redaction' meant the same thing as 'censorship,' that's what it would mean, and next year's dictionaries would have to be updated to reflect that.

Re:And these are...

[wolenczak]

Don't mind getting into a PhD, it is not only about money, with your orthography (or careless writting) you would have your teachers all over you the very first day.

Re:And these are...

[drc500free]

"If you want my opinion (or even if you don't...:-p) this is the achelle's(sp) heel of our society today, most people are lazy bastards that just want to get done with somethign without learning anything about it."

"Another thing that pisses me off is incopetence."

Oh, the irony.

Re:This proves it:

[Svartalf]

Excuse me, any electronic format, unless it is a bitmap format, will have this problem unless
all the viewers 100% honor the redaction as it's intended. In the case of a bitmap format,
you can burn a black or white rectangle into the original image and then add an annotation
a la TIFF's annotations that contains the original portion of the image that was redacted
in an encrypted format so that it's difficult to expose the redaction- IF you need to have
the redaction exposed. If not, you hand across the redacted image as-is without annotations.

This has NOTHING to do with PDF or ODF at all- trying to make this a connection to these
is bogus to say the least. In this case, I believe that the people doing it used the MS Office
redaction capabilities and then exported the redacted content to PDF, which the export
carried the same sort of redactions across to the other format. What happened is because
someone didn't understand the tools they were using, not because of PDF or ODF.