×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

National Archives Cuts Back On Web Site Archiving

CmdrTaco posted about 6 years ago | from the i-remember-everything dept.

Government 45

hhavensteincw writes "The National Archives and Records Administration (NARA) is coming under fire for a new policy to stop the "harvesting" of a digital snapshot of all federal agency and Congressional Web sites after every Presidential and Congressional term. NARA, which archived more than 75 million Web sites in 2004 after George Bush's first term ended, will not harvest agency and Congressional Web sites when his current term is over because it says agencies are supposed to be archiving Web content on their own. But NARA has been criticized by some for opting out of preserving these important historical archives on the Web."

cancel ×
This is a preview of your comment

No Comment Title Entered

Anonymous Coward 1 minute ago

No Comment Entered

45 comments

interesting in consideration..... (3, Interesting)

3seas (184403) | about 6 years ago | (#23046530)

... the price of storage dropping as it has.

So what is the real reason for this? Its certainly not cost.

Is it possible that nobody is interested in the data?

Re:interesting in consideration..... (5, Insightful)

bumburumbi (1047864) | about 6 years ago | (#23047048)

Is it possible that nobody is interested in the data?
People may not be interested in the data now, but as time passes, it will become more and more important. I am a bit surprised that the National Archives and the Library of Congress collect so little of the American cultural heritage. In Iceland, where I live, the National Library collects everything on the national TLD (is) three times a year, important sites are crawled more frequently. I know that the US web is several orders of magnitude larger than the Icelandic web. One would however assume that the resources available to the NARA and LC are significantly larger than what the Icelandic National Library has to spend on collecting websites. Collecting a subset of the US web every four years should be well within the means of the US government.

Re:interesting in consideration..... (1)

FishWithAHammer (957772) | about 6 years ago | (#23048936)

One would however assume that the resources available to the NARA and LC are significantly larger than what the Icelandic National Library has to spend on collecting websites.
Oh god, I laughed so hard.

Re:interesting in consideration..... (4, Insightful)

Kwirl (877607) | about 6 years ago | (#23047092)

I think we all know that the less history remembers of George W Bush's term as president of the free world, the better off we will look in our children's eyes. If he gets lucky he might get off easy with a 'worst president to ever hold the office' footnote.

Re:interesting in consideration..... (3, Insightful)

Foobar of Borg (690622) | about 6 years ago | (#23047182)

If he gets lucky he might get off easy with a 'worst president to ever hold the office' footnote.
Of course, like with Nixon, you will still have slavering beasties defending him for the next few decades and blaming everything on liberals and campus radicals.

Re:interesting in consideration..... (1, Interesting)

Anonymous Coward | about 6 years ago | (#23047226)

Well, if you can tell me what part of a) opening up dialogue with pinko, commie China and b) getting OUT of Vietnam where Kennedy and Johnson (both "liberals," last time I checked) makes Nixon a "conservative," then please let me know.

Frankly, I'd have thought the bastard would be more palatable to lefties than to the Buckley crowd -- then again, it may just be that the neocons have confused the definitions so much as to make them meaningless.

Re:interesting in consideration..... (1)

Foobar of Borg (690622) | about 6 years ago | (#23047810)

Well, if you can tell me what part of a) opening up dialogue with pinko, commie China and b) getting OUT of Vietnam where Kennedy and Johnson (both "liberals," last time I checked) makes Nixon a "conservative," then please let me know.
Nixon was not a leftie by any stretch of the imagination. Opening up a dialogue with Communist China was about realpolitik, not ideology. Plus, Communist China was hardly a bastion of liberalism. It was/is an autocratic regime. You seem to be confusing Communism and Socialism, which are two incredibly different ideals (Sweden and Canada, for example, are not going to be opening up any re-education camps anytime soon). As far as getting out of the Vietnam War, that was something Kennedy likely wanted to do, but Johnson didn't. While Johnson supported a lot of liberal causes (civil rights and so on), the Vietnam War was something *liberals* were protesting him about. It was the *conservatives* in the US who were all for the war and had the whole "America, love it or leave it" mentality, without pausing to ask why we were even in Vietnam (sound familiar?).

Frankly, I'd have thought the bastard would be more palatable to lefties than to the Buckley crowd -- then again, it may just be that the neocons have confused the definitions so much as to make them meaningless.
Again, you are cherry picking. And Buckley liked Nixon. Sure, Nixon looks like a liberal now compared with the neo-Nazis we have running the country, but at the time he would be considered very conservative. In the US, we don't have a liberal party anymore. We have the right-wing party (Democrats, with a few being so "liberal" as to be centrists) and the neo-Nazi party (Republicans).

Re:interesting in consideration..... (0)

Anonymous Coward | about 6 years ago | (#23048092)

Canada and Sweden do have "reeducation camps" -- when they lock people up for "hate speech" and the like, for the crime of suggesting that maybe, just maybe a Nigerian isn't a Swede no matter what it says on their passport.

Why? Because "racism" is the only thing that stands in the way of the international worker's brotherhood crap. Capitalism and Communism are two sides of the same coin. Is socialism bad? No. is social marxism a problem? yes. it defies reality.

Re:interesting in consideration..... (1)

UncleTogie (1004853) | about 6 years ago | (#23047204)

I think we all know that the less history remembers of George W Bush's term as president of the free world, the better off we will look in our children's eyes.

Far better our children be aware of history, so they might be less inclined to repeat it.

Looking good in the eyes of another is not nearly as desirable as acting good and eliminating that worry.

Re:interesting in consideration..... (0)

Anonymous Coward | about 6 years ago | (#23049338)

I once worked on a proposal for the Electronic Records Archive for NARA. At the time, the agency's budget was on the order of $30M/year. They anticipated 17 Exabytes of data by 2017. Needless to say, they couldn't afford anything that would have done the job. Don't discount funding -- or a ploy to have it increased, anyway. They're short-funded as it is.

Wrong Time to Quit (5, Insightful)

Doc Ruby (173196) | about 6 years ago | (#23046544)

The NARA should not be considering quitting right when the Bush regime is caught red-handed deleting vast amounts of incriminating digital content that it was legally required to archive.

If anything, NARA should be required to archive even more now, to guard against losing the unique copies at the other ends of official communications and publications. It should upgrade to a policy of redundant archivers keeping separate copies under separate policies, so that a rogue Executive can't flip one switch and toss all the evidence of their actions into the fire.

Re:Wrong Time to Quit (2, Insightful)

Anonymous Coward | about 6 years ago | (#23046806)

I'm not certain if you read TFA (or TFS, for that matter), but these are public websites that the NARA was archiving. They were doing it ONCE every term. If you want to see just what the NARA was doing, click on "Cached" on Google's search page...same idea.

Honestly, I'm not pro-Bush by any stretch of the imagination, but the NARA's decision is NOT going to help the Bush "regime" hide anything that wasn't already readily accessible to the public.

Re:Wrong Time to Quit (1)

Doc Ruby (173196) | about 6 years ago | (#23047058)

I'm familiar with NARA's archiving, and how they're thinking of turning it over to the Internet Archive.org. But there's a difference between publishing and archiving. Yes, those websites are available to the public, but if there isn't a mandated archiving system, then lots will slip away. The sheer volume of published materials, so often revised to cover up abuses after it's slipped out, means that relying on the public to archive it piecemeal will risk lots of important evidence being lost. That's the entire point of the government archiving even public materials.

Probably the best system would be for libraries, public and private (eg. universities and research institutes), to each independently archive the public sites, the way they do now with newspapers (microfilm, and lately CD/DVD-ROM). If we'd just relied on newspapers to keep their own archives, lots of coverups by newspapers could have revised history in their own master archives by now (and surely have tried). The government material is even more essential, and subject to coverup by an organized criminal regime like the one we're living through right now.

Re:Wrong Time to Quit (0)

Anonymous Coward | about 6 years ago | (#23047212)

And I understand your points as well. However, I honestly have to wonder, with NARA's practices even BEFORE this recent change, what are the odds of picking up something "incriminating" during that once every term archive? It's not like this is a hourly, daily, or even weekly backup...we're talking once every four years.

What you're talking about is something I DO agree with - the NARA should be taking much more regular archives of internet materials. However, with the current scope of the project, it doesn't make any sense to even bother, imo.

Re:Wrong Time to Quit (2, Insightful)

Doc Ruby (173196) | about 6 years ago | (#23047312)

I think something is better than nothing. That volume of evidence and "virality" of distribution means that even a snapshot will preserve traces that are hard to totally expunge from the entire Federal government's public records. But if that snapshot isn't even taken, that's much harder.

The dropping from inadequate archiving to none has crossed a threshold where people are now paying attention and demanding adequacy. The inadequacy of the prior policy means that both those in power in the Bush regime and many outside it agree on changing the program, which is a start for political compromise. Switching to the Internet Archive is a mediocre interim measure, but one which Republicans probably don't like, because even though it's their trademark privatization, it's still publicly funded, and not to a crony just skimming a contract while failing to expensively fulfill it. All of which creates political conditions and momentum towards a more distributed archival process, which could fund archives including libraries as I described.

So instead of giving up, now is a good time to demand more and better. Because it's the right thing to do, and because the way it's happening shows a path to actually getting it.

Re:Wrong Time to Quit (4, Insightful)

the pickle (261584) | about 6 years ago | (#23047266)

The NARA should not be considering quitting right when the Bush regime is caught red-handed deleting vast amounts of incriminating digital content that it was legally required to archive.

Am I the only one who read this story and thought that maybe the NARA isn't choosing to do this? I think it's a mighty strange coincidence that they'd be doing this on their own in the last year of a presidency that, for the past seven years, has shown a willful disregard for the law, especially when it comes to the administration's own recordkeeping. Dubya's White House has made the missing files associated with the Clintons look like a single lost receipt by comparison.

p

Just outsource it to Google (0)

Anonymous Coward | about 6 years ago | (#23046620)

I for one welcome our new Googlovernment.

Re:Just outsource it to Google (0)

Anonymous Coward | about 6 years ago | (#23047782)

Well, whether or not it is outsourced to Google, it would certainly be very "Not Evil" of Google to make (a subset of) their cached pages available to projects like archive.org or the Natioal Archives. Say, one of their robot hits per month per site, or quarterly, or even more. Whatever.

Either way, they have the data, and the robots, and I am all about independently derived data, and multiple parallel datasets. It provides another layer of transparency, y'know?

Should we be surprised . . . (2, Interesting)

TXISDude (1171607) | about 6 years ago | (#23046658)

It really should not come as a surprise that yet another federal agency has decided not to do its job, but only what it wants to do. . . The reality of the situation is simple, the web is becoming a major communications method for the government, and the content will be a lens into the history of the government's interaction with the people. I am actually afraid that this "ignoring the present" is not some form of conspiracy to prevent the recording of history, but more of a case of senior government officials not understanding the world as it is. Not recording the communications of the government to the people, in the form and context of how they were presented is a complete abdication of the responsibilities assigned to NARA and I hope that this story gets the US Congress to intervene and tell teh agency to do its job. Of course, I also hoped that Santa would bring me a new car, and the Easter bunny would bring golden eggs. So, I am ready for another disappointment.

Easy answer (1)

witherstaff (713820) | about 6 years ago | (#23046842)

Take whatever budget they have for the web archive and give it to archive.org, let them do the work. Include some long term DVD tech to stash at the library of congress. If the gov't can't do its job, pay someone else to do it.

corepirate nazis reducing chance of history.... (-1, Troll)

Anonymous Coward | about 6 years ago | (#23046690)

coming back to haunt them? yet another big surprise? let yOUR conscience be yOUR guide. you can be more helpful than you might have imagined. there are still some choices. if they do not suit you, consider the likely results of continuing to follow the corepirate nazi hypenosys story LIEn, whereas anything of relevance is replaced almost instantly with pr ?firm? scriptdead mindphuking propaganda or 'celebrity' trivia 'foam'. meanwhile; don't forget to get a little more oxygen on yOUR brain, & look up in the sky from time to time, starting early in the day. there's lots going on up there.

http://news.yahoo.com/s/ap/20071229/ap_on_sc/ye_climate_records;_ylt=A0WTcVgednZHP2gB9wms0NUE
http://news.yahoo.com/s/afp/20080108/ts_alt_afp/ushealthfrancemortality;_ylt=A9G_RngbRIVHsYAAfCas0NUE
http://www.nytimes.com/2007/12/31/opinion/31mon1.html?em&ex=1199336400&en=c4b5414371631707&ei=5087%0A

is it time to get real yet? A LOT of energy is being squandered in attempts to keep US in the dark. in the end (give or take a few 1000 years), the creators will prevail (world without end, etc...), as it has always been. the process of gaining yOUR release from the current hostage situation may not be what you might think it is. butt of course, most of US don't know, or care what a precarious/fatal situation we're in. for example; the insidious attempts by the felonious corepirate nazi execrable to block the suns' light, interfering with a requirement (sunlight) for us to stay healthy/alive. it's likely not good for yOUR health/memories 'else they'd be bragging about it? we're intending for the whoreabully deceptive (they'll do ANYTHING for a bit more monIE/power) felons to give up/fail even further, in attempting to control the 'weather', as well as a # of other things/events.

http://video.google.com/videosearch?hl=en&q=video+cloud+spraying

dictator style micro management has never worked (for very long). it's an illness. tie that with life0cidal aggression & softwar gangster style bullying, & what do we have? a greed/fear/ego based recipe for disaster. meanwhile, you can help to stop the bleeding (loss of life & limb);

http://www.cnn.com/2007/POLITICS/12/28/vermont.banning.bush.ap/index.html

the bleeding must be stopped before any healing can begin. jailing a couple of corepirate nazi hired goons would send a clear message to the rest of the world from US. any truthful look at the 'scorecard' would reveal that we are a society in decline/deep doo-doo, despite all of the scriptdead pr ?firm? generated drum beating & flag waving propaganda that we are constantly bombarded with. is it time to get real yet? please consider carefully ALL of yOUR other 'options'. the creators will prevail. as it has always been.

corepirate nazi execrable costs outweigh benefits
(Score:-)mynuts won, the king is a fink)
by ourselves on everyday 24/7

as there are no benefits, just more&more death/debt & disruption. fortunately there's an 'army' of light bringers, coming yOUR way. the little ones/innocents must/will be protected. after the big flash, ALL of yOUR imaginary 'borders' may blur a bit? for each of the creators' innocents harmed in any way, there is a debt that must/will be repaid by you/us, as the perpetrators/minions of unprecedented evile, will not be available. 'vote' with (what's left in) yOUR wallet, & by your behaviors. help bring an end to unprecedented evile's manifestation through yOUR owned felonious corepirate nazi glowbull warmongering execrable. some of US should consider ourselves somewhat fortunate to be among those scheduled to survive after the big flash/implementation of the creators' wwwildly popular planet/population rescue initiative/mandate. it's right in the manual, 'world without end', etc.... as we all ?know?, change is inevitable, & denying/ignoring gravity, logic, morality, etc..., is only possible, on a temporary basis. concern about the course of events that will occur should the life0cidal execrable fail to be intervened upon is in order. 'do not be dismayed' (also from the manual). however, it's ok/recommended, to not attempt to live under/accept, fauxking nazi felon greed/fear/ego based pr ?firm? scriptdead mindphuking hypenosys.

consult with/trust in yOUR creators. providing more than enough of everything for everyone (without any distracting/spiritdead personal gain motives), whilst badtolling unprecedented evile, using an unlimited supply of newclear power, since/until forever. see you there?

"If my people, which are called by my name, shall humble themselves, and pray, and seek my face, and turn from their wicked ways; then will I hear from heaven, and will forgive their sin, and will heal their land."

meanwhile, the life0cidal philistines continue on their path of death, debt, & disruption for most of US. gov. bush denies health care for the little ones;

http://www.cnn.com/2007/POLITICS/10/03/bush.veto/index.html

whilst demanding/extorting billions to paint more targets on the bigger kids;

http://www.cnn.com/2007/POLITICS/12/12/bush.war.funding/index.html

& pretending that it isn't happening here;

http://www.timesonline.co.uk/tol/news/world/us_and_americas/article3086937.ece
all is not lost/forgotten/forgiven

(yOUR elected) president al gore (deciding not to wait for the much anticipated 'lonesome al answers yOUR questions' interview here on /.) continues to attempt to shed some light on yOUR foibles. talk about reverse polarity;

http://www.timesonline.co.uk/tol/news/environment/article3046116.ece

Getting sick of this (-1, Offtopic)

Anonymous Coward | about 6 years ago | (#23046760)

screw it. im just going to create a colony in space and declare independance.

Re:Getting sick of this (0)

Anonymous Coward | about 6 years ago | (#23046852)

You have to get launch clearance from the government to do that.

These archives are useless.... (4, Insightful)

Anonymous Coward | about 6 years ago | (#23046762)

Any archives done by the government are useless because those who control the government can modify them if they so desire. This data needs to be archived by multiple independent private parties.

Re:These archives are useless.... (0)

Anonymous Coward | about 6 years ago | (#23047630)

Most people who work at the National Archives are not political appointees that have reason to change archives; they are career civil servants. It is also easier to control the quality of archival preservation and conservation through a single government agency, which is extremely important if we want these documents available in fifty years. While there are problems in the government, there are also massive problems with quality control in the private sector.

The national archives exists for exactly this. (4, Informative)

DragonTHC (208439) | about 6 years ago | (#23046828)

their job is to archive public records. Every document produced by the US government is public record unless classified.

Re:The national archives exists for exactly this. (0)

Anonymous Coward | about 6 years ago | (#23047380)

And they are doing that. I think NARA is right about this. I work as an engineer for the federal government (NRC). I have a couple of thoughts about this.

1) NARA is like the storing house for the government. It is each individual agency's job to develop and maintain a Records Disposition Schedule that identifies what documents are considered Offical Agency Records (OARs) and how long they need to be archived for. The schedules usually dictate that important records will be transfered to NARA after a certain amount of time. While I can't speak for every agency, our agency's disposition schedule already contains a category for information posted to our website so why should NARA waste even one iota of effort to re-harvest information that they is already getting archived anyway?

2) Most websites nowadays are just views into a Content Management System with a database back end. What good does it do to archive a snapshot of all the webpages within a website on a particular day? The next day, the links get updated to point to new content and the old ones disappear. Even if I concede that it is worthwhile to grab a snapshot in time of a website's static html pages, what you probably end up with in reality is a set of html pages in which 75% of the links don't work. Yeah, that's useful.

Re:The national archives exists for exactly this. (0)

Anonymous Coward | about 6 years ago | (#23047410)

Exactly. And let's not forget that it's not up to any one person to decide if any given record is important. Even something of seemingly little relevance now could be sown to be very important in retrospect.

In short, it's not up for us to decide what documents are important and which aren't. It's up to history.

So as far as I'm concerned, any reasons they have for not keeping records of documents just doesn't hold water.

Repetative? (1)

tastypotato (1271894) | about 6 years ago | (#23046840)

Doesn't google do this already on their own servers?

Re:Repetative? (0)

Anonymous Coward | about 6 years ago | (#23047348)

IFAIK, Google only keeps the most recent snapshot from it's crawlers. That's not too useful after the sites are taken down.

National Archives have become redundant (1)

museumpeace (735109) | about 6 years ago | (#23046946)

why should the national archives repeat all the captured page loads that FBI and NSA are getting from the big telecom providers?...they don't just spy on your e-mail you know.

doublespeak (4, Interesting)

osssmkatz (734824) | about 6 years ago | (#23047576)

Back when archives.org was archiving whitehouse.gov, we saw changes in speeches to match the current rationales etc. Is this why they don't want to archive?

--Sam

NARA should continue archiving (1)

siculars (103175) | about 6 years ago | (#23048060)

I think it is a big mistake for NARA to stop what they are doing. A centralized authority bearing the imprimatur of NARA for creating, implementing, executing and enforcing a standard of archiving is desperately needed. This standard is critical for future historians to be able to make sense of our collective legacy.

Halting now and distributing responsibility amongst the various federal agencies will foster a haphazard distorted view of the past.

Other Significant National Archive Redactions (1, Informative)

Anonymous Coward | about 6 years ago | (#23048434)

Prior to 9/11, the presidential records of the first Bush presidency had been scheduled to be turned over to the National Archives, but the second Bush delayed their release.

Right after the 9/11 incident, these records were reclassified. Around the same time, there was a wholesale reclassification of documents in the National Archives going back to WWII, making them unavailable to the public.

Problem is bigger than Natl. Archives. (2, Informative)

joebob2000 (840395) | about 6 years ago | (#23049114)

Private archiving, (e.g. archive.org) coverage is not what it once was either, though maybe for different reasons.

More and more operators are choosing to protect their "intellectual property" using robots exclude, noarchive, or similar policies.

More and more websites use dynamic methods to present data, or use more complex interfaces involving javascript, flash, java, etc that make them technically hard to capture.

Conversations that formerly occurred on usenet now happen on proprietary bulletin board systems that are technically difficult to crawl. Furthermore, most BBS TOS forbid automated crawling.

It is interesting that as more and more content is backed by databases, it is getting harder and harder to access and search for the desired content.

Yes, but... (1)

webdog314 (960286) | about 6 years ago | (#23051514)

"NARA, which archived more than 75 million Web sites in 2004 after George Bush's first term ended, will not harvest agency and Congressional Web sites when his current term is over because it says agencies are supposed to be archiving Web content on their own."

Um, are these agencies the same ones that were supposed to be archiving all their e-mail as well? You know, the e-mail that was all conveniently deleted according to "procedure" just before it was needed in a major congressional investigation?

To be a historian in 100 years... (1)

Deviant (1501) | about 6 years ago | (#23052626)

I have studied a bit of history at the University level and I am not sure whether the digital age will make that job easier or harder in the future. With the overwhelming amount of online content in blogs and such it will be easier to find accounts of events but harder to seperate opinion from fact. It will be easier to search through being electronic but harder to sort through due to the overwhelming quantity of information on the current internet. It is also much easier to alter unless things like electronic hashes are stored along with the content. And that is with HTTP which is easily readable and not proprietary - I wonder how formats like MS Word docs are going to far with the test of time.

Are there even organizations out there archiving the wider internet for posterity? With published books they tend to be edited and distributed to libraries and preserved in a physical form where you can find them on the shelves 50 years from now. I don't know of any libraries storing/preserving electroic materials in the same way...

NARA Response (1)

paulwester (1273984) | about 6 years ago | (#23089978)

We read with interest your postings on this topic. The National Archives and Records Administration (NARA) has posted background information regarding our web harvest decision at http://www.archives.gov/records-mgmt/memos/nwm13-2008-brief.html [archives.gov]. This background document includes links to our guidance products related to web records and the decisionmaking process we went through to arrive at our decision. Paul M. Wester, Jr. Director, Modern Records Programs National Archives and Records Administration
Check for New Comments
Slashdot Account

Need an Account?

Forgot your password?

Don't worry, we never post anything without your permission.

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>
Sign up for Slashdot Newsletters
Create a Slashdot Account

Loading...