Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Storing Data For the Next 1,000 Years

kdawson posted more than 6 years ago | from the long-now dept.

Data Storage 243

An anonymous reader writes "This may be an interesting take on creating long-term storage technologies. A team of researchers at UCSC claims to have come up with a power-efficient, scalable way to reliably store data for a theoretical 1,400 years with regular hard drives. TG Daily has an article describing this technology and it sounds intriguing as it uses self-contained but networked storage units. It looks like a complicated solution, but the approach is manageable and may be an effective solution to preserve your data for decades and possibly centuries." Nice to see research on this using the kinds of real-world figures for disk lifetimes that recent studies have been turning up.

Sorry! There are no comments related to the filter you selected.

Only half the problem (4, Informative)

Raindance (680694) | more than 6 years ago | (#23167886)

Part of the solution to very long-term storage, of course, has to involve a method to read the data you've archived.

I tend to think systems such as the one described in the article aren't good long-term solutions. If their math works on the failure rates, that's fantastic- but just try to hook up a 2028 computer to one of these things to pull the data off.*

(Ever tried to get data off an obsolete tape backup?)

I think the most reliable archival system is going to be an active one, where data is saved on modern storage hardware and always copied to more modern tech as it arrives.

The other side of this is, for anything more advanced than text-- given that you can get at the data, what do you open it with? File types die over time and it's basically impossible to find programs to open certain files nowadays, much less such programs that will run on a modern OS. I think the answer to this has to be virtualization. Store the data *and* programs that can open the filetypes you need opened inside a portable virtual machine (e.g., a Windows vmware image). Over time, you may have to layer virtual machines inside virtual machines as OSes grow obsolete. But that's okay- virtualization is only going to become more elegant, and the end result is that you'd have your data in its original environment, completely accessible by native programs.

*Some elements of this problem could be solved by having backup servers use wireless and filesharing protocols that might stand the test of time- e.g., 802.11n and SAMBA. No need to just pick one 'most likely to be future-proof' combination, either: run bluetooth and serial access, webdav and a http fileserver, etc. Still, *not* storing data on modern hardware is always going to be a risky kludge.

There's probably room for a lucrative business based around this-- figuring out the most elegant way to archive and retain meaningful access to data under various computing/disaster scenarios. Hey, I do consulting. :)

Re:Only half the problem (4, Informative)

LoudMusic (199347) | more than 6 years ago | (#23167958)

(Ever tried to get data off an obsolete tape backup?)

I think the most reliable archival system is going to be an active one, where data is saved on modern storage hardware and always copied to more modern tech as it arrives.
Oh man, the headaches involved here. It only takes five years and archived data is obsolete. And yes, virtualization can help, but in the past I've resorted to keeping an entire system available, off-line, to guarantee that the client be able to open their data. Sometimes you get lucky and there's either a plug-in for the old app to export to the new app, or one for the new app to import from the old app. But even on the rare chance that one is available, I've never seen a 100% conversion - even on simple stuff.

Maybe old data was meant to die.

Re:Only half the problem (1)

IKILLEDTROTSKY (1197753) | more than 6 years ago | (#23168290)

I remember doing electrical work in the server room 2 years ago for a fairly large supermarket chain and over in the corner they had this IBM computer from the early 80's still running.

Re:Only half the problem (1)

inasity_rules (1110095) | more than 6 years ago | (#23168392)

Its not just old data... I recently had to virtualize an entire system using qemu, because the hardware died and the software did not support modern hardware.

On the plus side, the virtual machine will now run on any modern hardware - as long as it has an ethernet port for the dumb terminals to connect to. What else can you do with people who can't/won't upgrade their systems?

Re:Only half the problem (1)

hvm2hvm (1208954) | more than 6 years ago | (#23168776)

well the problem is not the hardware itself but the software. we need some basic standards that we can trust to be there in 50 or 100 years from now for networking, documents, video, music and image files. we have the wrong conception that we have to change everything every 5 years and that doesn't count only for computers.

but as i said in a post for the drm article the only completely reliable way to store data is in a form that doesn't need auxiliary devices to work: books for documents and pictures for images; for music, video and other types of storage we could create some devices that can give you the information by themselves, like an mp3 player that can last 1000years without failing (or a phonograph or cassette recorder), a manual powered video projector :P, etc

Re:Only half the problem (4, Funny)

oGMo (379) | more than 6 years ago | (#23168044)

There's probably room for a lucrative business based around this-- figuring out the most elegant way to archive and retain meaningful access to data under various computing/disaster scenarios. Hey, I do consulting. :)

Find a chisel. [wikipedia.org]

Re:Only half the problem (1)

erlehmann (1045500) | more than 6 years ago | (#23168066)

The other side of this is, for anything more advanced than text-- given that you can get at the data, what do you open it with? File types die over time and it's basically impossible to find programs to open certain files nowadays, much less such programs that will run on a modern OS.
Simple: You use only formats that are openly specified and free software. HTML and everything XML-based actually is text, while format descriptions and decoders for Theora, Vorbis etc. will be around for a long time, probably due to the decoders being free software.

Try harder (4, Insightful)

daBass (56811) | more than 6 years ago | (#23168174)

(Ever tried to get data off an obsolete tape backup
There are loads of people that can make this work. The most important thing is having the specs of what is on it, how it was recorded. (even just a few hints and some knowledge of how computer systems in that era might have recorded data is enough) That the machine used is no longer functioning and had an interface that doesn't work with your USB-only modern PC anyway is of no relevance.

Given the media, specifications and some time and money, a trio of engineering, electronics and CS students will make a machine that will read any old tape, punchcard, early HDD, etc. A CD is laughably simple technology, an engineer 100 years from now will build a player (in a way that may not look anything like our current players) in no time at all.

Today's technology is even more well documented and certainly not beyond the capabilities of future generations to make readers for.

If you find an old tape and want to do it in an afternoon, you are out of luck. If you are an historian that really, really wants to get to the data, it is not all that hard.

Try harder == pay more (1)

Colin Smith (2679) | more than 6 years ago | (#23168780)

What's the data worth to you? That is the question you have to ask with archival.

an engineer 100 years from now will build a player (in a way that may not look anything like our current players) in no time at all.
What on earth makes you think there will still be electricity in 100 years? Civilizations don't expand exponentially for ever. They hit a limit and in the following economic collapse there is all sorts of chaos. Ultimately the only assumption you can make for storing information for very very long times is that a human being be able to see and touch it.

At the moment, the very, very best method of long term archival we have involves the sacrifice of calves, sheep, or other animal... The UK for instance still prints all of it's acts of parliament on vellum.

http://news.bbc.co.uk/2/hi/uk_news/politics/502342.stm [bbc.co.uk]
 

Re:Only half the problem (1)

TheLeopardsAreComing (1206632) | more than 6 years ago | (#23168204)

There's probably room for a lucrative business based around this--
Recording temperature/ weather related patterns Sports scores to every game Schematics on certain technology etc. etc. etc. The possibilities are nearly endless for it's uses. Pretty much everything we need to keep the future heading in the right direction... The only problem is: Could relying to heavily on this technology be leading us to the next Library of Alexandra scale information loss?

Re:Only half the problem (2, Funny)

Zencyde (850968) | more than 6 years ago | (#23168206)

Only problem I can see with it is generation loss. Copy something over and you're missing a couple of bits. Okay, not too much harm done. Copy it again and you're missing even more. Okay.. a bit of a hit we can keep going. By the time you've copied it twenty times, it sounds like this: http://www.youtube.com/watch?v=Yu_moia-oVI [youtube.com]

Re:Only half the problem (1)

prattp (896401) | more than 6 years ago | (#23168400)

Moving from analog to digital technology solved this. And you can include a checksum with your file to be quite confident that your current copy is identical to the original.

Re:Only half the problem (1)

PhireN (916388) | more than 6 years ago | (#23168652)

I see what you did there.

Emulation may work better (2, Insightful)

prattp (896401) | more than 6 years ago | (#23168274)

I agree that virtual machines are a solution to file formats becoming obsolete, but I think that emulation may be more appropriate than virtualization for this purpose. VMware can only be used on x86 computers, and even on x86 computers future processors may have subtle differences that could affect old virtual machines. An emulation of an entire computer, including the processor, can be ported to any computer, and have exactly identical behavior.

Also, it may not be necessary to layer virtual machines inside each other, if you have an emulator that that is easy to port new machines, such as by being open source and relatively simple. That is a large part of the motivation for the Macintosh Plus emulator I maintain.

Web. (1)

SanityInAnarchy (655584) | more than 6 years ago | (#23168280)

Some elements of this problem could be solved by having backup servers use wireless and filesharing protocols that might stand the test of time- e.g., 802.11n and SAMBA. No need to just pick one 'most likely to be future-proof' combination, either...

I'm going to, anyway: The Web. Straight up HTTP, with HTML documentation. Fall back to plain-text if you're extra-paranoid, but if you don't do any styling, straight HTML is very future-proof and backwards-compatible. If you do anything on top of that (Dav, etc), document it as completely as you can in that documentation.

I don't really see how wireless is any more likely to be accessible than a plug -- I would argue less so, as wireless standards can change, but no one can legally prevent you from having functioning Ethernet (or Token Ring, etc). If there's a concern of making this thing outlast Ethernet and (say) ipv4, include anything you're paranoid about losing, and put a durable physical interface on the thing so your great-grandkids can read the minimum documentation they need to rig an interface, then read the rest of it comfortably in whatever a web browser looks like by 2095, as they try to code an interface to it.

File types die over time and it's basically impossible to find programs to open certain files nowadays, much less such programs that will run on a modern OS. I think the answer to this has to be virtualization.

Or simply include all the programs needed in the original hardware, along with full specs of said hardware, so that virtualization can be built as-needed.

I'm not entirely sure which one is easier. I suspect that an active system is most reliable, as long as you have the money to pay people to look after it. I'd imagine that if you can't guarantee that constant flow of cash, the sanest thing to do is build the most durable physical system possible -- and then build two more -- and lock them away in a vault somewhere, so that a few thousand years from now, archaeologists can reverse-engineer whatever you had. I think that'd be a lot better than hoping there isn't a political upheaval in a few hundred years that cuts off funding, and making the job even harder for those inevitable archaeologists -- now they not only have to understand the original machine, they also have to understand several layers of virtualization.

At the same time, I really, really wouldn't want to try to maintain anything in which I wasn't allowed to upgrade.

A focus on preservation and restoration (0)

Anonymous Coward | more than 6 years ago | (#23168382)

Part of the long-term archival of data has to be a focus on restoration activities. Whether this is the refreshing of old hardware with new storage technologies, or reworking into new document formats while still retaining original works for the future use of scholars. It can even involve translation or modernisation of texts and images over time.
More and more our digital data will need to be viewed through the same lens as we apply to the preservation and restoration of other kinds of works, its unique properties exploited and its unique challenges dealt with. Our most important digital assets will need to be preserved by skilled professionals in library or museum -style professions.

I have talked about this issue before [soundadvice.id.au] .

Re:Only half the problem (1)

inode_buddha (576844) | more than 6 years ago | (#23168450)

How bad do you need that pr0n? Seriously tho, I've heard stories that (for example) General Motors can pull your medical data from 30 years ago within a matter of days, off of tape. Imagine a room full of tapes and a very old machine still on the LAN -- got family members that work there. Once in a while they move it onto newer media.

Re:Only half the problem (1)

RKBA (622932) | more than 6 years ago | (#23168648)

Ever tried to get data off an obsolete tape backup?
No problem. I just slap it into my tape drive. [wikimedia.org]

Why stop at the software level? (1)

HetMes (1074585) | more than 6 years ago | (#23168768)

I agree with you on active storage being the future of longterm storage. Eventually it will get automated for data that has already been archived, and then the only point is to get your current data into archive. Look at how small PDA's can be. Just store any hardware device that can interface with the datastorage. Make sure its interfaces are completely documented and readable from the device itself. Continuing from that, why not just print the interface specifications of the storage device on high-quality paper an store them with the storage unit? I know, current specs are quite long, but if you design for easy storage and long-term retrieval, this shouldn't be a real problem.

Tried and true method (0)

Anonymous Coward | more than 6 years ago | (#23167902)

Cave paintings work too.

Re:Tried and true method (2, Funny)

megaditto (982598) | more than 6 years ago | (#23167976)

What if they did have petabyte-level holograms and optical storage 12,000 years ago but the whole lot got eaten by a fungus because of the organic die or something? And all that survived were those fingerpaints up in a French cave originally made by a Down syndrome kid...

Sometimes old tech is best (5, Insightful)

erroneus (253617) | more than 6 years ago | (#23167906)

No, not punch cards... but close!

Stone and chisel. That's the way to store data for 1,000 years. The reason why I say this is simple. The more "religious" the world's populations become, the closer to the dark ages we become. (The reverse is true as well as history illustrates.) I expect there will be a second "dark ages" at which point all other technologies will simply not be available.

Re:Sometimes old tech is best (1)

Kinky Bass Junk (880011) | more than 6 years ago | (#23168036)

I take it weather will not be available in this future of yours?

Re:Sometimes old tech is best (0)

TarZ (954747) | more than 6 years ago | (#23168074)

Stone and chisel.
... and parchment.

# pergamenum in Latin.

Re:Sometimes old tech is best (1)

Ixitar (153040) | more than 6 years ago | (#23168508)

Stone and chisel.
... and parchment.
Ewe horrible person.

Re:Sometimes old tech is best (4, Interesting)

martin-boundary (547041) | more than 6 years ago | (#23168078)

Why not microscopic etching [zyvex.com] . One advantage over the stone and chisel approach is that you can carry the mountain in your pocket until the next civilization figures out how to read it...

Uh, what? (1, Insightful)

ChePibe (882378) | more than 6 years ago | (#23168162)

The more "religious" the world's populations become, the closer to the dark ages we become. (The reverse is true as well as history illustrates.)

I realize that taking swipes at religion at /. is simply common fare and is an easy way to boost karma, but seriously, what? Where is this link between religion - one would assume all religion, as the OP discuss the population of the entire world - and this surge to the dark ages?

From the demographic viewpoint, a simple look at the high rate of belief in deity/practice of religion and the United States - the world economic leader, and still, in spite of some losses in this area, the center of innovation in all (well, at least most) things technological - would seem to indicate that the causal link between a belief in religion and a return to the "dark ages" is tenuous at best. For fun, compare the rate of technological advance in the U.S. with that of the devoutly non-religious Soviet Russia or Communist China throughout the cold war.

Then, one could look at individuals - Mendel, Newton, a wide assortment of Muslim mathematicians and astronomers, etc. Even a look at more mundane topics, such as engineers and inventors shows a broad array of other religious folks as well. As a Mormon, the first two that come to mind are Browning, a perhaps unrivaled genius to this day in the design of firearms, and Farnsworth - largely responsible for the electronic television.

Now, I'll be the first to concede the point that several religious groups have shown less technological advance over time, Wahabi Muslims in particular come to mind, but so do numerous others. Some groups have eschewed technology altogether, such as the Amish, but these are exceptional cases. But to argue that the act of being religious at all is somehow tied to a magical turn to the dark ages is absurd, and to argue that a lack of religion has always led to some drive away from the dark ages is no better.

Re:Uh, what? (2, Insightful)

Hal_Porter (817932) | more than 6 years ago | (#23168358)

For fun, compare the rate of technological advance in the U.S. with that of the devoutly non-religious Soviet Russia or Communist China throughout the cold war.
I think you could make an argument that Russia and China were theocracies for much of the Stalinist period. For example I read that Mao apparently gave a speech which was interpreted as him saying that quarks were the fundamental constituent of matter. After that Chinese physicists were careful not to publish papers that might contradict the great man. In Russia Lysenkoism [wikipedia.org] was famously the officially supported theory of agriculture. And in in Nazi Germany relativity and quantum mechanics were denounced as "Jewish physics" and physicists studying them were fired, which in a rare instance of poetic justice was probably not very helpful to the German atom bomb project. I think the Nazis would have messed up science far more if they had stayed in power longer and created the sort of Dark Ages agricultural slave empire they obviously planned.

Obviously Chinese, German or Russian social scientists were under much more obvious pressure to publish ideologically orthodox papers during their respective theocracies than physicist or biologists. Regardless of whether Nazism as religions, they behaved like intolerant monotheisms socially. In fact they were probably far worse since they existed in an age where orthodoxy could be enforced, rather than mere orthopraxy. This by the way is what Orwell was worried about - the ability of 20th Century totalitarianism to get inside people's heads.

By contrast America has lots of religion, but more importantly it has lots of religions, possibly because the Constitutional prohibition on an established state church allows them to survive. In the China or Russia lots of believers in the official religion ended up being crushed by the State because they were on the wrong side of a doctrinal dispute.

So at the risk of stating the obvious I'd say that a theocracy leads to science being suppressed, not a large number of competing religions. Competition is good, and that something that atheists, Communists and Gaia worshippers should understand as well as the believers in older traditional religions.

Re:Uh, what? (1, Insightful)

Anonymous Coward | more than 6 years ago | (#23168708)

Competition is good, and that something that atheists, Communists and Gaia worshippers should understand as well as the believers in older traditional religions.
Something I never understood while living in the USA. If one believes in one god how come there are hundred different ways to believe in that god. All claim to follow the bible, yet most have their own "version" of the bible .... makes no sense. Ohh and let's not forget the part where Americans pick and choose which one of god's laws to abide by at the moment....

Love and piece my ass:
- "When you go to war against your enemies and you see a beautiful woman and find her desirable, you may take her. If she ceases to please you send her away." Deut. 21:10

- "They waged war as god had commanded them and killed every male. But they kept the women as captives and took their wealth as spoil. Moses was enraged. 'So you spared the women? Kill every woman who has had sexual intercourse and kill every little boy, but keep the virgin girls for yourself. Divide them up evenly.'" Num. 31:7, 14

- "I saw that the people were marrying foreigners. Their children were even learning foreign languages. I called down curses on them. I struck them and tore the hair out of their heads and made them swear by god, 'you will not marry foreigners.'" Neh. 13:23 "So I purged them of everything foreign. I drew up regulations defining everyone's duty. Remember me, oh god, for my happiness." Neh. 13:30

I am now reading the Qur'an which is just as hateful as it dictates that non believers have no rights and should be punished which is exactly what is happening right now.

Re:Uh, what? (0)

Anonymous Coward | more than 6 years ago | (#23168606)

Neil deGrasse Tyson on the negative, measurable influence the Muslim religion had on astronomy:
http://video.google.com/videoplay?docid=-102519600994873365#0h24m55s

(starting at 24m55s, though the whole video is good)

In addition to that, Newton and many other scientific pioneers are mentioned and how when they encountered something they couldn't figure out, they gave up and attributed it to "God".
Brilliant people hobbled because they assumed there were things that are impossible to know.

Re:Sometimes old tech is best (1)

biobogonics (513416) | more than 6 years ago | (#23168268)

Stone and chisel. That's the way to store data for 1,000 years. The reason why I say this is simple. The more "religious" the world's populations become, the closer to the dark ages we become.

You've found Leibowitz's grocery list.

Re:Sometimes old tech is best (4, Informative)

evanbd (210358) | more than 6 years ago | (#23168438)

You could, of course, update the technology a bit: Rosetta Project [rosettaproject.org] . High density, readable with a high quality microscope, and partially readable with the naked eye -- the spiral of shrinking text should make the usage instructions obvious: "get a magnifying glass, there's more here."

From TFA, quite sick, really. (2, Informative)

DaedalusHKX (660194) | more than 6 years ago | (#23167922)

From TFA:

Santa Cruz (CA) - Have you ever thought how vulnerable your data may be through the simple fact that you may be storing your entire digital life on a single hard drive? On single drive can hold tens of thousands of pictures, thousands of music files, videos, letters and countless other documents. One malfunctioning drive can wipe out your virtual life in a blink of an eye. A scary thought. On a greater scale, at least portions of the digital information describing our generation may be put at risk by current storage technologies. There are only a few decades of life in tape and disk storage these days, but a team of researchers claims to have come up with a power-efficient, scalable way to reliably store data with regular hard drives for an estimated (theoretical) 1400 years.

My "digital life"? Scary to lose it? Man.. these people never heard of backups, or having a real life, eh? Jeez, I can store my whole "digital life" on a 1 gig USB key, with room to spare.

I've lost my backups more times than I can count, my computers are toys, mostly for communication and play. Amazing how many people put their whole LIVES on a hard disk. Remarkable actually. What would I lose? About a dozen passwords and I'd need to reinstall and re-customize my system... OH WAIT... I backed up the important scripts and source code to a DVD.. TWO in fact. Bummer, guess I don't have to cry endless tears over the loss of my "digital life".

Re:From TFA, quite sick, really. (1)

Hal_Porter (817932) | more than 6 years ago | (#23168088)

It's pretty funny how the "technorati" that keep repeating pretentious Web 2.0 claptrap about "digital lives" seem to be totally ignorant of Usenet 1.0 concepts like sarcasm, irony and trolling and as well as technology and are therefore very vulnerable to being teased. Looks like the Eternal September [wikipedia.org] will continue to bring fresh meat to bitter old veterans like us.

Another slice of flame grilled newbie anyone?

Re:From TFA, quite sick, really. (1)

DaedalusHKX (660194) | more than 6 years ago | (#23168232)

Speaking of storage. I still have a server with a multi drive RAID, and several storage bays. Built it by hand, tinkered the case, etc. Cooling system moves as much air as the wall mounted AC that cools the computer room.

I had to download the entire Debian and Gentoo archive JUST so I could use some of the space and feel like I hadn't just wasted all that space. Gonna pull the Red Hat RPM ftp directory as soon as I decide to actually RUN fedora on something.

Maybe I should run some Web 2.0 app. I could always use the extra spam. After all, I'm a growing boy. ;-)

Wait.. did you say USENET? I haven't been on there since college. Its still around and active? No, I'm serious, it is??

Re:From TFA, quite sick, really. (2, Funny)

Anonymous Coward | more than 6 years ago | (#23168368)

Speaking of storage. I still have a server with a multi drive RAID, and several storage bays. ...

As opposed to what? Single-drive RAID?

Re:From TFA, quite sick, really. (1)

Creepy Crawler (680178) | more than 6 years ago | (#23168568)

I store a crapload of movies and music on mine.

1.2 TB movies/seasons/anime .. Some 0day, some legit :D Lotta good.

And its all fed to our "AV" computer at our tv. On demand music/movies/games (from psx,psx2,snes,n64,dreamcast emulators). Ubuntu's never seen soo good (running 8.10).

The best is when you put a dvd in the drive, it rips to the server as an iso for play anywhere on the network. Thats why im staying away from BD until I can code that level of automation. If I cant, its piratbay instead. Their content always plays...

Re:From TFA, quite sick, really. (1)

Ihmhi (1206036) | more than 6 years ago | (#23168170)

My "digital life"? Scary to lose it? Man.. these people never heard of backups, or having a real life, eh?

I guess you've never heard of Ghost In The Shell?

Re:From TFA, quite sick, really. (3, Funny)

DaedalusHKX (660194) | more than 6 years ago | (#23168202)

I did. Hence why I will not put my brain inside a Tachihoma tank :)

That being said, i'm also not a fan of jacking myself up on drugs so I can "hack" wandering vehicles. I'm thinking any weapon I may wield in such a world would have to be capable of A, using some sort of warp singularity to disrupt all technological defenses of the target, and B, use that same singularity to power down the defender.

Why killem when you can simply turn them off? If that hot animated chick can kill people by fucking with their computerized brains, I can also generate singularity charges with my ham radio set and obliterate enemy cyborgs :)

Dear God, I really am overdoing the sarcasm lately, aren't I?

Re:From TFA, quite sick, really. (1)

Loki P (1170771) | more than 6 years ago | (#23168212)

As a writer, it would be a real pain to rescan printed manuscripts (which might have some incidental markup or printing defects) and then try to find what was incorrectly OCR'd. Admittedly, a 1GB USB key can store most of my critical data, but in my experience they die far more often than hard disks do so they're not a reliable backup system. CD-ROM may be the best, together with paper printouts as a last resort, but that's not going to last 1000 years unless you were to use controlled environments. Vellum is good for 1000 years but isn't friendly to the calves. I'm in favour of some kind of miniature embossing on a hard surface (ideally something like manufactured diamond), with the actual data written in human-readable form in the middle of a plate with checksums around the outside to facilitate OCR and correct defects. Then a reader only needs a magnifying apparatus to know there is information there and start decoding it.

Re:From TFA, quite sick, really. (1)

DaedalusHKX (660194) | more than 6 years ago | (#23168262)

CD-ROM may be the best, together with paper printouts as a last resort, but that's not going to last 1000 years unless you were to use controlled environments.

Well, the upside to that remark is simple. "Neither will you." :)

That little issue aside, I'm fairly sure that I've yet to have a USB key die on me. But then again, I am fortunate I don't use Vista Ready Drive. I've hard of people killing their keys in 3 months of heavy gaming action. I call that OUCH. Its almost as expensive as me blowing 200 rounds of .45 at the range now and again.

Re:From TFA, quite sick, really. (1)

GordonCopestake (941689) | more than 6 years ago | (#23168580)

I don't think it needs to be human readable as long as the compression is a well understood algorithm as computers of the (far?) future are likely able to reconstruct the original data no matter what. Even encrypted data would probably be fair game to a multi-petaflop CPU. That being said, your idea of physically marking a surface to store the data is likely the best one. Magnetic storage is susceptible to damage without physical contact, and even the dyes used in optical storage rot over time.

Re:From TFA, quite sick, really. (1)

natslovR (530503) | more than 6 years ago | (#23168256)

Everything that's important to you may fit in only 1GB and that's great, but for other people it's not so easy.

For my daughters first birthday I burnt a collection of videos to DVD that i thought where worth while and sent it up to her grandparents to enjoy. This isn't HD or anything just 640x480 captured with my camera. That was the highlights of the last year i had to narrow down for them to get it in to just 4GB.

Then there's the photos.

It'd be a shame to lose all that, and as the article says, it's a problem to keep it forever.

My mum said my daughters snarly face looks like my younger brother when he was her age, so she got out the slide, scanned it in and emailed it to me. That's because the slide has survived for 25 years in relatively good condition.

They don't have slides of my snarly face at that age because cyclone tracey ripped the roof off their home and covered everything they owned in water and filth in Christmas '74.

How will I be able to preserve images and videos of my daughters childhood since it won't fit on a 1GB USB?

It may not be an issue for you, but for anyone that would like to preserve some of their memories it will be, it's got nothing to do with 'Real Life' vs nerdity.

Re:From TFA, quite sick, really. (1)

NewbieProgrammerMan (558327) | more than 6 years ago | (#23168304)

Bummer, guess I don't have to cry endless tears over the loss of my "digital life".
I know what you mean; I've lost backups (and stuff that I didn't back up), and it was really inconsequential. I really think if I hadn't lost it I'd never have looked at it again anyway. Keeping important stuff for work is one thing, but like you said, so far it will all fit on one USB key or some DVD's.

I suppose I'm an un-cool 21-st century luddite, since I don't feel the need to construct and preserve some massive digital emo-temple to myself.

Re:From TFA, quite sick, really. (1)

Cheapy (809643) | more than 6 years ago | (#23168436)

I was talking to a friend of mine in the computer labs in my uni's CS building today. We were talking about Windows 98, then Windows 3. This reminded my friend of something that happened. He's a part of the Tech Support mailing list that our uni has. Just recently, someone sent out an e-mail asking if anyone had the hardware and software to get some data off of some 8 inch floppies. Some medical group needed data on Vietnam vets, and had the floppies with the data on it.

Congratulations, you have 2 DVDs with backed up configs and passwords.
But just having the medium with the data on it won't help you much 20 years from now, if you can't access it. They may (probably will) rot. It seems to me that that's what this thing is about (who reads TFA anyway?): making the data readable in both the sense that it's there, and we can actually get to it.

Re:From TFA, quite sick, really. (1)

tsjaikdus (940791) | more than 6 years ago | (#23168464)

>> Jeez, I can store my whole "digital life" on a 1 gig USB key, with room to spare
.
Jeez, you must have a _very_ interesting life. I can store it all on a 5 1/4 inch floppy

Wow! (1, Funny)

Anonymous Coward | more than 6 years ago | (#23167942)

Almost as good as clay tablets or pyramids but easier to manage and with a higher data density. Cool!

Maybe /. needs something that lasts a bit longer.. (4, Funny)

Tmack (593755) | more than 6 years ago | (#23167944)

Since those "recent studies" links have already degraded into 404's. Maybe something like what was covered a few days ago? [slashdot.org]

tm

But what about... (3, Insightful)

bigredradio (631970) | more than 6 years ago | (#23167952)

Since there will be many holes shot into this theory, let me be one of the first to fire a shot. Electricity (as we know it) may not be around then. I am not predicting the dark ages, but who's to say that far in advance there is still a live socket.

Any storage device that relies on outside power cannot be guaranteed for 100 years, let alone 1400. I would have more faith in a stone tablet.

This is a fine example of "academic" research dollars at work.

Re:But what about... (3, Insightful)

fucket (1256188) | more than 6 years ago | (#23168042)

Good point. Also, in 1400 years there may no longer be any humans on earth to read the tablets you store so you might want to lock a human or two in the vault with your data.

Re:But what about... (1)

cgenman (325138) | more than 6 years ago | (#23168052)

This is a fine example of "academic" research dollars at work.

As opposed to the pragmatic issues of industry, this long-term thinking is actually is the sort of problem that academia is supposed to tackle, because it sometimes gives the major breakthroughs which revolutionize life. Like, for example, some sort of giant computer system which would survive a nuclear attack... in case you really need those trajectory tables calculated remotely during nuclear winter.

And it does have pragmatic uses. It is a large, redundant disk array which uses clever algorithms to only activate HDD's at worst 5% of the time. Not only that, but by their estimates a 10 PB backup system could be created for 5 grand with only 50 dollars per year in power and cooling costs.

At that rate, even if the system only lasts for 100 years, it is still by far the best long-term storage option we've currently got.

Re:But what about... (2, Insightful)

superdave80 (1226592) | more than 6 years ago | (#23168062)

"Electricity (as we know it) may not be around then."

I'm not sure how you expect electricity to 'change' in the future.

If a civilization can't generate electricity, then they wouldn't have the technical knowledge to even know what to do with digital data, so the whole point would be moot.

Re:But what about... (2, Insightful)

Ihmhi (1206036) | more than 6 years ago | (#23168184)

It's electricity, not Greek Fire. It's not some big mystery on how to generate it. Even if we're using microscopic black holes to generate power, it would not be hard to set up a windmill and some copper wire.

The bigger issue would be being able to actually read the data.

Re:But what about... (1)

pokerdad (1124121) | more than 6 years ago | (#23168560)

I'm not sure how you expect electricity to 'change' in the future.

Like every civilization in history, we think we know almost everything about the physical workings of the universe; there's just a few tiny holes that need to be plugged, then the tapestry is complete.

Even if we don't discover some better way to transfer power in the next 1000 years(if you can't grasp how much technology can change in that amount of time, just look a 1000 years the other way), don't you think we'll at least optimize our use of electricity? Eventually the connectors of today will be obsolete, the voltages of today will be unsupported and likely forgotten and there will be no practical way to power a data storage device from way back in the 21st century.

Wikipedia (1)

G3ckoG33k (647276) | more than 6 years ago | (#23167978)

This is SO scary! I had just been looking at Wikipedia looking up some obscure phenomenon, and went over to Slashdot. While the page is loading my thoughts drift and I think how important isn't Wikipedia, for now and the future. Someone should print it out and... What? A Slashdot article claims that someone will print the German edition. I manage to collect my thoughts and login, and, notice THIS article... I'm drifting in a black void by now... We wikipedians have come to bring you back home... Sorry for some digressions. Still, the digital age must be preserved. How? Continuously updating and using data! No more diskstorage of WordPerfecrt 1.17 files for ppc Mac 2.23 in a damp basement. Use archive proof paper? 10,000,000,000 trees a months? Oh, what isn't the price for being future proof?

Re:Wikipedia (1)

Tuoqui (1091447) | more than 6 years ago | (#23168110)

Future proof yourself by using plain .txt files.

Re:Wikipedia (1)

Neil Hodges (960909) | more than 6 years ago | (#23168376)

What about file encodings? Unicode won't last forever.

Born for this job (5, Funny)

Anonymous Coward | more than 6 years ago | (#23168004)

Did anyone else notice that the lead researcher's name is Mark Storer? How perfect is that?

What about filling it up? (1)

TubeSteak (669689) | more than 6 years ago | (#23168006)

Since TFA talks about 2 & 3 MB/sec throughput rates...
How long will this array take to fill up the first time around?

A 10 PB storage system could be built for about $4700 with an annual operational cost (power for running and cooling the system) of about $50.
Unless 10 PB (petabytes) means something other than what I think (10,000 terabytes), where did they get the $4700 number?
I even read their definition of static cost [usenix.org] (You have to go up a few paragraphs) and I still don't know.

Re:What about filling it up? (4, Informative)

Blkdeath (530393) | more than 6 years ago | (#23168040)

Unless 10 PB (petabytes) means something other than what I think (10,000 terabytes), where did they get the $4700 number? I even read their definition of static cost [usenix.org] (You have to go up a few paragraphs) and I still don't know.

Table 3: Comparison of system and operational costs for 10 PB of storage. All costs are in thousands of dollars and reflect common configurations. Operational costs were calculated assuming energy costs of $0.20/kWh (including cooling costs).

Does $4.7 million sound a bit more realistic?

Re:What about filling it up? (1)

Dahamma (304068) | more than 6 years ago | (#23168146)

Yeah, exactly. They are clearly off by an order of magnitude, unless they found a secret source of 1TB drives for $4 each....

It's based on distributed storage... (0)

mbessey (304651) | more than 6 years ago | (#23168198)

The idea is that you'd use unused capacity on existing machines. Their cost estimates are just for the additional equipment - it doesn't include the cost of the drives, since you'll have to buy them anyway...

Steganography and P2P (4, Funny)

Chairboy (88841) | more than 6 years ago | (#23168016)

One thing remains constant in thousands of years of recovered cave paintings, manuscripts, papyrus drawings, and more. And that constant... is pornography. It lasts, it's popular, and it's always in demand.

Clearly, the answer for long term data storage is to use steganographic techniques to encode your data into various types of creative skinpics. Pick famous folks, pretty folks, strange fetishes... the whole gamut. Pick things that people will keep. A hundred years later, all someone needs is the key phrases to search for.
"We need that Higgs Boson experiment data from 2012, how will we get it? The infocalypse has destroyed all of our cataloged data!"
"No problem, my great grandfather left a note in his journal telling his descendants to search for 'Britney spears enema' and use 'wet riffs' to decode the LHC data in whatever we use for files."
"President Spears? That's crazy!"

Voila!

soooo (1)

thermian (1267986) | more than 6 years ago | (#23168226)

I should write my important information on any available boobies? Is that what you're saying?

Re:Steganography and P2P (1)

laejoh (648921) | more than 6 years ago | (#23168248)

Message to all slashdotters: for a small fee I am willing to tattoo the source code listings of your programs onto female porn stars thus preserving your works for eternity!

Please subscribe to my mailing list if you're interested!

Re:Steganography and P2P (1)

alshithead (981606) | more than 6 years ago | (#23168312)

Popular papyrus porn from Persia?

Sounds like it could be in a movie... (1)

SKPhoton (683703) | more than 6 years ago | (#23168064)

"This hard drive will self-destruct in 1,400 years."

Constant data migration is the key. (1)

Kenja (541830) | more than 6 years ago | (#23168076)

Any long term data I keep gets moved to new mediums as they become available. There is no single medium that will last for the times described. The good news is that digital data has a very low corruption rate and a copy can be reverified for a guarantied duplicate every time its needed. I've moved from floppy drives, 44MB WORM, to ZIP, to CD, to DVD and am now using a 12 drive 1TB RAID-5 with AIT backups.

Rotate your media (2, Insightful)

profplump (309017) | more than 6 years ago | (#23168084)

Wouldn't it be a lot easier to simply keep the archive on a live system, and rotate it to new media from time to time as the old media dies and new storage systems become available? After all, if no one is looking after this system, what's to keep it from being forgotten in the basement of a long-abandoned building?

In addition to taking advantage of the falling cost of storage for a fixed-size data set -- making future replacement media purchases much cheaper than redundant media purchases today -- you also have the opportunity to re-process the data into new formats, so that you'll still be able to read it when you want it.

Well, I'm sure impressed... (1)

Ethan Allison (904983) | more than 6 years ago | (#23168114)

Wow. It's a really big RAID hooked up to a flash drive filled with metadata and deferred writes. Good job guys, you just invented a hybrid hard drive. Oh, wait...

Of course, this has a pretty cool real-world application in that hybridizing storage systems, that is adding a flash drive to defer disk writes and store metadata and such, appears to reduce MTTFs in any real-world archive, be it a 3-disk RAID for a home office or an incremental backup of the entire internet.

Re:Well, I'm sure impressed... (1)

mochan_s (536939) | more than 6 years ago | (#23168510)

Why does it reduce MTTF? Failure of disk or failure of the system as in the first data loss?

Because of spin-up and spin-downs? Then, maybe it's a bad write defer algorithm or not enough flash memory in there?

Insufficient Research (2, Interesting)

jr76 (1272780) | more than 6 years ago | (#23168126)

They completely ignored the fact that the chips and memory managing the system will likely have some degree of failure in the 1400 years the data will survive on their media architecture.

Look, I am into genealogy quite a bit and see this as a tremendous problem.

The only thing approaching a viable solution is the Rosetta Disk ( http://www.rosettaproject.org/ [rosettaproject.org] ) using etched nickel media (rock) in a human readable format, which you could theoretically create a binary cipher for a global archival format.

But, that would take a lot of foresight, which unfortunately us people don't have (yet).

However, seeing that as completely inaffordable for us mere mortals, that leaves me with PAPER, yes, paper, as the only trustworthy medium-term solution.

I do hope everyone here realizes that if we had some sort of cosmic EMP-like event traversing the globe, we'd lose 99% of data and be plunged into the dark ages, right? We couldn't even re-create all of the machines that surround us since virtually all designs are kept digitally now. Factories would just shut down and never be able to be brought back up and every history of our existence would be forgotten in a few generations.

Our civilization is sitting on a house of cards.

Re:Insufficient Research (2, Funny)

mochan_s (536939) | more than 6 years ago | (#23168490)

Then, if a super-termite or some sort of paper eating worm ravaged the world and ate all the paper in the world, then we'd be in the same situation.

Re:Insufficient Research (2, Interesting)

utnapistim (931738) | more than 6 years ago | (#23168664)

You don't need a supertermite :)

Just an idiot with a political agenda and authority on his hands:

The Nazis used to burn books if I'm not mistaking.

Also, if I remember correctly, there was some pasha or other in the ottoman empire who said that either the kuran is the only truth and then other books have no purpose, or the kuran is not the only truth, and then the fact that there are other truths must be hidden; thus, he burned the library.

It only takes a bunch of idiots.

Broken links in the summary (1)

a_nonamiss (743253) | more than 6 years ago | (#23168150)

I find it subtly ironic that the last two links in summary of the article about data loss are broken.

Lasers. (2, Interesting)

menace3society (768451) | more than 6 years ago | (#23168168)

Laser engraving, seriously. There's some project out there....
ah yes, here, [rosettaproject.org] that seeks to preserve all the languages of the world by laser-engraving them onto stainless steel plates. They've changed things up a bit, but the basic idea is the same: put it somewhere it won't get lost or corrupted, and if it's important, people will figure it out later. If it's not important, then it doesn't matter.

Very few things in the world are really worth keeping for even a lifetime. If your grandkids inherit all of your stuff, what will they save and keep, and what will they throw away? If you know what they will throw away, why not save them the trouble and toss it yourself?

We've gotten ourselves into this mindset where making backups of every piece of data you've ever owned ought to be saved, for no other reason than because it's easy and cheap. I think everyone should have a periodic storage meltdown to force them to reconsider what it is they really need to have.

Assuming I'm understanding this right... (0, Redundant)

grasshoppa (657393) | more than 6 years ago | (#23168176)

Let's look at some numbers
A 10 PB storage system could be built for about $4700 with an annual operational cost (power for running and cooling the system) of about $50.

Ya, 10 PETA BYTES for 4700 bucks? I don't think so. And an annual operating cost of 50 bucks, that includes power and cooling? Again, no. Now, let's focus on the administrative overhead of replacing disks and failed system. The larger the setup, the more administrative work there would be.

The rest of the idea has merit, but it almost seems to be that they are trying to compare apples to oranges with their comparison to tape. Tape's appeal is that it is long term storage that requires little maintenance. The same can't be said for this.

Re:Assuming I'm understanding this right... (0)

Anonymous Coward | more than 6 years ago | (#23168276)

The TG Daily folks misread a caption in the paper. The paper says that 10 PB will cost about $4700 thousand dollars, or $4.7 million. Annual cooling cost is also in thousands of dollars, so it's $50K. I agree that 10 PB for $4700 is off.

Oh, and if you don't think tape requires any maintenance, consider that some high-speed tape drives are typically rated for 100,000 exchanges, so they'll fail every year (on average), requiring a new $3000 drive every year. And that doesn't count checking the tapes to make sure they're OK.

Re:Assuming I'm understanding this right... (0)

Anonymous Coward | more than 6 years ago | (#23168278)

It is actually $4.7 Million...

Here is a link to the paper :

http://www.ssrc.ucsc.edu/Papers/storer-fast08.pdf

Alien archaeologists (1)

lusiphur69 (455824) | more than 6 years ago | (#23168178)

I hate to be a pessimist, what with me being a believer in the power of technology and all, but seriously: we have made no credible progress in technologies needed to push space exploration, minus a few notable exceptions. If we do not spread our eggs to many baskets, so to speak, and instead simply squat on Earth for the next 1,000+ years, it is highly probable (based on my own assumptions and observations only), given human nature, that we will wipe ourselves out or return to some kind of post-futurist stone age.

So this technology has one of two possible final uses: humans attempting to re-learn what was lost, or the equivalent of hieroglyphics for alien archaeologists investigating 'that funny little race of air-breathing bipeds on the third planet of the Sol system'.

On a more relevant note, I have trouble understanding how, even if we do reverse course and realize a future where humans have populated the cosmos, this technology will be useful in even a hundred years, let alone 1000. With the rate of technological gains in certain areas, this is almost doomed to be obsolete before it is ever used.

Inscriptions on stone (1)

Cannelloni (969195) | more than 6 years ago | (#23168190)

The only way to store data for more then 1,000 years is to inscrive it in on virtually indestructible materials, such as granite or basalt. These will last for thousands of years. The rune stones in Scandinavia were made in 800-1300 AD and they are still more or less unchanged. They also have pictures. The stelea (stone slabs) in Egypt are some 4,000 years BC and still intact. What we need is a laser printer that writes on stone surfaces...

Two methods for long-term reliable storage (2, Interesting)

Raul654 (453029) | more than 6 years ago | (#23168210)

There are two sure-fire proven techniques for storing data long term - using a reliable non-volatile storage medium (engraving in a non-oxygen reactive metal will do nicely) and making many redundant copies of them.

Electronic storage is by its very nature unreliable -- electromagnetic properties (like charge accumulation, ferromagnetic hysteresis, etc) are inherently volatile.

And even if you manage to solve the problem of transporting your data into the future, you're still faced with the problem of making sense of it. Electronic formats change (just ask the guy out in California who makes a *FORTUNE* charging law people to retrieve files from obsolete formats and/or media). In the physical realm, this is true as well - languages change and become very difficult to read. (If you don't believe me, try reading Beowulf in its original old-English form, circa 700 AD).

Hey guys... how about books? (1)

zibix (654122) | more than 6 years ago | (#23168214)

Seriously. Assuming that scanning methods continue apace with other technology, wouldn't a simple hardcopy in easily scanable format be the best longterm storage. Maybe we should be looking at more durable forms of paper with smaller print, more fire-resistant and technology that will scan this data fast. Like a scroll. Why does long term data storage HAVE to be electronic or digital in nature?

Is this based on ZFS? (1)

complete loony (663508) | more than 6 years ago | (#23168236)

... hash tree-like structures ... staggered rebuild ... large redundancy stripes ... "scrub" ...
Is this based on ZFS?

Why have physical storage at all? (3, Interesting)

pclminion (145572) | more than 6 years ago | (#23168252)

And I mean it literally -- why have any physical storage at all? Why not just bounce chunks of data around forever on the Internet? Presumably the 'net is going to be here for a long, long time. Imagine a mass P2P network where the data being traded is just encrypted chunks of the data of other users. It needn't ever get written to a mass storage device at all -- just received from one peer and immediately sent to others.

A protocol could be developed to allow one peer to request, or steer, the network to locate and deliver requested blocks on demand. This might be a high-cost operation, akin to bringing data in from backup tape. Or, a client could just wait for the right chunk of data to recirculate to its position in the network. But storing data is easy -- just encrypt it, format it a certain way, and inject it into the network.

A natural model for the topology of such a network, and the protocol itself, is the circulatory system. Here, cells move in a fluid, generally in one direction, but through a complex network of vessels, and in a circulatory manner. The immune system might provide inspiration for directed movement of data chunks. (See? The Internet really is just a series of tubes.)

Over time, the infrastructure of the Internet, the P2P clients, and the exchange protocol itself could evolve, as long as enough redundant chunks are allowed to constantly recirculate. Specialized clients could cache data to "long term" storage for periods of a few days or weeks, in case of large, random outages, but permanent data storage would never rely on any specific technology at all -- even TCP/IP itself. It's all just this mass of recirculating encrypted chunks of data, like cells in the blood stream.

Sloppy reporting (1)

baboonlogic (989195) | more than 6 years ago | (#23168264)

From TFA

A 10 PB storage system could be built for about $4700 with an annual operational cost (power for running and cooling the system) of about $50.

Wtf? That is 0.044 cents/GB. That's impossible! No one can do it that cheap. Sloppy reporting again I guess... Perhaps they meant 10 TB.

Good (1)

Post-O-Matron (1273882) | more than 6 years ago | (#23168288)

Something to leave for the grandchildren.

Inphase failed. Crystal holography (1)

zymano (581466) | more than 6 years ago | (#23168340)

3d crystal holography like in startrek would be cool.

With no moving disc of course.

"perfect holographic storage could store 4 gigabits per cubic millimetre"

http://en.wikipedia.org/wiki/Holographic_memory [wikipedia.org]

Porn for the Ages (0)

Anonymous Coward | more than 6 years ago | (#23168344)

Just think...our distant descendants may one day be able to twist one off to Nina Hartley.

Better Solution. (1)

spydabyte (1032538) | more than 6 years ago | (#23168502)

My Solution: Google [google.com] . No really, find a cheap service (or two) that are likely to stay in business, keep updated, and offer online access. What else do you need? Sure it's not as secure as an offline system, but do the majority of people care about elite crackers seeing their family photos?

Isn't this a plan (1)

someone1234 (830754) | more than 6 years ago | (#23168506)

Isn't this a plan to find a place for obsoleted technology?
For all i know, regular HDD is going out of season soon, replaced by chips and memory crystals.
Why would anyone want to use HDD in the future?

don't use OOXML... n/t (1)

advocate_one (662832) | more than 6 years ago | (#23168600)

n/t

Idiotic (4, Insightful)

Eivind (15695) | more than 6 years ago | (#23168602)

This is completely idiotic.

First, it ignores physics. MTBF can't be used in reverse. Yes, it is possible that the MTBF on a newish disc is 300K hours or more, put differently, if you've got 1000 such discs running, then every 300 hours, about every 2 weeks, one will die.

This does however:

  • NOT imply that a average disc will last for 300K hours of operation, i.e. 47 years.
  • NOT imply that a disc that is idle 90% of the time will last for 470 years.
  • NOT imply that a disc that is idle 95% of the time will last for a millenium.


It would offcourse if degradation in idle state was -ZERO-. If aging made -ZERO- difference and if the MTBF-rates quoted are realistic AND constant over centuries (i.e. older discs DONT start to fail more often, not even if they're centuries old)

In short: bullshit. It's overwhelmingly likely that not a single disc out of 1000 will remain functional after a millenium, even if it is powered down 97% of the time. At which point no amount of redundancy, distributed or not, will help.

Also, the exersize is pointless. As long as storage-capacities keep growing exponentially, nearly the entire cost of storing a set of data is in the first few years. If you've paid what it costs to safeguard data for a decade, you've already paid 95% or thereabouts of what it costs to store it forever.

So, storing something safely for a very long time is actually a easy task, all you need to do is:

  • Create multiple copies at geographically distinct sites.
  • Regularily transfer the copies to newer larger media


Yeah, this -does- mean that data that nobody cares about will die. Tough luck.

For example, if you -currently- have a petabyte you want stored, you could buy 3 petabyte enterprise storage-servers, at a cost of perhaps $3million. You host these at three separate companies, say one in europe, one in japan, one in usa. For this you may pay $300.000/year. Total cost for first 5 years: $4.5 million

After 5 years you buy 3 new entry-level storage-servers. Storage/dollar has doubled ever 18 months, or a factor of 12 over 5 years. The servers now cost let's say $300K, and they're 4U-units rather than complete racks now, so hosting-costs is down to $50.000/year.
Total cost for years 5-10: $550.000

After 10 years you buy 3 new 1U "small office" servers. They cost $21K in total. Hosting is $10K/year. Total cost for years 10-15: $71K.

After 15 years you sign up for the needed amount of space on 3 separate servers and pay $3K/year, or $15K for the period.

After 20 years you put the data on 3 thumbdrives and store them however one can cheaply store a thumbdrive, total cost perhaps $1000
Or you sign up with 3 separate el-cheapo hosting-providers and pay $300/year.

After 25, you send the data as an attachment to your choise of 3 free email-providers, they all come with atleast 500PB free storage anyway, it's not as if you'll notice the extra 1PB attachment.

More likely though, you've got much MORE data to take care of in the future, so you're still paying $1million/year. Only now that buys you a storage-solution where the old 1PB-archive is a completely trivial file, taking up a so minute fraction of the array that it's not even noticeable and the incremental cost is essentially zero.

Reinventing Honeycomb (2, Informative)

zdzichu (100333) | more than 6 years ago | (#23168616)

So, they are proposing Sun StorageTek 5800 [sun.com] (codenamed Honeycomb) as their research?

Compare article with this whitepaper [sun.com] , especially Figure 13 on page 28. Networked nodes with 4 disks each, grouped in cells of 16 + 1 management node. Each object is stured redundantly on disks of different storage nodes. Everything self-contained, accessible by nice API. Oh, and the software is Open Source.

missing the point (3, Insightful)

nguy (1207026) | more than 6 years ago | (#23168644)

It's easy to build distributed, reliable storage that theoretically lasts thousands of years if you assume that you can just keep going down to the corner computer store and buy replacement parts that more or less work like today's parts, that operating systems keep doing what they have always been doing, and that networks keep working the way they always have. But those are bad assumptions.

Modern form of punch card is the solution (1, Interesting)

LostMyBeaver (1226054) | more than 6 years ago | (#23168666)

I'm sure that anyone who recalls the punch card first hand will cringe just at its' mention. However if you were to picture a DVD or CD as being a circular punch card, you wouldn't be wrong.

The biggest problem with archival solutions is that to create a method which is small enough to store in a few boxes, the technology will typically be required to operate on a miniscule scale. This increases the probability that the media won't be properly identified as an archival media a thousand years from now.

While it's possible we'll have simple flatbed scanner like devices in the distant future capable of simply scanning a DVD or BluRay disc and then simply applying an image recognition program to read the data, it's more likely these forms of media will not be sufficient.

Recordable CD and DVD suffer an obvious flaw which is that the method of recording requires the disc itself to be a degradable form of media. Something that can be burned through at a rate of billions of times per minute with a single laser with high precision. Obviously, even using a relatively high power laser, the material must be thin enough to support this. Therefore, it stands to reason that even in a perfect DVD recordable media, time, sunlight, cosmic radiation, and new-age music is bound to degrade the disc past error correction friendly levels in relatively short times (likely years, maybe decades, certainly centuries).

In the case of circular media of high density, it requires precisely timed devices to read a disc. The disc spins and the bits are positioned in locations that are identified by precise timing. The device to read this type of media is very complex. The bandwidth of the laser required to read the disc is also precise. An archival grade media should not have such technical difficulties, otherwise, the effort required to read the media a century from now if no device is left in existance is substantial. Especially since the plans for the reader is likely to be stored on one of these discs.

Punch cards are wonderful since they are linear, rectangular and can be read relatively easy using somewhat primative equipment. Of course, I'm not suggesting using holes large enough to push a pen tip through, instead I'm suggesting a relatively high density punch card where at least a gigabyte can be stored on the surface of a drink coaster sized card.

The card can be made of many different types of materials, in fact, it could be paper or uranium. I would suggest personally a dense metal with a half life relative to the desired duration the media should live.

On one side of each card the plans to build a reader should be stored as human readable images, although on a microscopic scale (similar to microfilm or microfiche) since anyone likely to be able to build a reader will of course have a microscope. The reader presented should be the simplest form and should clarify the encoding used for characters. The device should be able to be built using parts that have been historically available. Meaning that if they existed 50 years ago, and they're still common today, they'll be common 50-100 years from now at least.

The data on the card should be stored through a process of laster engraving or etching for example. Punching directly through the card is ideal, but could interfere with the location of the design. Of course, if you have a box of 1000 cards, you only need the design once.

The process of reading and writing these cards does not need to be incredibly fast. After all, the purpose is for archival. If a single inexpensive device can write one gigabyte an hour, that means with 20 devices, 20 gigabytes can be archived an hour. Besides, unless you're backing up film masters which are typically 3-5 terabytes each or audio masters which can usually be 4 gigabytes, all other files (pictures included) can be backed up in little time.

Every time I read about long term archival, this solution always seems obvious to me. I highly doubt that there's any new patents to be had on it. Pretty sure that pool is not only tapped, but most of the patents should have even expired by now. But if anyone actually decides to make this device from my description or something similar to it, I would love a chance to try one.

Load More Comments
Slashdot Login

Need an Account?

Forgot your password?