Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Recoverable File Archiving with Free Software?

Cliff posted more than 10 years ago | from the redundancy-for-your-files dept.

Data Storage 80

Viqsi asks: "Back in my Win32 days, I was a very frequent user of RAR archives. I've had them get hit by partial hardware failures and still be recoverable, so I've always liked them, but they're completely non-Free, and the mini-RMS in my brain tells me this could be a problem for long-term archival. The closest free equivalent I can find is .tar.bz2, and while bzip2 has some recovery ability, tar is (as far as I have ever been able to tell) incapable of recovering anything past the damaged point, which is unacceptable for my purposes. I've recently had to pick up a copy of RAR for Linux to dig into one of those old archives, so this question's come back up for me again, and I still haven't found anything. Does anyone know of a file archive type that can recover from this kind of damage?"

cancel ×

80 comments

Sorry! There are no comments related to the filter you selected.

where have you been? (3, Informative)

Anonymous Coward | more than 10 years ago | (#8382938)

ever heard of parity archives?

Re:where have you been? (5, Informative)

jason.stover (602933) | more than 10 years ago | (#8383045)

Here's the parchive sourceforge site [sourceforge.net] .. Links to PAR2 utils, spec, etc...

wow man (3, Funny)

Anonymous Coward | more than 10 years ago | (#8382953)

the mini-RMS in my brain

You really ought to have that looked at..

Re:wow man (4, Funny)

Viqsi (534904) | more than 10 years ago | (#8383557)

Y'know, I would've done that a long time ago, but my health care provider doesn't cover ideologuectomies. They claim that it doesn't threaten your physical life, just your social one. The bastards.

:D

Re:wow man (0)

RedLeg (22564) | more than 10 years ago | (#8393605)

...my health care provider doesn't cover ideologuectomies. They claim that it doesn't threaten your physical life, just your social one...


Ah.... I've met RMS a number of times, and having once made the mistake of standing downwind, am familiar with this problem. Try Soap and Water, augmented with a long-handled stiff brush. Pardon the Pun, but Lather, Rinse, Repeat. Works wonders for your complexion, with immediate secondary positive effects on the `ol social life...

Re:wow man (1)

GoRK (10018) | more than 10 years ago | (#8385134)

I suggest vrms [debian.org] as a healthier replacement.

Are you sure tar is unacceptable? (1, Insightful)

nado (101599) | more than 10 years ago | (#8383103)

Are you sure it's unacceptable that tar archives are breakable? The way I see it, you'll tar your files then bzip them and finally put them on a backup server/CD/DVD. The bzip layer will provide the auto-repairing features, I don't see how it could break between having the tar and bzipping it. Is this for a normal environment? If your harddrive breaks during or after creating the tar, then the bzip would fail, no? Please tell us more about your situation if not.

Re:Are you sure tar is unacceptable? (5, Informative)

wiswaud (22478) | more than 10 years ago | (#8383316)

if you make a big tar then bzip2-it, then store the file on a CD.
then 2 years later you want the data back.
there's a read-error at some point within the .tar.bz2, and it gives you some garbage data.
bunzip2 will actually be able to recover all other 900kB chunks of the original tar file, except for this missing chunk or part of it.
Tar will just choke at that point and you lost everything past the read error. bunzip2 was able to recover the data past the error, but tar can't use the data.
It's quite frustrating.

Re:Are you sure tar is unacceptable? (1)

sql*kitten (1359) | more than 10 years ago | (#8384216)

Tar will just choke at that point and you lost everything past the read error. bunzip2 was able to recover the data past the error, but tar can't use the data.

I've been there with .tar.gz and now .cpio.bz2 is my archiving technique of choice, with a block size of 100k you get (slightly) less compression but (slightly) more resilience.

depends on the definition of lost (1)

leehwtsohg (618675) | more than 10 years ago | (#8384542)

I've had a backup of my hard-drive on another drive, in tar.gz form.
Ofcourse, when the big day came, and my hard drive broke, it turned out the other drive had bad sectors!
First, a comment: never ever ever ever use tar.gz to back up anything you'd like to have back.
You can recover stuff easily from tar past the break point - files in tar are basically concatenated together. So you miss the rest of the current file, but you can find the next header+file easily.
But gzip does not byte-align its data! That's, in my oppinion amazingly stupid. It saves a couple of bits per file, but makes recovering a real hassle.
I had to go through the file past a bad point bit by bit (literaly) to figure out if that is that next data block.

Re:depends on the definition of lost (1)

Trejkaz (615352) | more than 10 years ago | (#8394386)

So what's the option? Use RAR for archiving your data? I would copy files straight to a burn if only the CD filesystem didn't have a piss-weak depth limit. :-/

Re:depends on the definition of lost (1)

leehwtsohg (618675) | more than 10 years ago | (#8395167)

That is actually exactly what I do - full, uncompressed backup - takes 5-6 CDs in my case. Then I do incremental backups of the changed files till that goes over 1 CD (level 1 & 2). Redundancy comes from having each file on several backups...
But rar would be better if it was more widely distributed and free (speech) - then i'd save a copy of the decompressor and its source code on every CD...

Re:depends on the definition of lost (1)

Trejkaz (615352) | more than 10 years ago | (#8395701)

How do you get around the directory depth issue? I discovered that I can't add anything to an ISO beyond 4-5(?) levels of directories.

Re:depends on the definition of lost (1)

leehwtsohg (618675) | more than 10 years ago | (#8396170)

You are right, it seems that iso9660 can not handle more than a depth of 7 (not including the root). I never encountered it till now - my home dir must not be very deep - I encountered problems with long file names instead.
It seems that the rock-ridge extension deals with this by putting deeper directories into a dir called RR_MOVED, so I think if you use mkisofs with rock ridge all is fine (which could be another reason that I didn't encounter the depth limit)

Re:depends on the definition of lost (1)

Trejkaz (615352) | more than 10 years ago | (#8396686)

Aha! Okay. I just heard that Rock Ridge couldn't be read on some systems, but if that's the only problem then I'll use it regardless. I have way too many Java source packages in CVS which are making the depth pretty big. :-)

Try apio (4, Informative)

innosent (618233) | more than 10 years ago | (#8383125)

There used to be a cpio-like archiver called apio, that was designed for those types of situations. Of course, that might not be much help for non-unix systems (unless you plan on running in Cygwin), but I remember having great success with it for the old QIC tapes, which were in my experience the worst backup medium for important data ever (better to have no backup than think you have a good one, but have a dead tape)

Re:Try apio (4, Informative)

innosent (618233) | more than 10 years ago | (#8383347)

Sorry, I believe it was afio [freshmeat.net]

Re:Try apio (1, Funny)

Anonymous Coward | more than 10 years ago | (#8384163)

hey no fair, double mod points for your typo!

Re:Try apio (1)

discogravy (455376) | more than 10 years ago | (#8399960)

you need to read that .sig again.

Re:Try apio (1)

Wolfrider (856) | more than 10 years ago | (#8395275)

--I experimented with afio a couple of years ago, but the problem I ran into was twofold:

o Passing a list of files to afio, by default, is a PITA; I much prefer tar / zip / rar for their convenience here, you can pass a wildcard at the command line *or* send them a list.

o I couldn't find a way to do a *partial* restore of a subset of files.

--Anyone got some tips on this? I did write a set of scripts for handling volume changes and the like; if anyone's interested in them, email me.

Re:Try apio (1)

tzanger (1575) | more than 10 years ago | (#8384711)

Yup you do mean afio. Just like tar but it creates a separate compression record for each file, instead of the entire stream. I have had DDS tapes fail on me (well I screwed them up actually) and yes there were a bunch of unrecoverable files but at the next compression header things started streaming out of the drive again and I recovered quite a lot.

Par2 works great (5, Informative)

dozer (30790) | more than 10 years ago | (#8383158)

Store the recovery information outside the archive. Par2 [sf.net] works really well. You can configure how much redundancy you want (2% should be fine for occasional bit errors, 30% if you burn it to a CD that might get mangled, etc.). It's a work in progress, but it's already really useful.

Re:Par2 works great (4, Informative)

Stubtify (610318) | more than 10 years ago | (#8383823)

Allow me to second this. Par2 is everything the first PAR files were and more. No matter what has been wrong I've always been able to recover with a 10% parity set. (even this seems like a lot of overkill, except on USENET). Interestingly enough Par files have revolutionized USENET, I can't remember the last time I needed a fill.

good overview here: PAR2 files [slyck.com]

comparison between v1 and 2: here [quickpar.org.uk]

Re:Par2 works great (2, Funny)

Zapper (68283) | more than 10 years ago | (#8383988)

Just a pity that no sane amount of PAR files will compensate for my ISPs lame news feed. :-(

Re:Par2 works great (0, Offtopic)

Stubtify (610318) | more than 10 years ago | (#8384088)

I use Usenet Server [usenetserver.com] its like $13 a month for unlimited 3Mbit downloads, has great retention and I've done 50+gigs in a month and never heard a peep from them.

Look out for Ashcroft (1)

BoomerSooner (308737) | more than 10 years ago | (#8386057)

Your ISP might get pissed.

Re:Par2 works great (1)

eamonman (567383) | more than 10 years ago | (#8388997)

Par files have revolutionized USENET


Translation: pron moves now.

Re:Par2 works great (1)

Trejkaz (615352) | more than 10 years ago | (#8394399)

I can't remember the last time I retrieved a whole binary from USENET. It's like Australia must have gone to shit with USENET while the rest of the world didn't.

rar/par/par2 IP issues (1, Interesting)

Anonymous Coward | more than 10 years ago | (#8385196)

There is a patent on a recovery scheme by M. Rabin (I don't have the number handy). The patent covers "n+k" recovery schemes, in which n blocks of data are protected using k recovery blocks. The patent is quite old.

I wonder if rar, par and par2 infringe on this patent?

Re:rar/par/par2 IP issues (1)

Trejkaz (615352) | more than 10 years ago | (#8394605)

If they do, then wouldn't RAID-5 also infringe on this patent?

Re:Par2 works great (1)

Stavr0 (35032) | more than 10 years ago | (#8387259)

Excellent suggestion. I've always wondered why noone has integrated PAR2 into INFOZIP, BZIP2, etc...

Errors would be detected and recovered automatically while PAR2 files scanned for recovery info. Heck, why not stream recovery packets right into the compression stream -- just like solomon-reed and CDROMs.

Can Par2 Do Binary Merge3? (1)

Vagary (21383) | more than 10 years ago | (#8392517)

A quick perusal of the QuickPar website [quickpar.org.uk] suggests that at least some Par2 clients can restore based on two damaged files and incomplete recovery files:

At this point you can have QuickPar load additional PAR2 files (to provide more recovery blocks) or scan additional data files (to see if they contain some of the data from the original files).

In the past, however, I've been dealing with getting remote files over a noisy connection where the remote server wasn't so thoughtful to create Par files or even set up Rsync. What I've thought would be a nice is an application that can look for correspondances between three checksum-failed files to try and create one good one. I don't suppose Parchive can do that?

Yeah (3, Insightful)

photon317 (208409) | more than 10 years ago | (#8383222)


The format you're looking for is any format you like stored on reliable storage.

Why bother with all the intricacies of a pseudo-fault-tolerant data structure? Ultimately the best archive format for recovery will be one that just duplicates the whole archive twice over, doubling space requirements and improving immunity to lost sectors on drives. At which point one asks, "Why don't I just stick to simple files and archives, and use reliable storage that handles this crap for me, for all my data, automagically?" Storage of any sort just keeps getting cheaper and bigger. If you have any interest in the longevity of your data these days, there's almost no excuse for not using the data-mirroring built into virtually every OS these days and doubling your storage cost and read performance while preventing yourself from worrying about drive failure.

Re:Yeah (3, Insightful)

Viqsi (534904) | more than 10 years ago | (#8383541)

Why bother with all the intricacies of a pseudo-fault-tolerant data structure?

I'm on a laptop. I like my laptop. It's a very nice laptop. However, it doesn't exactly support those kind of hardware upgrades, and I am still ultimately on a bit of a budget.

I kind of put forth the question not only out of the hope that a Magical Solution To All My Archival Problems would Mystically Appear (puff of smoke optional but appreciated) but because I want to find something I also feel like I can unreservedly reccomend to non-ideological friends who are looking for, say, something slightly more reliable than ZIP files. I could've mentioned that in the article post, but it was already getting long. :)

Re:Yeah (0, Troll)

ISayWeOnlyToBePolite (721679) | more than 10 years ago | (#8384600)

Par archives is just a scam popularized by cluless usnet abusers. Think about it, if those files really could reconstruct a corrupt rar archive, why not post only the smaller par files (pigeon hole principle comp.compression faq)? What they do is just add redundansy. They contain the first and last parts of the rar archive, as those are the most likely to dissapear or not propagate. Get youself double copies and you'll be far better off, diffrent brands cd's/dvd's are easy, a backup hd is cheaper in the long run. (you could score a point by giving good reasons for the extra importance of first/last-part, but it's still just adding redundancy, and you can only get as much redundancy as you add or the technique would be better used as compression in the first place). Sorry if this sounds a bit harsh, but you're not the only one being tricked by those par files.

Re:Yeah (4, Insightful)

sasami (158671) | more than 10 years ago | (#8387929)

Par archives is just a scam popularized by cluless usnet abusers. Think about it, if those files really could reconstruct a corrupt rar archive, why not post only the smaller par files ... Get youself double copies and you'll be far better off

Ignore this post. It's either a troll or an idiot.

PAR files substitute for missing pieces. They don't regenerate the whole file by themselves. Go look up how RAID 5 parity [somacon.com] works. They're not called PAR files for nothing [utk.edu] .

Just because you don't understand how something works has no bearing on the fact that it does work. Except in certain performance-sensitive cases, doubling up is the least intelligent way of adding redundancy.

---
Dum de dum.

Re:Yeah (0)

Anonymous Coward | more than 10 years ago | (#8396116)

I'm on a laptop. I like my laptop. It's a very nice laptop. However, it doesn't exactly support those kind of hardware upgrades, and I am still ultimately on a bit of a budget.
You should backup/mirror to another machine over the network, or to an external HD (FW or USB2 enclosure). No archive format will save you if your laptop HD dies.

Re:Yeah (1, Insightful)

QuantumG (50515) | more than 10 years ago | (#8383568)

Ultimately the best archive format for recovery will be one that just duplicates the whole archive twice over, doubling space requirements and improving immunity to lost sectors on drives.

Obviously you know nothing about error correction, so STFU.

Re:Yeah (1)

portscan (140282) | more than 10 years ago | (#8385966)

What if the error produces another valid tar archive? How would the computer know which one was the correct one? You actually need *3* copies to reliably dectect a single error if you are doing it this stupid way.

There are WAY better (by better I mean take up less space and can detect more errors) methods of error detection (and correction), which have filled volumes of research publications and books, so I will not try to get into them here, but a (maybe not so) simple software trick will definitely save a lot of space over the method you suggest. Whether there is an archiving system (or a file system) that does this natively, I can't really say, but I would imagine that this "Ask Slashdot" is not the first time anyone has thought of it.

doesn't quit hold true (1)

SmallFurryCreature (593017) | more than 10 years ago | (#8388854)

Mirroring is dumb, no not as in dumb to use it, dumb as in the way it works.

Mirroring HD's only protect against fatal failures of a single HD. Motor stops spinning? Then the other HD takes over.

It does NOT protect against failures on the disc. Errors while writing or reading or other fun stuff.

For true backup you need the following.

  • An original wich you know to be correct. Obvious but I seen failures start right here.
  • A reasonably secure storage medium of wich you know the life expectancy.
  • A checksum or similar wich can be used to verify that a copy is the same as the original the smaller the checksum the less sure you are your copy is 100% like the original.
  • Ideally extra data wich be used to recover any errors in the copy (crc I think it is called). Without it you only know your data is good or bad. Not how bad.
  • TWO copies both indentical each with their own checksum and recovery data.
  • The checksum on a extreme long life storage medium (written on the container).

If you follow all that and make sure you use proper storage conditions and stop unauthorized access and make sure that you keep equipment around to read you storage medium then you can make backups with a reasonble certainty that it will actually work.

Of course this never happens in real life. Only goverments can be bothered. In real life people invest in expensive tapes and backup everything on fresh tapes. Then throw away the tapedrive during an upgrade.

cpio (5, Informative)

Kevin Burtch (13372) | more than 10 years ago | (#8383286)


True, tar cannot handle a single error... all files past that error are lost.

On the other hand, cpio (and clones) can handle missing/damaged data without losing the undamaged portions that follow (you only lose the archived file that contains the damage). It is the only common/free format I can think of (from the top of my head) that is capable of this.

Re:cpio (2, Informative)

Anonymous Coward | more than 10 years ago | (#8383912)

On the other hand, cpio (and clones) can handle missing/damaged data without losing the undamaged portions that follow (you only lose the archived file that contains the damage). It is the only common/free format I can think of (from the top of my head) that is capable of this.

ZIP also supports this (the command is "zip -F" with Info-ZIP, the standard zip/unzip program on Linux).

Mod parent up (0)

Anonymous Coward | more than 10 years ago | (#8385162)

ZIP also supports this (the command is "zip -F" with Info-ZIP, the standard zip/unzip program on Linux).

Zip is the way to go. Sure, it doesn't produce smaller archives, but it *is* a standard. It's also "Free", for the ideologues among us. Every system has it, including default installs of Windows. The originator of the article didn't say whether he needs this in a corporate environment. If he does, then zip is the failsafe archive choice. Who's to say his workplace won't convert to Windows in the future? If they do, he's assured to have an archive format that can be repaired.

Re:cpio (1)

Detritus (11846) | more than 10 years ago | (#8384865)

A particular implementation of tar may not handle errors well, but that isn't a defect of the file format. The program should be able to skip over damaged sections of the tape and recover the rest of the files.

Re:cpio (1)

Kevin Burtch (13372) | more than 10 years ago | (#8385037)


Nope, it's a design flaw - and a well known and documented one at that. See the O'reilly backup book (written by former co-workers) for more details (though it was well documented long before that book was written).

Just try to restore anything (past the error) using *any* version of tar, from a tar file (or tape) with an error in the middle. It will bomb out as soon as it hits the error.

Re:cpio (2, Informative)

Detritus (11846) | more than 10 years ago | (#8389529)

I know that I've recovered data from damaged tar archives in the past. I just ran some tests with intentionally damaged tar files, using GNU tar from FreeBSD 5.2.1. GNU tar successfully recovered the data from all of the damaged tar files. It just skips over the damaged bits and resynchronizes at the next valid file header.

Tar options (2, Insightful)

aster_ken (516808) | more than 10 years ago | (#8383435)

Wouldn't simply running tar with --ignore-failed-read achieve the desired results? It wouldn't simply stop once it hits an error. Instead, tar will proceed beyond the error and probably just write out junk data (if anything at all) for the corrupted part of the archive.

DISCLAIMER: I haven't tried this, and I'm not entirely sure this is what you want.

Re:Tar options (1)

Bastian (66383) | more than 10 years ago | (#8383472)

Woohoo! Manually parsing binary data to put together pieces of damaged files!

I think an important feature of the issue here would be that life is a lot easier if you never get the junk data in the first place.

RAR isn't completely non-free (3, Informative)

Kris_J (10111) | more than 10 years ago | (#8383508)

RAR compression is free for decompression [rarsoft.com] with source available, heaps of precompiled binaries for decompression on your OS of choice and it's included in a whole heap of popular free archive programs. Just burn the latest source on every CD you make and you should be fine.

Re:RAR isn't completely non-free (1)

Viqsi (534904) | more than 10 years ago | (#8383592)

Non-free, true. Non-Free, though, is a different story. The license for UnRAR's source is pretty restrictive (basically, "you can use this as intended anywhere, but you can't sell it or modify it to create RAR files"). So, unfortunately it doesn't totally work out.

The fact that that much exists does ease my mind about existing archives I've got (which is why I didn't mass-convert them ages ago). It's creating new, future archives that I'm worried about. :)

Re:RAR isn't completely non-free (3, Insightful)

Kris_J (10111) | more than 10 years ago | (#8383678)

But if you purchase it, as I have, you get a product you can use from now until forever, so long as your OS supports it, plus you can get the decompression source so that you (or someone else) can always write a decompressor for a future platform. Surely you don't need to worry about replacing it until both the following are true: None of the versions you've purchased run on your current platform AND no version compatible with your current platform is available (at a reasonable price). At that point you stop creating RAR archives and simply keep the decompressor around (porting and recompiling as necessary).

(Personally, I don't care about recovery records, I just keep two copies of everything, and I moved to 7-zip -- which can decompress RAR -- about six months ago.)

What if both copies develop a small fault? (1)

SmallFurryCreature (593017) | more than 10 years ago | (#8388935)

Both are useless. Of course if you know were the fault is and they are in different checksum parts then you can mesh them together to get a working file. But that is what recovery records are for.

No two files failing isn't likely to happen. We are however dealing here with disaster recovery. Disasters are always disastrous.

Of course error recovery won't work with a total failure like say a fire. Then your second copie is the better solution.

So two copies is a good idea. Error recovery is a good idea.

Two copies with error recovery is absolutly brilliant.

What do you mean do I own stock in media-storage companies. What a silly questiuon.

Re:What if both copies develop a small fault? (1)

Kris_J (10111) | more than 10 years ago | (#8392224)

Okay, well at work the backup system involves a two week rotation, off-site backups, monthly snapshots archived to DVD and two-hourly backups of the main database (during work hours), for something in the order of twenty copies at any one time -- each slightly different versions in case that someone makes a mistake and it takes a while to spot (which only happens about once every six months now, as opposed to once or twice a week the last place I worked). That's where I care about "disaster recovery".

At home, I just keep at least two copies of stuff at any one time. Big archives are tested upon creation and backups on media are turned over fast enough to tell if there's been any data loss. Since I returned a Zip Drive I haven't lost anything I've cared about, despite fairly dramic failure of two CDs (the surface just lifted right off).

Take a look at dvbackup/rsbep (3, Interesting)

jhoger (519683) | more than 10 years ago | (#8383567)

They are backing up data to a MiniDV camcorder adding forward error correction using a simple command line utility to allow holes in the tape the size of a pin without any data loss.

-- John.

Yes... (4, Funny)

caesar79 (579090) | more than 10 years ago | (#8383734)

its an amazing technology...only quite involved.
Basically you concatenate all the files together (cat should do), print it out on good 32lb paper, get a professor's signature and file it in a college lib...heard those things stick around for centuries

Re:Yes... (0)

Anonymous Coward | more than 10 years ago | (#8385083)

Basically you concatenate all the files together (cat should do), print it out on good 32lb paper, get a professor's signature and file it in a college lib...heard those things stick around for centuries

32 pounds? What'd you print out the Linux source code?

tar/gzip recovery toolkit (4, Informative)

wotevah (620758) | more than 10 years ago | (#8383776)

A quick google search [google.com] turns up the link shown at the end of this post, from which I quote:

The gzip Recovery Toolkit

The gzip Recovery Toolkit has a program - gzrecover - that attempts to skip over bad data in a gzip archive and a patch to GNU tar that enables that program to skip over bad data and extract whatever files might be there. This saved me from exactly the above situation. Hopefully it will help you as well.
[...]
Here's an example:

$ ls *.gz
my-corrupted-backup.tar.gz
$ gzrecover my-corrupted-backup.tar.gz
$ ls *.recovered
my-corrupted-backup.tar.recovered
$ tar --recover -xvf my-corrupted-backup.tar.recovered > /tmp/tar.log 2>&1 &
$ tail -f /tmp/tar.log

http://www.urbanophile.com/arenn/hacking/gzrt/gzrt .html

RAR Archives (4, Funny)

vasqzr (619165) | more than 10 years ago | (#8384946)


Back in my Win32 days, I was a very frequent user of RAR archives.

Bablefish translation: I was a huge warez kiddie.

On a related noted, were there any wide-spread, legitimate uses of .RAR? I only remember .ARJ and .ZIP

Re:RAR Archives (1)

Viqsi (534904) | more than 10 years ago | (#8385783)

Ha ha. No, I don't touch warez. The first time I almost did (with the Win95 private beta) I was caught by my father (I was raised by geeks, so it's taken a lot to be able to adjust to human society ;D ) and got a nice long lesson on that one. Pirating software is bad, 'mkay?

No, I got into RAR because a friend of mine (who was and still is into video game console emulation, especially music, which is where she discovered the format, I presume) used it to distribute her music compositions for a period, and I liked (still like) her music. I was curious as to what this "RAR" thing was, anyhow, got into it and found myself liking it, and so I ended up using it to archive old homework assignments... well, and IRC logs and Civ2 saved games. :D

Re:RAR Archives (2, Insightful)

jonadab (583620) | more than 10 years ago | (#8386031)

> were there any wide-spread, legitimate uses of .RAR?

RAR was heavily used in Germany, among the gamer community. A lot of Descent
players for example distributed their custom levels, missions, textures,
hogfile utilities, savegame editors, and whatnot in RAR format. It was
annoying; I had to go hunt down and download a RAR extractor just to install
some of the stuff.

The usual argument was that RAR was "better" than ZIP either because of the
compression rates or because of the partial recoverability or whatever. My
opinion on the matter has always been that for distributing stuff over the
internet, the most ubiquitous format is automatically the best, so ZIP is
better than RAR irrespective of technical issues, due to compatility concerns.
By the same reasoning, gzip is automatically better than bzip2, and no amount
of technical superiority makes a good enough reason to use bzip2 over gzip.
Frankly, for anything that's not inherently *nix-specific, ZIP is better than
gzip for the same reason. Not everyone agrees with me about this, obviously.

Re:RAR Archives (0)

Anonymous Coward | more than 10 years ago | (#8393576)

the most ubiquitous format is automatically the best, so ZIP is better than RAR irrespective of technical issues

So you are saying that Windows is better than Linux because more people use it??

Re:RAR Archives (1)

jonadab (583620) | more than 10 years ago | (#8394092)

> So you are saying that Windows is better than Linux because more people use it?

No, I was talking about an archive file format, and specifically about
selecting an archive file format for distributing stuff on the internet.
Using RAR or bzip2 for that is like sending richtext email or using
browser-specific markup in a web page.

Additionally, how many people *use* something is almost always irrelevant.
What I was talking about was how many people have software (any software)
that's capable of opening the format -- i.e., ubiquity.

Re:RAR Archives (1)

MikeCapone (693319) | more than 10 years ago | (#8394306)

My opinion on the matter has always been that for distributing stuff over the internet, the most ubiquitous format is automatically the best, so ZIP is better than RAR irrespective of technical issues, due to compatility concerns.

You do have a point, but on the other hand, everything has to start somewhere. Things have to evolve, we must move on to better things sometimes.

Just the fact that .rar is extremely popular in some circles is proof that it can work.

Otherwise we'll be using the .zip format in 2050.
br. Personally, I'm all for having compression softwares that can handle different formats (like 7-zip which handles a bunch, or most linux distros that at least come with 3-4 (never really counted the others, I always used gzip and bzip2).

Re:RAR Archives (1)

MrNerdHair (678335) | more than 10 years ago | (#8412091)

Um... anyone using .tar.gz (or .tgz) is either using (USUALLY, I'M GENERALIZING) *nix, which comes with a bz2 util too, or WinZip, which will decompress both. (It won't make either type, but it'll decompress them.) So, use the most technologically advanced format.

Just make duplicates. (1)

TheLink (130905) | more than 10 years ago | (#8385025)

Why bother with recoverability?

Total loss of the file seems more likely than bit flipping by themselves.

When your storage hardware/media starts flipping bits, it's probably going to die pretty soon.

And more often than not, your storage hardware/media just dies before you experience any bit flips.

You talk about your laptop computer and being on a budget. If you can't afford to make copies of your important files and store them elsewhere, then either your files aren't important, or successfully maintaining a personal computer system is beyond your financial means.

The odds of your laptop failing within the next 3 years are near 90%.

I suggest you get big HDDs (different brands) and backup everything on them. Then burn stuff you want to archive onto CD-Rs or DVD-Rs.

I've had more than a few HDDs fail in the past year (I still have my data tho). Ironically two of
the HDDs were used to back stuff up.

Re:Just make duplicates. (1)

Viqsi (534904) | more than 10 years ago | (#8386374)

Historical paranoia's the primary reason. I've had hard drives headcrash suddenly for no apparent reason, and typically I have to rush to get data off of them, and I'm usually lucky. And when I wasn't as lucky one time, RAR helped. I've never had a hard drive just suddenly Quit Working on me before; they've all been Slow, Painful Deaths.

Also, I almost never find the time in the week to do backups. I work for a nonprofit, so My Time Is Semiwillingly Not My Own. :) (We *do* do backups over there, though, so no work data's at risk.)

Oh, and the laptop's an IBM ThinkPad T23. Given the ThinkPad line's reputation (I have friends whose ThinkPads have lasted for half a decade or more), I think it's more likely to become obsolete first. :) (Of course, it's expensive enough that it's the reason why I'm on a budget at the moment... :) ) I do plan on getting one of those Second Hard Drive bay modules with another HD sometime and using it for backup (it's the method my parents use with their Omnibook 6000s, and it's worked quite well for them), but the money I could put towards that is currently earmarked for Other Things.

But this is getting into pretty in-depth detail, and I was hoping to start something at least resembling a general discussion rather than a I Need My Own Slashdot Support Team session. :)

That is an incorrect assumption (1)

atomic-penguin (100835) | more than 10 years ago | (#8385100)

Tar alone can recover past a damaged point it will 'read' past the erroneous data, and recover your data. I believe cpio exhibits the same behavior. It is when you compress the archive (with .gz or bz2) it may become unrecoverable. If you use tar alone however, you will always be able to recover some of the data in a damaged archive.

Re:That is an incorrect assumption (not always) (1)

Helen O'Boyle (324127) | more than 10 years ago | (#8395062)

Blockquoth the parent:
Tar alone can recover past a damaged point [...]
It is SOMETIMES an incorrect assumption. Not all versions of tar can do that; some tar utilities based on older code cannot.

Re:That is an incorrect assumption (not always) (1)

Wolfrider (856) | more than 10 years ago | (#8395320)

1. Upgrade your tar

2. Ditch the old *nix box and switch to Linux / OS/X / *BSD

3. Switch to GNU tar or Joerg Schilling's "star"

4. ???

5. PROFIT! ;-)

tarfix (3, Insightful)

morelife (213920) | more than 10 years ago | (#8385292)

tarfix

may help some of those archive issues.

But, the archive format is not going to save you. Use multiple media. You need more than one physical archive for better safety, regardless of format. Hell, you'll probably die before some of today's media fails.

a different question (1)

an_mo (175299) | more than 10 years ago | (#8386105)

... my subversion archive is now more than can fit in a cd. Is there a tool I can use to split the big file in two cd's, hopefully something that doesn't need another piece of software to reinstall the big file.

Re:a different question (1)

AdamPiotrZochowski (736869) | more than 10 years ago | (#8387344)

just use RAR volume (split file) capabilities

originally designed to let you fit large files
onto floppies, but can be used for everything,
splitting dvds onto cds, etc. from the manual:

-v[k|b|f|m|M]

Create volumes with size=*1000 [*1024 | *1].
By default this switch uses as thousands (1000) of bytes
(not 1024 x bytes). You may also enter the size in kilobytes
using the symbol 'k', in bytes using the symbol 'b',
in megabytes - 'm', in millions of bytes - 'M' or select
one of several predefined values using the symbol 'f'
following the numerical value. Predefined values can be
360, 720, 1200, 1440 or 2880 and replaced with corresponding
floppy disk size.

If the size is omitted, autodetection will be used.

If volumes are created on removable media, then after
the first volume has been created, user will be prompted
with:

Create next volume: Yes/No/All

At this moment in time, you should change the disks. Answering
'A' will cause all volumes to be created without a pause.

By default RAR volumes have names like 'volname.partNNN.rar',
where NNN is the volume number. Using -vn switch it is
possible to switch to another, extension based naming scheme,
where the first volume file in a multi-volume set has
the extension .rar, following volumes are numbered from .r00
to .r99.

When extracting or testing a multi-volume archive you must use
only the first volume name. If there is no next volume
on the drive and the disk is removable, the user will be
prompted with:

Insert disk with

Insert the disk with the correct volume and press any key.

If while extracting, the next volume is not found and volumes
are placed on the non-removable disk, RAR will abort with
the error message:

Cannot find

Archive volumes may not be modified. The commands 'd', 'f', 'u',
's' cannot be used with Multi-volume sets. The command 'a' may
be used only for the creation of a new multi-volume sequence.

It is possible, although unlikely, that the file size, of a file
in a multi-volume set, could be greater than it's uncompressed
size. This is due to the fact that 'storing' (no compression if
size increases) cannot be enabled for multi-volume sets.

Archive volumes may be Self-Extracting (SFX). Such an archive
should be created using both the '-v' and '-sfx' switches.

Example:

create archive in volumes of fixed size:

rar a -s -v1440 floparch.rar *.*

will create solid volumes of size 1440000 bytes.

Re:a different question (1)

Laur (673497) | more than 10 years ago | (#8389891)

What's wrong with the split command? I believe the file parts are reassembled by just cating them together.

Rar has one of the best Recovery methods (2, Informative)

AdamPiotrZochowski (736869) | more than 10 years ago | (#8387068)


rar has one of the best recovery methods, as it has mutliple of them.

during compression:
Recovery Record (-rr option)

it has Recovery Record, this is data appended to the actual
rar file that lets you recover from errors within a file. The
default RR takes 1% of the archive and lets you recover 0.6%. You
can change this behaviour to going more recoverability by
specifying -rr[N]p and telling it larger percantage for recoverability.

Recovery Volume (-rv option)

further more, rar supports PAR like volumes called REV
That can recover full missing files. For all you are concerned REV is
PAR, except its integrated to RAR utility. all you type is unrar *.rar
and rar will recover files for you, either through RR or REV. No need
to muck around twenty different utilities just to ensure proper file.

Non Solid Archiving (-s- option)

Further more, rar support non solid archiving, meaning each file is
saved using new compression statistics. You will lose some space due
to this method, however you will gain speed (you dont need to decompress
first 20 files to gain access to 21st file), as well as you will gain
partial recoverability (if file 20 is corrupt, you can still decompress
file 21)

during decompression:
Keep Broken Files (-kb option)

By default, like most archiving software, rar will not save a file
that is known that is corrupt, unless you explicitly force it to do
so.

I highly recommend checking out the command line manual to RAR,

Eugene Roshal is GOD

Re:Rar has one of the best Recovery methods (1)

Wolfrider (856) | more than 10 years ago | (#8395336)

--You know, if he would lower the registration price to ~$15 or so I bet he'd get a lot more incoming money. The economy SUCKS, and $29-30 is just way too much for what basically amounts to a zip-competitor, even though RAR is a good archiver.

Re:Rar has one of the best Recovery methods (1)

AdamPiotrZochowski (736869) | more than 10 years ago | (#8399250)

I dont see what you are trying to say. Compare
winzip to winrar:
- same price
- winrar comes with full command line, and gui interface
- both support variaty of decompression schemes
- zip provides worse compression ratios than rar
- zip has virtually no recovery methods
- zip has no multi archive support (unless you
consider the current hack as a valid system)
- zip uses pathetic encryption (password breakers
exist for over a decade, rar still has not a
single password breaker)

So crappy Zip has same asking price as amazing
Rar. I fail to follow your logic.

Re:Rar has one of the best Recovery methods (1)

Wolfrider (856) | more than 10 years ago | (#8400169)

--Having not had a job for a long time, I won't pay for Winzip either. Why should I, when "zip" and "unzip" comes free with Linux? The only problem with Zip currently is the pathetic 2GB filesize limit. This should have been fixed (at least on the Linux side) circa 1999 or so.

--However, I *would like to* support Rar, as I perceive it to be a superior archiver. I could see my way to donating $10-$15 to help the guy make a living, but the current price is too much.

(I hardly ever use Windoze for anything but games - and certain websites that don't render properly with Linux browsers - anymore. Most of my time is spent in Linux, and I much prefer using the commandline to GUI.)

There is a Free Software extractor for RAR files (1, Informative)

Anonymous Coward | more than 10 years ago | (#8387328)

You definitely shouldn't use RAR for archival purposes, but for extracting existing archives, try unrarlib [unrarlib.org] . It includes a library for accessing the contents of RAR files, and an "unrar" utility based on this library. It is dual-licensed under the GPL and a more restrictive license.

DAR & Parchive (1)

UnderScan (470605) | more than 10 years ago | (#8392029)

DAR - Disk ARchiver & Parchive combined sounds like it would work wonders.

From http://dar.linux.free.fr/ [linux.free.fr] :
dar is a shell command, that makes backup of a directory tree and files. It is released under the GNU General Public License (GPL in the following) and actually has been tested under Linux, Windows and Solaris. Since version 2.0.0 an Application Interface (API) is available to open the way to external independent Graphical User Interfaces (GUIs). An extension of this API (in its version 2) is in the air, for release 2.1.0, and would overcome some limitation of API version 1. This API relies on the libdar library which is the core part of DAR programs and, as such, is released under the GPL. In consequences, to use it, your program must be released under the GPL, no commercial use will be tolerated

...

...
Archive Testing

thanks to CRC (cyclic redundancy checks), dar is able to detect data corruption in the archive. Only the file where data corruption occurred will not be possible to restore, but dar will restore the other even when compression is used.


Parchive http://parchive.sourceforge.net/ [sourceforge.net] :

Parchive: Parity Archive Volume Set

The original idea behind this project was to provide a tool to apply the data-recovery capability concepts of RAID-like systems to the posting and recovery of multi-part archives on Usenet. We accomplished that goal. Our new goal with version 2.0 of the specification is to improve. It extends the idea of version 1.0 and takes the recovery process beyond the file-level barrier. This allows for more effective protection with less recovery data, and removes some previous limitations on the number of recoverable parts. See Par1 compared to Par2 for a more detailed view of the differences.

You can, too, recover tar archives!! (see: tarx) (2, Informative)

Helen O'Boyle (324127) | more than 10 years ago | (#8395009)

{ The poster is looking for alternatives to tar, because he has concerns about tarball content recovery. }

It's been possible to do that for well over a decade, using various utilities such as tarx. I've successfully recovered files after a damaged point in a tarball many times. (Sigh, I used to use an old AT&T UNIX with a #$*@# broken tar, which occasionally created corrupt tarballs).

See this post [sunmanagers.org] on the Sun Managers list circa 1993, and the venerable comp.sources.unix collection, volume 24, for the sources.

Check for New Comments
Slashdot Login

Need an Account?

Forgot your password?