Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Volume Shadow Copy For Linux?

Soulskill posted more than 4 years ago | from the restorative-tux dept.

Linux 300

An anonymous reader writes "I was asked to manage a number of Linux servers at work. I would like to use volume snapshots to improve my backup scripts and keep recent copies of data around for quick restore. I normally manage Windows servers and on those I would just use Microsoft's Volume Shadow Copy for this. I tried Linux LVM snapshots, but most of the servers I manage run regular partitions with ext3 file systems, so LVM snapshots will not work. I found some versioning file systems out there like ext3cow and Tux3. Those look interesting, but I need something I can use on my existing ext3 file systems. I also found the R1Soft Hot Copy command-line utility, but it does not yet support my older 2.4 Linux servers. What are you using to make snapshots on Linux?"

cancel ×

300 comments

Sorry! There are no comments related to the filter you selected.

hey retard: (-1, Flamebait)

larry bagina (561269) | more than 4 years ago | (#32540642)

move your partitions onto lvm.

Re:hey retard: (1, Troll)

suso (153703) | more than 4 years ago | (#32540710)

Hey, IRC called, they want their asshole back.

Re:hey retard: (0)

Anonymous Coward | more than 4 years ago | (#32540760)

Your asshole called, it wants the 1990's back.

Re:hey retard: (-1, Offtopic)

Anonymous Coward | more than 4 years ago | (#32541112)

Ahh yes, the 1990s... before suso lost his ass cherry but found anal warts. No wonder his asshole wants to go back in time!

Re:hey retard: (0)

Anonymous Coward | more than 4 years ago | (#32541002)

How's he being an "asshole"? LVM is exactly what the topic submitter needs. It's filesystem-independent, can work with existing filesystems, has been supported since the later releases of Linux 2.2, is well-tested in production environments, and has a very minimal performance impact.

Re:hey retard: (4, Insightful)

Omnifarious (11933) | more than 4 years ago | (#32541196)

In case you hadn't realized this, It is possible to tell people to migrate to LVM without calling them names.

Re:hey retard: (1)

larry bagina (561269) | more than 4 years ago | (#32541982)

that's not as much fun.

l0lz0rs (-1, Offtopic)

Anonymous Coward | more than 4 years ago | (#32540646)

GNAA 4 LIFE. Goatse is my god!

nilfs (2, Informative)

pinkishpunk (1461107) | more than 4 years ago | (#32540652)

Nilfs will have those things, but you`ll have to wait till its production ready.

Re:nilfs (4, Funny)

Chainsaw (2302) | more than 4 years ago | (#32541218)

Nerds I'd Like to... You know what - nevermind.

Re:nilfs (1)

HogGeek (456673) | more than 4 years ago | (#32541722)

Nerds I'd like to file system?

I don't get it... /sarcasm

Re:nilfs (0)

Anonymous Coward | more than 4 years ago | (#32542398)

...fsck, of course.

You're confused (0, Troll)

suso (153703) | more than 4 years ago | (#32540668)

I tried Linux LVM snapshots, but most of the servers I manage run regular partitions with ext3 file systems, so LVM snapshots will not work.

Last time I checked they worked fine. I think you mean something else by "snapshots".

Re:You're confused (4, Informative)

pbowen (9912) | more than 4 years ago | (#32540720)

LVM snapshots only work fine if you are using LVM. I think the OP uses "regular partitions" to mean no volume manager in use.

Re:You're confused (1)

verbatim_verbose (411803) | more than 4 years ago | (#32540732)

I think you might have misread what he said.

ext3 file systems on regular partitions wouldn't be able to use LVM snapshots, because, well, they're not running on LVM volumes.

Re:You're confused (1, Informative)

nschubach (922175) | more than 4 years ago | (#32540836)

Isn't that like complaining that your FAT32 partitions in Windows are not supported by Shadow Volume Copy then?

I think there's a bit of double standard here in the question. The OP is stating that they want to use a feature on an older server (2.4 kernel?) that isn't available unless you update the server, reformat, or what have you. The same applies to a Windows environment.

I think the question is mis-guided. They should be asking for a SVC like feature for kernel 2.4 and ext3 systems.

Re:You're confused (1)

nschubach (922175) | more than 4 years ago | (#32540864)

Replace: "Volume Shadow Copy" and "VSC" respectively in my post above. Sorry.

Re:You're confused (-1, Flamebait)

Anonymous Coward | more than 4 years ago | (#32540930)

How is pointing out the solutions that won't work a "double standard"? Or more to the point - why are you being a defensive asshat? I didn't see anything anti-Linux in the OP's question. The problem isn't what question he asked, it's how you chose to interpret the background he provided.

Re:You're confused (3, Insightful)

nschubach (922175) | more than 4 years ago | (#32541042)

The double standard being that the Linux servers wouldn't need updated where the Windows servers would. There's an update that has to happen to support the feature. Linux is not immune to this (though it would likely do the update without a total rebuild opposed to Windows.)

Re:You're confused (1)

verbatim_verbose (411803) | more than 4 years ago | (#32541118)

Um... I'm not really following anything you're saying.

Kernel 2.4 supports LVM, he just isn't using it.

Re:You're confused (1)

nschubach (922175) | more than 4 years ago | (#32541290)

I tried Linux LVM snapshots, but most of the servers I manage run regular partitions with ext3 file systems, so LVM snapshots will not work.

They have, presumably, tried and failed. It could support it, but it would have to be installed, updated or something to get it to work. There's something they're not doing on an old server that needs to be changed to support the feature they want. I was pointing out how this is not exclusive to the Linux servers.

Re:You're confused (5, Insightful)

nashv (1479253) | more than 4 years ago | (#32541334)

He isn't complaining. You seem to be responding to his mentioning that "he knows how to do this on Windows" , by interpreting it as "Why is Linux so broken that it can't do a simple thing like that?" This isn't a Linux versus Windows thing. This is a Windows user, migrating to Linux and wants to know how to accomplish something. Constructive answers are more useful in such cases than getting defensive by alleging hypocrisy and double standards.

Re:You're confused (1)

nschubach (922175) | more than 4 years ago | (#32541500)

You are getting hung up on the choice of words.

Re:You're confused (1)

suso (153703) | more than 4 years ago | (#32540746)

Nevermind, I was the one that was confused. You currently have ext3 fs that are NOT on LVM. In the future, choose LVM.

Re:You're confused (4, Informative)

amorsen (7485) | more than 4 years ago | (#32540996)

You currently have ext3 fs that are NOT on LVM. In the future, choose LVM.

The choice isn't that simple. LVM comes with its own complications, including a tendency to get volume offsets "wrong" so the file system data doesn't align nicely to RAID stripes. This is not good for performance.

Also, LVM has only recently acquired barrier support, and the combination of no barriers + write cache can be quite dangerous if power is lost. Even battery backed cache won't save you if you use a journalling file system (and everybody does these days) because request ordering isn't guaranteed.

I haven't touched Solaris since it had a 2 in front of its version number, but I must admit that I suffer from ZFS envy.

Re:You're confused (1)

Crackez (605836) | more than 4 years ago | (#32542350)

Can you elaborate on how LVM gets volume offsets "wrong" and it's impact on RAID performance? I've never heard of that before. Not saying you're wrong, it's just news to me.

Also, IIRC Solaris still has a 2 in front of it. Ie, Solaris 10 is really Solaris 2.10, and is simultaneously SunOS 5.10. They have retarded naming conventions. Oh well, Oracle can only fsck it up further...

ZFS does rock btw, we've got about 80-90TB (guesstimate) spinning with ZFS right now. Our thumper has 32.5GB alone, and another HP box running x86 Solaris has 45.3TB. There's others, but not as big as those two. ZFS really does make it a breeze to manage that much storage.

Re:You're confused (0)

Anonymous Coward | more than 4 years ago | (#32540748)

LVM snapshots may work fine for LVM volumes, but not for straight ext3 partitions. converting to LVM requires repartitioning.

Re:You're confused (1)

yargnad (1456405) | more than 4 years ago | (#32540774)

Thanks for elaborating, we will try your suggestions and report back.

Re:You're confused (5, Informative)

impaledsunset (1337701) | more than 4 years ago | (#32540862)

If they are indeed regular partitions, he can't use LVM snapshots. However, the best solution is to convert from partitions to LVM volumes. It's a little effort to do so, but switching is worth it. Second best is to wait until btrfs is more usable. As a ZFS user, I can say that filesystem-level snapshot are much nicer than LVM snapshots in lots of ways.

Another possibility is to abuse hardlinks. You can create a copy of a directory with cp -al, and then overwrite (not modify) files on the original, you'd have copy-on-write copy. If you make your backups with rsync, you can configure rsync to never write to existing files and always overwrite, then use cp -al each weak or day, to store "incremental" backups for weeks and maybe more. I personally found this solution nice, but then I installed Solaris on the backup machine and used ZFS snapshots which do the same safer, simpler and more efficiently. If the backups are stored on a separate machine, switching it to Solaris is an option.

Another thing that can be done is to keep the LVM snapshots on a separate machine, and leave the current partitions as they are. It can be done with rsync, or a drdb device can be used to sync with the server (it can be created without reformatting the partitions, but you still need to make some changes like shrinking and/or moving the data, which might destroy the data if you don't know what you're doing).

Re:You're confused (2, Informative)

Jason Earl (1894) | more than 4 years ago | (#32541098)

With judicious use of --link-dest rsync can do this for you without having to use cp at all.

Upgrade path. (1)

leuk_he (194174) | more than 4 years ago | (#32541944)

Isn't there some upgrade path to convert "old" partitions to LVM partition? Just like windows supports upgrading to dynamic partitions and FAT32 -> NTFS conversions.

Yes, you always need a backup, but having a backup + doing inplace upgrade is far more convinient.

no offense, but what a windows mentality (5, Funny)

Anonymous Coward | more than 4 years ago | (#32540680)

Linux servers don't get backed up, they only get migrated to new hardware every five years. (BSD is the same, except for the second part.)

Re:no offense, but what a windows mentality (5, Funny)

mikemsd (225775) | more than 4 years ago | (#32540734)

Sweet, I'm going to install Linux on all my systems. I didn't know that Linux could prevent natural and man made disasters as well as being a stable operating system. We've been wasting all this money on backup for all these years.

Re:no offense, but what a windows mentality (0)

Anonymous Coward | more than 4 years ago | (#32541000)

the linux server is probably sitting next to all the contracts your clients signed, which are actually more important for you

Re:no offense, but what a windows mentality (5, Interesting)

Unoti (731964) | more than 4 years ago | (#32541452)

Sweet, I'm going to install Linux on all my systems. I didn't know that Linux could prevent natural and man made disasters as well as being a stable operating system. We've been wasting all this money on backup for all these years.

There's a mix of humor and catty vitriol here all around, but here is something that addresses the serious point made in Grandparent's statement about it being a "Windows" way of thinking.

Take a look at Infrastructures.org [infrastructures.org] which describes a whole way of thinking about server reliability and configuration. Where I work we essentially use this approach. The fundamental concepts around this approach concentrate more on system configuration, ability to pick a random server and drop it out the window and have another one just like it online in moments. It's less about backups, and far more about a more comprehensive disaster recovery/prevention type of thing. The types of approaches described there are probably more easily implemented using Unix/Linux, but is probably also possible with Windows boxes.

Re:no offense, but what a windows mentality (0, Flamebait)

alexborges (313924) | more than 4 years ago | (#32541478)

"We've been wasting all this money on backup for all these years."

Yes, by not using AMANDA or another FOSS backup solution, youve thrown money down the drain.

Re:no offense, but what a windows mentality (1)

Anomalyst (742352) | more than 4 years ago | (#32541618)

Yes, by not using AMANDA

If only one could install it on Ubuntu 10.4 LTS, oh well maybe it will be fixed next quarter.

Re:no offense, but what a windows mentality (1)

dajekels (1831654) | more than 4 years ago | (#32540824)

Linux servers don't get backed up, they only get migrated to new hardware every five years. (BSD is the same, except for the second part.)

this guy probibly runs all his windows servers via xwindows manager. nice >.

Suck it up (4, Insightful)

mewsenews (251487) | more than 4 years ago | (#32540686)

You will have to migrate your servers with plain ext3 to LVM-based ext3. Short term pain for long term gain.

Re:Suck it up (0)

Anonymous Coward | more than 4 years ago | (#32540712)

Agreed.

Fix it.

Re:Suck it up (1)

Omnifarious (11933) | more than 4 years ago | (#32541078)

That would be my advice too. Migrate.

Eventually you're going to want to migrate those filesystems to btrfs as well, and that has really nice built-in snapshotting capability. But until then (a few years from now) move to LVM. It will save you so much hassle and be much more tested and stable than some hack like ext3cow

Re:Suck it up (0)

Anonymous Coward | more than 4 years ago | (#32541352)

And this is why I like Linux so much. The oh-so friendly community and the nice robust all-in one technologies. :P

Re:Suck it up (1)

alexborges (313924) | more than 4 years ago | (#32541512)

You cant do this in windows either, not with partitions. Thats why their solution is called shadow VOLUME. Cause it need VOLUMES to work.

OUr solution is called LVM Snapshots cause it needs LVM VOLUMES to work.

Now is that so hard to understand?

Re:Suck it up (0)

Anonymous Coward | more than 4 years ago | (#32541836)

Ditto. LVM provides exactly what is needed. It isn't that hard to move the native partitions in to logical volumes. Where it is (because you have a truly ancient release of some distro or other) just backup with tar or something.

And stop building machines without LVM. At most only /boot should be native. Everything else in LVM. Always.

rsnapshot (0)

Anonymous Coward | more than 4 years ago | (#32540690)

maybe rsnapshot (http://www.rsnapshot.org) will fit your bill...

Re:rsnapshot (1)

fearlezz (594718) | more than 4 years ago | (#32540846)

rsnapshot has no actual snapshotting functionality. It's just a (very useful) wrapper around rsync that takes care of de-duplication. While making a rsync/rsnapshot backup, files on the system can still be changed.
For example: A LVM snapshot would give you a consistent MySQL dump if you're using innodb. Rsync/rsnapshot does not.

"does not yet support my older 2.4 Linux servers" (4, Insightful)

tlambert (566799) | more than 4 years ago | (#32540714)

"does not yet support my older 2.4 Linux servers"

So upgrade your servers to a supported release instead?

-- Terry

Re:"does not yet support my older 2.4 Linux server (3, Informative)

0racle (667029) | more than 4 years ago | (#32540908)

There are Enterprise Linux distributions that are both supported and still run 2.4, though not for much longer. Not everyone runs Ubuntu.

Re:"does not yet support my older 2.4 Linux server (0)

Anonymous Coward | more than 4 years ago | (#32540976)

2.6 came out many years ago. It's hardly bleeding edge.

Re:"does not yet support my older 2.4 Linux server (-1, Offtopic)

QuantumRiff (120817) | more than 4 years ago | (#32541038)

Your post..

your Sig.

My head, spinning...

Re:"does not yet support my older 2.4 Linux server (2, Informative)

Bratmon (1649855) | more than 4 years ago | (#32541104)

Can you give us an example?

The obvious one is Redhat. RHEL 3 is the newest version to still use a 2.4 kernel. It may still be supportes, but that expires October of this year, so the OP should already be getting ready to upgrade.

Re:"does not yet support my older 2.4 Linux server (0)

Anonymous Coward | more than 4 years ago | (#32542322)

There are Enterprise Linux distributions that are both supported and still run 2.4, though not for much longer. Not everyone runs Ubuntu.

I'm sure most do not run Ubuntu. I work for a very large company and we only use RHEL and SuSE on physical hardware and mainframe LPAR.

Re:"does not yet support my older 2.4 Linux server (4, Informative)

impaledsunset (1337701) | more than 4 years ago | (#32541028)

I'm troubled why people still run 2.4 server. I remember the time when I was reluctant to upgrade to 2.6, and I used prefer the older 2.4, which felt more comfortable than 2.6, regardless of how tempting the new changes sounded. But now I don't see any reason I would run this anywhere, even my router runs 2.6. Especially on newer hardware, 2.4 is really really too old.

I know there are people who probably still run Linux 2.2, but that are probably systems that are running some task well enough to require any changes, and leaving them as they are is the best. Servers are usually not like that. They need security updates, upgrades to catch up with the times, and many other changes required by the circumstances (for example, adding snapshot abilities, for which some person asked recently on Slashdot). Most production servers are not systems that you just leave running, so upgrades to the kernel are also expected and highly recommended. Not to mention that most recent distributions require 2.6.

Re:"does not yet support my older 2.4 Linux server (1)

richie2000 (159732) | more than 4 years ago | (#32541344)

I'm troubled why people still run 2.4 server

They fit in well with his other servers, still running Windows 98 Advanced Server Edition.

But seriously, migrate to LVM already.

ummmm (1)

ProfessorKaos64 (1772382) | more than 4 years ago | (#32540716)

dd command? Works well for image snapshots for me, copys raw bit by bit

Re:ummmm (3, Informative)

pavon (30274) | more than 4 years ago | (#32540906)

Only works if the partitions don't change while you are copying them. The big advantage of using LVM for this is that you can create snapshots on a live system, without resorting to remounting the partition read-only (and all the problems that will cause).

But really, those are his only options. If you insist on using plain ext3 and won't add a layer of between the FS to allow for this, then you have no choice but to freeze the partition while doing a volume-level back up.

Re:ummmm (0)

Anonymous Coward | more than 4 years ago | (#32541054)

Put in a couple of new drives and run something like:
dd if=/dev/hdx of=/dev/hdy
overnight, once that's done, repartition the existing drives for LVM and start restoring/rebuilding...

LVM Snapshots (2, Informative)

theunixman (538211) | more than 4 years ago | (#32540722)

I use them on ext3 with no problems. It's true that very early on there was a problem with them and journaled filesystems, but that has long since been solved.

There's some sort of confusion here (4, Insightful)

vadim_t (324782) | more than 4 years ago | (#32540724)

LVM snapshots work on a block level and don't care about the filesystem. A snapshot of any data in a logical volume should work fine, even if it's not a recognized filesystem.

A nice use for this is using a read/write snapshot to try different strategies for recovering a broken filesystem.

Re:There's some sort of confusion here (1)

Trepidity (597) | more than 4 years ago | (#32541198)

I think he's saying that he can't use LVM snapshots, because some of his servers have ext3 directly on a plain partition, not on top of LVM (used to be a common setup).

Re:There's some sort of confusion here (2, Insightful)

Unoti (731964) | more than 4 years ago | (#32541274)

Yes, and that's what he should change. Don't you think?

Not Enough Info (1)

crow_t_robot (528562) | more than 4 years ago | (#32540728)

I'm going to answer your question with a question:

How many servers and what is the approximate amount of data that you need to backup?

ZFS? (3, Informative)

corran__horn (178058) | more than 4 years ago | (#32540770)

I will admit that I have not tried it on Linux, but zfs is the best of the next gen filesystems. It does cryptographically assured reads and writes (remember that transitory undetected disk malfunctions occur at a rate of ~1/TB of data), it can snapshot changes, it fricken slices bread. If it had a gender, I would probably marry it (well, I guess I can date it for a while and see how things work out). http://zfs-fuse.net/ [zfs-fuse.net]

Re:ZFS? (0)

Anonymous Coward | more than 4 years ago | (#32540990)

zfs is great under OpenSolaris; if you never tried doing things with fuse under Linux, then suggesting it as a solution is not well informed. pretty much all fuse solutions are lousy. zfs-fuse only serves to put a lousy wrapper around a very strong fs.

rsnapshot (0)

Anonymous Coward | more than 4 years ago | (#32540794)

It sounds like nilfs is more along the lines of what you're looking for, but something like rsnapshot might be useful to you.

http://www.cyberciti.biz/faq/linux-rsnapshot-backup-howto/ HowTo on setting it up on Debian/Ubuntu

It takes timed snapshots of the filesystem, so you'll only be able to roll back to the last snapshot.

rsnapshot is what you're looking for (4, Informative)

xee (128376) | more than 4 years ago | (#32540812)

RSnapshot uses a clever blend of rsync + hard links to do what you want... you can store many incremental backups in just a little more space than a full backup. you can run rsnapshot on a backup server with lots of disk space, and all you need to expose on your target machines is SSH.

you'd create "backup" users on all the target hosts, generate a PKI key pair, and put the private key on your backup server. put the public key in the "backup" account on each target machine so the backup server can securely login without a password. then you just set up rsnapshot to log into your targets and it will use rsync-over-ssh to pull the data.

http://rsnapshot.org/ [rsnapshot.org]

Re:rsnapshot is what you're looking for (0)

Anonymous Coward | more than 4 years ago | (#32541092)

Rsnapshot really does work perfectly for this situation.

We use a combination of samba + rsnapshot to make the snapshots available through the shadowcopy interface, if you're interested in doing a similar thing you can find the scripts that you need to do so here: http://sites.google.com/site/evanquirk/Home/shadowcopy-through-samba-with-rsnapshot-backups

Re:rsnapshot is what you're looking for (2, Informative)

dfn_deux (535506) | more than 4 years ago | (#32541370)

This is no good for a true snapshot since the rsync operation is non-atomic on a live filesystem.

Re:rsnapshot is what you're looking for (2, Informative)

jgrahn (181062) | more than 4 years ago | (#32541894)

[RSnapshot] is no good for a true snapshot since the rsync operation is non-atomic on a live filesystem.

I cannot help wondering when I read stuff like that who *really need* atomic, and who just like it because it sounds cool ... If that 2.4 guy doesn't really *need* theoretical atomicity, and he can do his work with something much more simple, he should.

Wrong (0)

Anonymous Coward | more than 4 years ago | (#32541490)

Rsnapshot is great, but it does not create a point in time snapshot of the entire volume or filesystem like an LVM or filesystem snapshot will.

It may be enough for the OP's purposes (it's enough for mine), but it just ain't the same thing as Volume Shadow Copy.

Hobbling Along (2, Informative)

kwalker (1383) | more than 4 years ago | (#32540816)

Since the situation is so hobbled (Old Linux kernel, no LVM) about the only thing you will be able to do is learn to use hardlinks [wikipedia.org] . The ext* filesystems support them but you will have to manage them yourself (cp -varl /source/* /destination/version). Yeah it's a huge hack, but unless you can actually fix the problem, it's about your only hope.

Re:Hobbling Along (2)

ckthorp (1255134) | more than 4 years ago | (#32540934)

Or use rsnapshot and be lazy...

Eh? (5, Informative)

ledow (319597) | more than 4 years ago | (#32540822)

If you have backups, then moving to LVM is obviously the way to go if you desire snapshots. The others options are short-term hackery, LVM was designed from the ground up to do such things. And Ext3 has nothing to do with the price of butter.

To clarify, let me rephrase your question for the other way around

"I was asked to manage a number of *Windows* servers at work. I would like to use volume snapshots to improve my backup scripts and keep recent copies of data around for quick restore. I tried Windows Shadow Copy, but most of the servers I manage run MBR partitions with FAT file systems, so Shadow Copy will not work. I found some versioning file systems out there... Those look interesting, but I need something I can use on my existing FAT file systems. I also found --random freeware--, but it does not yet support my older Windows NT 3.5 servers. What are you using to make snapshots on Windows?"

Except, in that case, it makes more sense because the filesystem is the determining factor, not the volume management. If you have LVM, it doesn't matter what the underlying filesystem is, really. Stop faffing about - if you have a server, with backups, that you need snapshots on, take the hit and wipe the drives to a config that supports that... while you're there upgrade that damn kernel already. If nothing else, it will test that the backups you're making are actually worth the effort. It's like complaining that 95 on FAT16 doesn't support Shadow Copy. If you absolutely *can't* take those servers down, or am unable to restore your backups to another machine for testing such changes (whether because of compatibility, software licensing and/or bad backups), you have bigger problems than some random desire for a feature you don't actually *need* at the moment.

Re:Eh? (4, Insightful)

gbjbaanb (229885) | more than 4 years ago | (#32541300)

If you absolutely *can't* take those servers down

If you can't take those servers down, nature will be getting ready to do it for you. At a time when you least want it's "assistance".

Why do mod points stop at FIVE? (1)

wonkavader (605434) | more than 4 years ago | (#32542304)

This is clearly the definitive answer for this article.

We can all stop posting now.

Rsync, dd (1, Interesting)

Anonymous Coward | more than 4 years ago | (#32540916)

Most people just use rsync for backups. LVM is a lower level then the file system... you create a storage pool and then create filesystems on top of that. If you really want to use LVM snapshots you would have to recreate your filesystems, which I gather you would like to avoid. There are lots of options for filesystems that include versioning in some form or another. If you do want to choose one of these filesystems, find the Wikipedia article for any filesystem and down at the bottom will be a list of filesystems you can explore. If you just want to snapshot the entire partition, you can remount it readonly and use dd to copy the block device to a file (this will take up space for every byte, including unused space unless you compress it). I congratulate you on the opportunity to work in a unix environment. Please don't get discouraged because it is different, it really isn't that hard to learn, and many people would say it's easer. Google and Wikipedia will be indispensable.

DRBD much? (0)

Anonymous Coward | more than 4 years ago | (#32540926)

Weee!

You have 2 good options for snapshot with linux: (0)

Anonymous Coward | more than 4 years ago | (#32540932)

The more costly, but by far best, is to invest in a small netapp and point the data to it. Netapp has the best snapshot technology of any other out there. If you want whole OS, you can provision some iSCSI on the netapp box and iSCSI boot, and take advantage of snapshots on your OS.

Secondly, build up a solaris box with a large ZFS filesystem hanging off of it (which has snapshots), setup some NFS exports and do the same thing but using ZFS snapshot technology on the backend. Short of those two options, you are going to have to wait for the next-generation linux supportable local filesystems to mature a bit further. There are other options out there, but from my experience, none that don't wont suffer a significant performance hit with the technology. Netapp's filesystem doesn't have that penalty, and zfs doesn't have as bad of a penalty as others.

Wow, where to start... (2, Informative)

CAIMLAS (41445) | more than 4 years ago | (#32540948)

Knowing where to start on this is a bit of a miffing point.

First: upgrade your shit. 2.4 kernel systems? Are you running Redhat 6? You know, from the turn of the millennia.

Second: upgrade your shit. Really,

Third: if your kernels are that old and you're using these machines for file storage/backup, chances are the hardware needs to be replaced before you even consider considering messing with them. Seriously: this stuff is ancient. Even Debian hasn't had a 2.4 kernel in 5+ years, I think.

Third: you can do what you're trying to do with rsync 'snapshots'. It works very well, failing filesystem level support. If you're sharing data over samba, this makes it easy: just put a '.snapshot' dir for these 'temporary' backups in their $HOME and hide dotfiles. Then make sure rsync ignores .snapshot. (Of course, there are other ways to do this.)

rsync snapshots [mikerubel.org] (and here [backupcentral.com] ).

There are other sources of information out ther on rsync snapshots. There's also rsnapshot.

Chances are you'll have to upgrade before this stuff even works for you, though.

Backup != snapshot != package management (4, Informative)

Alex Belits (437) | more than 4 years ago | (#32540964)

People expect a snapshot to be immediately usable and reliable, however in practice a state of device, even if synchronized with filesystem through its transaction is not a state of all data -- some data may be in buffers, prepared to be written, and rebooting into a restored filesystem may require some cleanup of such state. In particular, SQL databases are completely unsuitable for this kind of backup (this is why they have their own backup and transaction log handling procedures), and database-like applications such as mail servers, may require reindexing.

However for purposes other than those applications, file-level backup is entirely adequate, so utilities like rdiff-backup end up providing more functionality than complicated snapshot-handling procedures -- incremental backups for subtrees, readable trees in backup media, etc.

It also should be noted that backups should not be used as a replacement of package management -- on Linux anything installed through a package manager can be uninstalled through it.

rdiff-backup (1)

xororand (860319) | more than 4 years ago | (#32541034)

Take a look at rdiff-backup [nongnu.org] .

rdiff-backup backs up one directory to another, possibly over a network. The target directory ends up a copy of the source directory, but extra reverse diffs are stored in a special subdirectory of that target directory, so you can still recover files lost some time ago. The idea is to combine the best features of a mirror and an incremental backup. rdiff-backup also preserves subdirectories, hard links, dev files, permissions, uid/gid ownership, modification times, extended attributes, acls, and resource forks. Also, rdiff-backup can operate in a bandwidth efficient manner over a pipe, like rsync. Thus you can use rdiff-backup and ssh to securely back a hard drive up to a remote location, and only the differences will be transmitted. Finally, rdiff-backup is easy to use and settings have sensical defaults.

I can confirm that rdiff-backup is indeed easy to use.

Next3 (1, Informative)

Anonymous Coward | more than 4 years ago | (#32541048)

Funny... I read this question while browsing /. during lunch. Next in line was Linux.com which had a link to an article on Next3: ext3 with snapshots.

Here it is: http://www.linux.com/news/software/developer/317784-next3-ext3-with-snapshots

What is wrong with just plain dump? (1)

mr_da3m0n (887821) | more than 4 years ago | (#32541056)

Seriously, what is wrong with dump(8)? It works on ext3. I use it on FreeBSD. It takes snapshots to do the dump, so you can shutdown your database, start the dump and then immediately start your database again. Of course you have to backup the entire volume, but still...

Re:What is wrong with just plain dump? (2, Informative)

Carl Drougge (222479) | more than 4 years ago | (#32541700)

I'm pretty sure the Linux version of dump doesn't do any snapshoting. The FreeBSD version can do it because the FS supports snapshots, but ext3 does not. (Maybe it will do snapshots automatically if you have a setup that will support them, but the original problem is that this is not the case.)

Definitely use an rsync-based program (1)

apexwm (1612713) | more than 4 years ago | (#32541184)

Thankfully, Linux is not nearly as complicated as Windows. Every time I've messed with Volume Shadow Copy it's turned into a huge headache. Doing similar functions on Linux just work right out of the box. I'd suggest anything that uses rsync, or you could use rsync directly. I use rsync which can synchronize gigs of data over the network, and is smart enough to parse partial sections of files. It's amazing and very efficient. Windows has nothing that even touches this, that I know of.

What Are You Talking About? (1, Informative)

Anonymous Coward | more than 4 years ago | (#32542100)

Every time I've messed with Volume Shadow Copy it's turned into a huge headache. Doing similar functions on Linux just work right out of the box.

What are you talking about? Volume Shadow Copy(VSC) is a simple checkbox to enable it and it works! Two snapshots per day by default. Users can restore previous versions directly from their workstations with a right click, select, OK. Or, with newer versions of Windows, drag the desired version from the VSC window to where ever you want it. How is that complicated? How could you screw that up?

Meanwhile, on Linux... Write a bash script with arcane rsync arguments, create a cron job for the script, ??? Does it work as desired/expected or do you have to debug your script, cronjob, permissions? Where are the backups? How do you restore them? How do you maintain permissions and access control, especially with users that are not working local to the machine?

Here's some more information for the deluded. Windows has it's own kind of rsync. It's called Robocopy and it does almost all of what rsync does, including maintaining Windows user information and ACLs, an area where rsync fails miserably in Windows environments. But, the built in Windows features like VSC and Distributed File System(DFS) replication make the use of rsync/robocopy painful, pointless and superfluous.

If a single checkbox is a headache for you, I don't see how rsync can possibly be less so.

Upgrade kernel + R1Soft (3, Informative)

msh104 (620136) | more than 4 years ago | (#32541204)

Just upgrade your kernel using a manual build of the 2.6 kernel.
Also install static versions of the modutils ( insmod, modprobe, etc )
Use an external ( a machine with decent software ) for this so your compile doesn't break.
I have done so in the past and it works fine. ( and plan an update for those machines, anything with 2.4 is way to old... )

After that you can just use R1Soft hot copy,
http://www.r1soft.com/tools/linux-hot-copy/ [r1soft.com]

This program is free ( as in beer ) to download and works with every block device.
You can even write to a block device if you really need to.

Their commercial offerings are pretty good as good. ( and DO work with the 2.4 kernel )
We use it here at work.

I heard btrfs supports something like this as well.

Any way, good luck!

Virtualise! (1)

tomm3h (1406683) | more than 4 years ago | (#32541262)

Dump the disks to a virtual image, virtualise the machines and then you can snapshot away to your hearts content. Be it via the virtualisation tech, or the host's file systems.

However, as already said, if you're still running 2.4 kernels, you're well over-due an upgrade/migration to newer software AND hardware environments. Why not combine both suggestions?

Understandably, uptime is really golden, but everyone understands that you have to upgrade and/or maintain systems at some point.

Re:Virtualise! (1)

Bruha (412869) | more than 4 years ago | (#32541526)

Uptime egotism is no excuse not to patch or upgrade a system. We've had 2.6 how many years now, and we still have people holding onto their 2.4 kernels like they were prized pets or something.

No brainer. (-1, Troll)

Anonymous Coward | more than 4 years ago | (#32541318)

Upgrade your server to FreeBSD 8 and use ZFS. Problem solved.

rdiff-backup (1)

gweihir (88907) | more than 4 years ago | (#32541446)

Works nicely. I use it for backups over the net. One very nice feature is that it does is reverse diffs, i.e. the nearer to the present time, the faster you get files restored. You also can remove older diffs without any fuss.

Copy On Write, Where for art thou? (2, Insightful)

Anonymous Coward | more than 4 years ago | (#32541528)

Why does Linux still lack this functionality?

Since the early 1990's Novell has had the ability to "Salvage" deleted files and even maintained a near limitless amount of previous versions with a Copy On Write functionality. It still exists, even on Linux in their NSS(Novell Storage Services) Volumes.

Microsoft finally got on board when their Server 2003 product implemented Volume Shadow Copies. This isn't nearly as good as Novell's implementation but, it was better than anything Microsoft had previously offered.

The original poster did mention etx3cow, which offers an awesome feature set. But, etx3cow has been "under development" for a long time without ever catching on.

Ext4 has recently been incorporated into the Linux kernel and there just isn't any excuse for its lack of a Copy On Write version history. Yet here we are, in 2010, yet again answering this question without any good answer. Linux should have a standard Copy On Write file system a long time ago. Its continued absence is shameful.

ext3cow should be merged into ext4 yesterday!

Veritas Storage Foundation (1, Insightful)

Anonymous Coward | more than 4 years ago | (#32541536)

http://www.symantec.com/business/storage-foundation-basic

Corporate advertising? (0)

Anonymous Coward | more than 4 years ago | (#32541616)

Is this article posted by an employee,
of R1Soft?

Two valid options. (1, Redundant)

Lost Penguin (636359) | more than 4 years ago | (#32541758)

ZFS and BTRFS, btrfs is included with Fedora 13 (maybe 12 too).

ZFS is found here http://wiki.github.com/behlendorf/zfs/

Check out RemoteShadow (0)

Anonymous Coward | more than 4 years ago | (#32542070)

http://www.advsyscon.com/products/rso/

Handles OPENVMS and UNIX

So, wheres the 6 bullet point solutions list? (1, Insightful)

stimpleton (732392) | more than 4 years ago | (#32542182)

I have scrolled to the bottem and replied after a scrolling skim. No answers for this guy yet? Just vague debate. No "use either a,b,c,d,or e".

Thats telling.

I am in this guys situation at work.

re (0)

JohnVanVliet (945577) | more than 4 years ago | (#32542240)

why not use "dd" it is already installed

Re: Volume Shadow Copy (1)

mpapet (761907) | more than 4 years ago | (#32542388)

Volume shadow copy might not be doing what you think it is. I know it makes a big file, but restoring a snapshot has never worked as expected.

It sounds like you are just trying to keep an old dinosaur alive. Reboot with a live cd and make a snapshot of the old dog.

dd if=/dev/hda conv=sync,noerror bs=64K | gzip -c > /mnt/sda1/hda.img.gz

That applies to Windows partitions too. It's faster and more reliable *restore* than Volume Shadow Copy on identical hardware. You can also move the Old Dog onto somewhat newer hardware.

gunzip -c /mnt/sda1/hda.img.gz | dd of=/dev/hda conv=sync,noerror bs=64K

You will need to check the disk (fsck) before booting the restored partition.

If you have a say in your priorities, it's time to migrate the services off the Old Dinosaur onto another server with an LVM disk and vaguely modern distro.

Load More Comments
Slashdot Login

Need an Account?

Forgot your password?