Beta

Slashdot: News for Nerds

×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Ext3cow Versioning File System Released For 2.6

kdawson posted more than 7 years ago | from the have-a-cow-man dept.

Software 241

Zachary Peterson writes "Ext3cow, an open-source versioning file system based on ext3, has been released for the 2.6 Linux kernel. Ext3cow allows users to view their file system as it appeared at any point in time through a natural, time-shifting interface. This is can be very useful for revision control, intrusion detection, preventing data loss, and meeting the requirements of data retention legislation. See the link for kernel patches and details."

cancel ×

241 comments

Moo? (-1, Offtopic)

Bob54321 (911744) | more than 7 years ago | (#18954741)

09 F9 11 02 9D 74 E3 5B D8 41 56 C5 63 56 88 C0

Re:Moo? (-1, Offtopic)

Anonymous Coward | more than 7 years ago | (#18954793)

FCKGW-RHQQ2-YXRKT-TG6W-2B7Q8

So which is it? (3, Interesting)

EveryNickIsTaken (1054794) | more than 7 years ago | (#18954749)

Ext3cow, an open-source versioning file system based on ext3, has been released for the 2.6 Linux kernel. Ext3cow allows users to view their file system...
Well, is it the file system, or the file system manager?

Re:So which is it? (4, Informative)

Bob54321 (911744) | more than 7 years ago | (#18954789)

From the example screenshot [www.ext3cow.com] it appears it is a file system. You take a snapshot of your system at some point in time and it stores this data even when files change. Of course, with any file system it is important to have functionality that allows you to view the files as well...

Re:So which is it? (2)

hpavc (129350) | more than 7 years ago | (#18955005)

You don't take a snapshot, thats the big deal with it.

Re:So which is it? (0)

Anonymous Coward | more than 7 years ago | (#18955409)

here is another screenshot [freshmeat.net] .

Re:So which is it? (-1, Flamebait)

Anonymous Coward | more than 7 years ago | (#18954797)

Well, is it the file system, or the file system manager?


Are you stupid or manager?

Can't tell, its slashdotted (2, Informative)

tinkertim (918832) | more than 7 years ago | (#18955399)

Well, is it the file system, or the file system manager?

I can't tell, the site is experiencing the /. effect.

Mirror of the patch (I grabbed it when I saw this in the firehose) can be grabbed here [echoreply.us] until my server gets sluggish too.

in /usr/src type : patch -p1 linux-2.6.20.3-ext3cow.patch

The site said its not been tested with other kernel versions, but if you feel brave just s/linux-2\.6\.20\.3/your-version/g. Haven't tried it, but should work.

It wen't dark just around the time I was getting the docs and utilities.

Did anyone happen to grab the utilities? Got a link?

What a name (3, Funny)

Anonymous Coward | more than 7 years ago | (#18954761)

So is it EXT or is it just a FAT cow?

Re:What a name (1)

morgan_greywolf (835522) | more than 7 years ago | (#18955499)

Well, they were originally going to call it "Rosie O'Donell Versioning File System" but the name was too long and the acronym ROVFS just conjured images of that awful rap [youtube.com] by "MC Rove" at the awards dinner.

Overhead? (2, Interesting)

HateBreeder (656491) | more than 7 years ago | (#18954765)

Couldn't find real-world information about space and performance overhead.

Does it store many copies of each file? or only the differences between the old and the new version?

Re:Overhead? (1)

xumio (840966) | more than 7 years ago | (#18954799)

it is done by storing different blocks.

Re:Overhead? (3, Informative)

JoeD (12073) | more than 7 years ago | (#18954963)

Check the "Publications" link. The first one is an article in "ACM Transactions on Storage".

It's a bit dry, but there is an explanation of how it stores the versions, plus some performance benchmarks.

Re:Overhead? (2, Informative)

DaveCar (189300) | more than 7 years ago | (#18955087)


Couldn't read TFA (slashdotted), but I would *imagine* that 'cow' is copy on write and that it just uses new blocks for the changes - so only the differences, but not minimal differences.

Re:Overhead? (3, Informative)

anilg (961244) | more than 7 years ago | (#18955347)

COW has been present for a long time in ZFS [opensolaris.org] since Solaris 10. The overhead there is negligible and its quite stable. Last I heard, it was being ported to FUSE on linux. Upcoming in the next releases of FreeBSD and OSX. Wiki has a pretty good article [wikipedia.org] .

Re:Overhead? (0)

Anonymous Coward | more than 7 years ago | (#18955117)

Generally speaking (the site is slashdotted, so I can't read the specifics on this implementation)- when you write out files to the drive they spread out all over the place and each chunk has an i-node or information node that tells a little about what file it is from, and points to the next and last inodes, etc.. what happens when you snapshot is, it starts creating an alternate chain if things change. So say you have a file with 3 inodes and you create a snapshot- and then change the middle i-node, you're now taking 4 inodes worth of space. They're not completely separate copies. Now if you change one of the remaining two, you're only taking 5 inodes worth of space. If you end up changing all of them, only then do you essentially have a duplicate.

Re:Overhead? (1)

init100 (915886) | more than 7 years ago | (#18955407)

Generally speaking - when you write out files to the drive they spread out all over the place and each chunk has an i-node or information node that tells a little about what file it is from, and points to the next and last inodes,

Umm, no. At least for ext3 and similar filesystems, each file or directory corresponds to exactly one inode. The inode contains information about its owner, group, filetype (plain file, directory, symbolic link, FIFO, device file, etc), as well as permission information and extended attributes (such as for ACLs, SELinux security contexts, etc). It also contains pointers to blocklists, but each block does not have a separate inode.

CVS/Subversion replacement ? (5, Interesting)

BuR4N (512430) | more than 7 years ago | (#18954769)

This might be far fetched but how far off is it to use these filesystems as a revision control system replacement ?

Never tinkered with any of these filesystems, but wouldnt it be very comfortable for at least us developers to have a filesystem that worked something like Subversion. Just hook up something on the network and use it as the central code repository.

Re:CVS/Subversion replacement ? (1)

markov_chain (202465) | more than 7 years ago | (#18954835)

There are systems like Clearcase that do that, but the maintenance is a pain. (Not to mention not FOSS).

The C in CVS. (4, Informative)

SharpFang (651121) | more than 7 years ago | (#18954929)

Concurrent...

Sure you can "go back in time", but two users working on the same file at the same time would be a pain. Networking would require additional layers - even plain SAMBA/NFS, but still. Plus a bunch of userspace utilities as UI to access it easily.

It's not bad as a backend for such a system, just like MySQL is good as a backend for a website, but by itself it's pretty much worthless.

The FS in SVN? (1)

Zaharazod (951446) | more than 7 years ago | (#18955357)

What about the possibility of using a filesystem with built-in history storage as the backend for a Subversion repository? Client access would not change at all; assuming the underlying versioned FS were at all scalable though, I would imagine that increased performance and decreased complexity over things like BDB and FSFS might be well worth it.

Re:CVS/Subversion replacement ? (1)

Bacon Bits (926911) | more than 7 years ago | (#18955451)

It's a bad idea to use this kind of thing for version control, IMX. The documentation through TFA is very... sparse.

Q: What happens to old snapshots when the disk begins to fill up?
Q: How do I manage snapshots?
Q: Are snapshots atomic?
Q: What happens when a snapshot fails? What can cause a snapshot to fail?

Windows Server 2003's Shadow Copies works in much the same way, AFAICT, and MS goes out of their way to caution against using Shadow Copies as a replacement for backup or version control. I expect this kind of thing will be used for the same uses as Linux: it [probably] allows you to access locked files, and lets you recover accidentally deleted or modified files.

Re:CVS/Subversion replacement ? (1)

mlock (648386) | more than 7 years ago | (#18955693)

Did you look at FSVS http://freshmeat.net/projects/fsvs [freshmeat.net] ? Might do what you need - snapshots your system using a subversion repository. And allows for multiple users - like every subversion repository.

Excellent work but... (1)

Toffins (1069136) | more than 7 years ago | (#18954787)

ext3cow looks like excellent work, but being an externally maintained add-on to the kernel, one problem is that it will not be not synchronously available with new kernel releases. The latest available version is 2.6.20.3-ext3cow.patch which is behind the latest kernel. It would be better if it could be accepted and maintained inside the kernel.

Re:Excellent work but... (1)

oliverthered (187439) | more than 7 years ago | (#18954991)

troll
It would be better if the kernel has a stable ABI/API then the patch would work for any 2.6 kernel. /troll

Re:Excellent work but... (2, Insightful)

ajs318 (655362) | more than 7 years ago | (#18955127)

The Linux kernel will never, ever have a stable ABI. Compatibility across versions is guaranteed only at the Source Code level, not the binary level. This is 100% intentional, and the only people it really hurts are those who would deny us access to the Source Code. And they deserve it.

Re:Excellent work but... (1, Insightful)

Anonymous Coward | more than 7 years ago | (#18955449)

How about us who don't want to recompile everything whenever a new kernel release comes out? It is a freaking pain in the butt.

Re:Excellent work but... (3, Insightful)

oliverthered (187439) | more than 7 years ago | (#18955471)

Your wrong, it also hurts those people who write drivers that aren't accepted into the kernel. And it also hurts end users or haven't you noticed the lack of Linux drivers for a lot of hardware.

kick ass (0)

Anonymous Coward | more than 7 years ago | (#18954825)

this is brilliant if it works reliably with minimal overhead.

lets hope it gets picked up by the major distros

I could really use this - can I have a nautilus add on for it?

True undelete (4, Insightful)

ex-geek (847495) | more than 7 years ago | (#18954827)

Undelete, not half-assed, desktop based trash can implementations, is something I've always been missing on Linux. And yes, I generally know what I'm doing, but i'm also human and do make mistakes.

Re:True undelete (1, Interesting)

Anonymous Coward | more than 7 years ago | (#18954967)

Could always just hardlink everything in certain directory trees to a backup dir that is swept periodically. Essentially just "tagging" not actually erasing.
We used that a lot on our servers to avoid accidents.
Does not protect against overwriting contents of course, and doesn't do revisions - but hey, you said undelete - not all the features of this tool.

Oh, and that has advantage of not taking up any extra disc space (apart from the tiny bit for the hardlinks and their tree).

Re:True undelete (1)

19thNervousBreakdown (768619) | more than 7 years ago | (#18955077)

I've always wondered about this. Aren't files always eventually deleted with an unlink() call? What reason is there that unlink() can't be modified to instead move the link to a .Trash/ which is then scrounged when more space is needed? You could either auto-delete the oldest files, or if you wanted to not affect FS fragmentation delete a file whenever you needed to clobber one of its sectors. Sure, performance will drop when you get a drive full of deleted files that have to be cleared every time you write, but it wouldn't be that bad, and it would only really be useful on /etc /usr/share /home and maybe /var. It doesn't even seem like you'd need to modify the underlying FS, except maybe for the sector clobbering part, but I don't know the kernel internals well enough to say for sure.

Any kernel/FS hackers know this one? There's probably a real good reason that I'm missing, if not I've been thinking for a while that it'd be a good place to start my kernel hacking.

Re:True undelete (3, Informative)

xenocide2 (231786) | more than 7 years ago | (#18955373)

There's a couple reasons for it not being in the kernel. First, it misleads users who expect some degree of data security. The good news is that sort of person likely follows kernel patches to the FS and would likely be aware of the problem, possibly even writing a script that replaces rm with a real-rm.

The second argument is that it's better handled in user space, so the OS doesn't have to make that sort of policy. There's no reason you can't just alias rm to some .Trash, or configure your Desktop Environment to do so (GNOME does, for example). There's all sorts of things you have to decide that might not suit everyone. For example, if I delete a file on a USB drive, does it go in a .Trash storage in the USB drive, or do we copy it over to a main .Trash folder? Many people don't realize they have to empty the trash to reclaim space on their thumbdrive in GNOME.

The final argument I can come up with is security problems. We can't have one global .Trash bin in a multiuser system. And quotas. And permissions.

Reading historic archives of the LKML [iu.edu] suggests it's at least come up once. I guess Torvald's opinion is that anything that CAN go in the userspace SHOULD. Can't explain the webserver in kernel though. Perhaps that opinion has changed some time in the last 10 years?

Re:True undelete (1)

rahimobius (1087399) | more than 7 years ago | (#18955643)

This sound very much like the functionality of the Recycle Bin in windows. Sometimes I wish I had a recycle bin as well. (I have a nag for typeing rm when I mean mv). But I agree with xenocide2 that this is something for user space. It certainly is not trivial to implement if you want a secure multi-user environment. But for single-user developer-systems a aliased/hacked rm and a deamon could be of great help when you accidentally rm two files instead of replacing one with the other ;)

Re:True undelete (-1, Troll)

Anonymous Coward | more than 7 years ago | (#18955097)

Windows has been a halfed assed trashcan solution for a decade now.

guess what, if you had enough energy to type www.google.com the first 8 links are all great projects and ways to do exactly what you said for the keywords "linux undelete"

but then, typing those letter into a web browser is simply way to much effort.

you must be either management or incredibly lazy.

Re:True undelete (1)

ex-geek (847495) | more than 7 years ago | (#18955325)

guess what, if you had enough energy to type www.google.com the first 8 links are all great projects and ways to do exactly what you said for the keywords "linux undelete"

but then, typing those letter into a web browser is simply way to much effort.

These options went out of the window with the introduction of journaling in ext3. But even with ext2, they barely worked, especially for large files. They didn't work for me anyway.

you must be either management or incredibly lazy.

I guess you are the 18-year-old intern who believes to know everything then?

Full-System restore (1)

squizzar (1031726) | more than 7 years ago | (#18955537)

Doesn't this provide some kind of system restore as well? Assuming your entire system is on this FS, then any changes made, no matter how complex could be rolled back? Attempted to install some driver and broke everything? Just revert to the state before you made the changes... Of course, that means it's probably patented by Microsoft...

Re:True undelete (2, Interesting)

jonadab (583620) | more than 7 years ago | (#18955655)

Undelete isn't what makes this really cool, IMO. I don't generally delete stuff I still want, so that isn't really a big issue.

What I want, that a versioning filesystem can deliver, is the ability to revert a file back to an earlier version, after I've saved changes that turn out to be undesirable. This is a mistake I *do* make from time to time, often enough that I have been really hoping for a versioning filesystem in modern operating systems. This, to me, is a killer feature. I'm currently using FreeBSD, but this feature would by itself be enough to bring me back to using a Linux distribution, once it gets to the point of being included. Without it, once you save your changes and exit the application you can't go back. The past is lost. With a versioning filesystem, that's no longer true. I consider this to be *THE* feature for filesystems, far more important than things like journaling, much less performance tweaks. I have been wanting it ever since I saw the automatic versioning on OpenVMS, and I've been waiting, waiting, hoping, wondering why we don't have it in modern operating systems. I *want* this.

Hooray for repeating history! (1, Informative)

Anonymous Coward | more than 7 years ago | (#18954833)

VMS had a versioning filesystem 20+ years ago. You'd create a file and any time you'd edit it a number would be appended as the version:

eg foo.txt;1 foo.txt;2

Re:Hooray for repeating history! (2, Interesting)

Anonymous Coward | more than 7 years ago | (#18954877)

So because it was a good idea 20 years ago, it somehow isn't good that it's been implemented now? Sure, in an ideal world we'd all have been using versioned filesystems since the advent of VMS, but we havn't.

Actually a tell a lie; the ISO9660 spec. copies the VMS design and also allows files to have a version number, using the exact same scheme I.e. the version # is appended to the file following a semi-colon. So "FOO.BAR;1" is a valid ISO9660 filename.

Re:Hooray for repeating history! (1)

ajs318 (655362) | more than 7 years ago | (#18955563)

This is one of the things I missed when I moved from VMS on a VAX 11/750 {or it might have been a 780?} to MS-DOS on a '286. The commands were kind of similar between the two OSes, though DOS didn't have EVE -- which for me was the killer app. Version numbers, EVE, case-insensitivity and commands that were not "telegraphese" {there not being such a word as "txtspk" in those days when mobile phones were analogue, half-duplex [you squoze a switch in the handset when you wanted to speak, and let go to listen to what the other person was saying] devices operating on VHF and where you had to know roughly where the receiver was, as each transmitter mast had its own STD code} were among the reasons why I preferred VMS to Unix.

Great days. Of course, Unix won in the end ..... it does that because after using it for awhile, the things you hated it for at first stop bothering you, and eventually start making sense. Maybe it's got cat powers!

So (2, Interesting)

El Lobo (994537) | more than 7 years ago | (#18954841)

Is this like MS Shadow Copies or like Apple's Time Machine? Not trolling but just somebody enlight me, what is new here?

Re:So (2, Interesting)

psbrogna (611644) | more than 7 years ago | (#18954947)

I don't think it's supposed to be new (it's one of the things I miss most about VMS). It's outstanding functionality to have both for end users and sysgeeks/devs; built right into the file system level (ie. LOW). I prefer this approach to the hacks that other O/S's have implemented at a higher level. It's always harded to do something like this down deep at the roots rather than add it on as superficial gloss later. Granted, the end users don't usually notice or appreciate the diff but over time it keeps complex sys's like O/S's from becoming a teetering tower of shims and bolted on widgets.

Re:So (2, Informative)

TodMinuit (1026042) | more than 7 years ago | (#18955007)

It's more like Plan 9's Fossil [bell-labs.com] , only without the extremely cool Venti [bell-labs.com] .

Re:So (1)

delire (809063) | more than 7 years ago | (#18955111)

Not trolling but just somebody enlight me, what is new here?
It is for Linux. That is what is new. The two examples you give are for other operating systems. Raising your eyes to the top of the page will reveal this article is in the section "Linux". It's a bit tricky I know.

Psst: it's not a race.

Re:So (1)

init100 (915886) | more than 7 years ago | (#18955519)

It is for Linux. That is what is new.

Actually, snapshots with copy-on-write functionality is not new in Linux, but it hasn't been available in the filesystem itself. The Logical Volume Manager is able to create and use COW snapshots, and has been for some time.

Re:So (1)

delire (809063) | more than 7 years ago | (#18955653)

Actually, snapshots with copy-on-write functionality is not new in Linux, but it hasn't been available in the filesystem itself.
Precisely.

Re:So (1)

spitzak (4019) | more than 7 years ago | (#18955627)

There are already versions of this on Linux. One I use at work does it instead by making a new directory with the older versions of the files in it. To see yesterdays version of ./foo.txt you look at ./.snapshots/yesterday/foo.txt. This seems a lot nicer as you can more easily see when files are created and deleted as well as modified, though it is possible that technical limitations prevented this from using this naming scheme. I not sure what the system is, as it is a large file server, it is running Linux, though the filesystem is possibly proprietary.

I think the big deal is that this is a modification of the ext3 filesystem, rather than an all-new filesystem.

Well, congratulations. (1)

jimicus (737525) | more than 7 years ago | (#18954871)

Well done to all who worked on this patch. Guess this means you've almost caught up with OpenVMS [wikipedia.org] now, then? [throws another log of karma on the fire].

All joking aside, I never really liked VMS much. It was extremely good at being very verbose whilst being extremely bad at clear English.

Re:Well, congratulations. (1)

MichaelSmith (789609) | more than 7 years ago | (#18955541)

Well done to all who worked on this patch. Guess this means you've almost caught up with OpenVMS now, then?

In the sense that you had multiple versions of every file? Well yeah but it is on a per file basis rather than a per volume basis so you can't ask it to give you the entire volume (or even a directory) as it was at a particular time.

And I remember being caught by the 32000 version number limit, with a batch job which maintained a status file and purged the file after every run. The version number still incremented and when it hit the ceiling the job started crashing.

VMS file versions someone? (4, Interesting)

ntufar (712060) | more than 7 years ago | (#18954881)

It reminds me of VMS file versions.

In VMS if you had a file named article.txt, each time you modified and saved it in editor, a new version was created named article.txt;1 article.txt;2 article.txt;3 and so forth. So after a long session of edit and saves you could end up with a hundred copies of file in your directory. A lot of clutter in the directory but easy access to older versions of the files.

With Ext2cow you basically get the same functionality in a bit different way. By default you see only article.txt file. If you need to access a previous version of the file you need to specify a cryptic code like this: article.txt@10233745. A bit cumbersome but, hey, how often you access older version of your file anyways. Looks better than VMS' approach.

This filesystem seems like a perfect solution for me as I am writing my Ph.D thesis. Currently I take backup every day and name it thesis20070420.tar.bz2, thesis200070421.tar.bz2, thesis20070422.tar.bz2 and so forth in case I need to go back and see how it looked some time ago.

However, in my home directory I have a lot of large audio and video files that I would never want to be versioned. I wander if Ext3cow keeps extra copies of the files if I move them around, change file named but do not modify the content. Probably I would have to make a new partition and put my text files I am working on there under Ext3cow and leave my media files on ext3.

Re:VMS file versions someone? (1)

JohnFluxx (413620) | more than 7 years ago | (#18954917)

Why don't you use svn?

Re:VMS file versions someone? (1)

ntufar (712060) | more than 7 years ago | (#18955027)

Why don't you use svn?


Frankly, using SVN would be just too much effort for me: I may forget to commit the changes after a day of work; the files are binary .odt files; I need to teach my wife to use it.

I would rather have a Ext3cow to sit there and so everything for me in background. But, you gave me a good idea. I never thought of using SVN for anything besides source files...

Re:VMS file versions someone? (1)

Yaztromo (655250) | more than 7 years ago | (#18955275)

Frankly, using SVN would be just too much effort for me: I may forget to commit the changes after a day of work; the files are binary .odt files; I need to teach my wife to use it.
  1. Why not just extract your ODF file before committing? Other than graphic figures it's all text data inside a ZIP wrapper.
  2. Why is your wife working on your thesis?
  3. Why would you be any more likely to forget to run "svn commit" than you would be to tar your files up every day? And if you're likely to forget either, why not just do it as a cron job to run daily?

Yaz.

Re:VMS file versions someone? (1)

karmatic (776420) | more than 7 years ago | (#18955365)

So mount SVN over webdav, and turn on auto-versioning. Whenever you make a change, it gets committed as a new revision.

Re:VMS file versions someone? (1)

porl (932021) | more than 7 years ago | (#18955589)

wait, your wife is writing your thesis now?? :)

Re:VMS file versions someone? (1)

osgeek (239988) | more than 7 years ago | (#18955639)

Or better yet, SVK [bestpractical.com] .

Re:VMS file versions someone? (0)

Anonymous Coward | more than 7 years ago | (#18954975)

This filesystem seems like a perfect solution for me as I am writing my Ph.D thesis. Currently I take backup every day and name it thesis20070420.tar.bz2, thesis200070421.tar.bz2, thesis20070422.tar.bz2 and so forth in case I need to go back and see how it looked some time ago.

Sounds like a job for version control (e.g. CVS or Subversion)

Security, backups (2, Interesting)

Midnight Thunder (17205) | more than 7 years ago | (#18954997)

This solution certainly helps if you accidentally delete something or need to go back to an older version. SVN is one solution, but it is a bit more explicit, while solutions like this and Apple's Time Machine help avoid needing to remember to update your repository. It should be noted that this doesn't replace backups, since this does not protect against hard-drive corruption. I do have a few of questions though:
    - what are the security considerations here?
    - can you delete the existence of file, as to ensure that it is not easily found again?
    - what are the effects on hard-disc storage space, ie are there any estimates to how much extra storage is needed for this?

Re:VMS file versions someone? (1)

GauteL (29207) | more than 7 years ago | (#18955123)

"If you need to access a previous version of the file you need to specify a cryptic code like this: article.txt@10233745. A bit cumbersome but, hey, how often you access older version of your file anyways. Looks better than VMS' approach."

This is exactly what a graphical file manager should abstract away through concepts such as time machine [apple.com] .

This announcement is just Linux file systems starting to catch up with features from file systems such as ZFS. Very good news.

Re:VMS file versions someone? (1)

arivanov (12034) | more than 7 years ago | (#18955209)

Not quite.

This is more like NetApp and other high-end NAS and SAN systems where a facility like this is used for backup. The backup system looks at a snapshot taken at X:00 and backs it up at leisure while the users continue to read/write to the filesystem on top of it. Once the backup is complete you obsolete the checkpoint on which the backup was operating. As a result you have a true backup of the filesystem at point X, not something that spread from X to X+N hours.

This is a killer feature as far as any enterprise is concerned and one of the main reasons people at one point start looking at SANs and NAS-es.

Re:VMS file versions someone? (1)

cortana (588495) | more than 7 years ago | (#18955341)

Interesting... but tracking the revisions of a file by name has some limitations. What happens if I rename a file (also to another directory)? What happens if I rename a directory itself? Is the file metadata (owner, access permissions, modification times, extended attributes (including selinux labels, ACLs and user extended attributes)) versioned?

I guess some of this info is on the project's home page, which is down at the moment...

Can No One Else INNOVATE? (-1, Troll)

macs4all (973270) | more than 7 years ago | (#18954949)

It isn't bad enough that MacroSuck(tm) has to copy Apple at each and every turn, now LINUX Devs have to do it, too?

I mean, REALLY? Given that Unix has had the concept of file "versioning" since I don't know when (but a long-azz time!), and Linux has had what, like fifteen years to come up with something like this, I find the timing of this "revelation" highly suspicious.

This is a reverse-engineering of Apple's Time Machine, through and through.

I never thought I'd see the Penguin stoop to MacroSuck(tm)'s "R&D" tactics. Bleh!

Mod me "troll" if you must; but you KNOW I'm right...

Re:Can No One Else INNOVATE? (2, Insightful)

heffrey (229704) | more than 7 years ago | (#18954995)

What evidence do you have that this is reverse engineering?

Or do you mean that they are re-implementing Time Machine?

Re:Can No One Else INNOVATE? (-1, Troll)

macs4all (973270) | more than 7 years ago | (#18955079)

That is a semantic non-issue. Either way, it's IP theft.

Re:Can No One Else INNOVATE? (1)

cortana (588495) | more than 7 years ago | (#18955403)

"Theft" how?

And of what IP?

Make a specific allegation or stop trolling, please.

Re:Can No One Else INNOVATE? (1)

init100 (915886) | more than 7 years ago | (#18955565)

Do you actually mean that it is "IP theft" to take functionality from the Linux Logical Volume Manager and implement it per file in the file system instead? Hardly.

Re:Can No One Else INNOVATE? (1)

siride (974284) | more than 7 years ago | (#18955013)

So I guess since it's old news, they just shouldn't bother right? Nobody is claiming this as an innovation. It's just a good feature that's getting added finally. After all, as people have said, VMS had this 20 years ago. Even MS and Apple didn't add something like it until the past two years. I should add that this project started back in 2005, so it's been worked on about during the same time Apple's stuff has.

Re:Can No One Else INNOVATE? (1)

siride (974284) | more than 7 years ago | (#18955083)

Actually, the project started in 2003. So they were ahead of the game. Apple didn't demonstrate Time Machine until 2006.

Re:Can No One Else INNOVATE? (-1, Troll)

macs4all (973270) | more than 7 years ago | (#18955283)

And I guess this just hopped out of Apple's womb fully-formed at WWDC last year?

If the Ext3[Tu]cow (even the name has Apple roots!) project was started in 2003, and it debuted in mid-2007, then by that same development timetable, Apple should have started their project in 2002, right?

Or are Linux devs inherently slower than OS X devs? **ducks**

Re:Can No One Else INNOVATE? (2, Informative)

siride (974284) | more than 7 years ago | (#18955385)

Because it wasn't REVEALED until 2006, so even if Apple was working on it in 2002 (not likely, since Open Source projects generally have longer cycles than proprietary ones due to manpower issues), the ext3cow people would not have been aware of it. Why do you think people are stealing this from Apple? It's a good idea that follows logically from ideas found in revision control software such as Subversion and its predecessors. And as others have pointed out, VMS had this 20 years ago. The idea certainly has been in existence for longer than Apple has. The wikipedia article indicates that the TENEX operating system in the 60s first had versioning filesystems. In any case, Apple hardly invented it, Apple was hardly the first to use it, and Linux implementations have been released before Apple even demoed Time Machine. So, basically, you are 100% wrong.

Re:Can No One Else INNOVATE? (-1, Troll)

macs4all (973270) | more than 7 years ago | (#18955179)

Unless you work for Apple as an OS dev, you have zero idea how long this has been in the oven at Apple.

As for the "why bother?" point: It is rude to steal, period. And if Apple has been able to patent any of this (yes, I know, software patents suck. I'm right there with you, but...), this may just have to be ripped right back out of the Linux kernel. Then what? As I replied to another post: It's IP theft, direct or indirect; but IP theft just the same.

Or is Linux now reduced to following Apple's taillights, just like MacroSuck(tm)? Which was kinda my point, anyway...

On another note: Pardon my gaffe regarding VMS vs. Unix. I did not know that versioning was a VMS feature. I had learned it was a Unix-y thing.

Re:Can No One Else INNOVATE? (0)

Anonymous Coward | more than 7 years ago | (#18955317)

If you've ever studied b-tree filesystems, or filesystems in general, I'd call forking the inode chain rather obvious. I independantly thought up the same idea when I studied these only to find out of course that that is precisely how it is implemented.

Re:Can No One Else INNOVATE? (0)

Anonymous Coward | more than 7 years ago | (#18955383)

Prior art, dude. Apple didn't invent filesystem snapshots. And btw. even Windows XP had system restore in 2001, which is a rudimentary version of that. Btw. does ZFS sound familiar?

Such a simple idea as taking a snapshot shouldn't even be patentable, it is too general. And it's implementations can differ significantly.

Re:Can No One Else INNOVATE? (0, Troll)

macs4all (973270) | more than 7 years ago | (#18955483)

You're right. Snapshots shouldn't be patentable. Apple's Time Machine GUI SHOULD, however. It was the "non-obvious" icing on an old, moldy cake.

But don't ever breathe the term "System Restore" and "Time Machine" in the same post again. Comparing those two concepts as if they were equals is like comparing the Space Shuttle to a Model-T Ford: Yes, they are both "mechanized transportation", but...

Re:Can No One Else INNOVATE? (1)

siride (974284) | more than 7 years ago | (#18955535)

I don't think ext3cow has a nice Apple-like GUI (which personally make me want to vomit in terms of functionality and looks). It's a command line interface. So they clearly aren't even TRYING to steal anything from Apple.

Re:Can No One Else INNOVATE? (0)

Anonymous Coward | more than 7 years ago | (#18955527)

Look, I know you're a troll, and that you don't really use a Mac. But just for the education of others, you can dig around at fsf.org and find archives of talks that Richard Stallman gave about the GNU project way back when it started in the 80s, where he describes his idea for a versioning file system in Unix.

So versioning was always supposed to be part of the GNU system; as Linux is the kernel for GNU, this is just a new implementation of an idea from years and years ago. Next time you want to troll as a Mac user, try to learn some history.

Re:Can No One Else INNOVATE? (1)

macs4all (973270) | more than 7 years ago | (#18955633)

As I said, when does debate end and trolling begin?

I've been using Macs since they were called Lisas.

Next. Idiot.

And my original post acknowledged that file versioning isn't a new idea. Learn to read.

Re:Can No One Else INNOVATE? (1)

init100 (915886) | more than 7 years ago | (#18955621)

As I replied to another post: It's IP theft, direct or indirect; but IP theft just the same.

As I replied to another of your posts: It isn't.

Re:Can No One Else INNOVATE? (1)

SaturnNiGHTS (1074969) | more than 7 years ago | (#18955089)

this isn't a "recent project"...it was started in july of 2003...hardly stolen from microsoft or apple if older incarnations existed before they were developed. this is merely a version release. http://en.wikipedia.org/wiki/Ext3cow [wikipedia.org]

Re:Can No One Else INNOVATE? (3, Insightful)

beezly (197427) | more than 7 years ago | (#18955285)

Go away MacTroll...

Veritas VxFS has had this for years. Snapshotting has been implemented in the Linux LVM layer for ages. This is just another way to do it.

I don't know anything about the technical implementation of Vista Shadow Copies or Apple's Time Machine, but if it's anything like ZFS [wikipedia.org] then I'll be impressed. I believe there are rumours about the next release of OS X using ZFS (which was developed by Sun), but I'll believe it when I see it.

Re:Can No One Else INNOVATE? (1)

macs4all (973270) | more than 7 years ago | (#18955389)

I believe that they may use ZFS for OS X Server first, but it will be another filesystem supported in addition to HFS+.

As for me being a troll: When does debate end and trolling begin?

I was simply pointing out that this "smelled" much like Time Machine, albeit a clumsy, wholly unintuitive version of the underlying technology.

I know that Apple didn't invent the idea of file versioning. What they invented (as is their trademark), was the way to make that technology USEFUL and ACCESSIBLE.

Re:Can No One Else INNOVATE? (1)

CCFreak2K (930973) | more than 7 years ago | (#18955321)

According to Wikipedia [wikipedia.org] , Apple's Time Machine isn't even released yet. Maybe the person reverse-engineered Time Machine in the future and used the code to come back to the past...?

Also from Wikipedia [wikipedia.org] , Windows XP Professional includes a similar feature, although it doesn't do as much as the facility included in Windows Server 2003.

Are you paid to make shit up or what? Can I get a job there?

Re:Can No One Else INNOVATE? (1)

macs4all (973270) | more than 7 years ago | (#18955573)

Speaking of being paid to make shit up: Of COURSE Time Machine, which relies on OS X 10.5 (Leopard) isn't RELEASED yet; but it was DEMOED, fully-formed, at Apple's WWDC last summer.

Hence my "Reverse Engineering" comment in my original post.

So what was your point, again?

Interesting sponsor for ext3cow (1, Informative)

Anonymous Coward | more than 7 years ago | (#18954951)

The ext3cow project sponsor SecurityEvaluators is a rather interesting company in terms of some of their funding arrangements (sorry, cannot publish details here).
(2006) FBI Head Wants Strong Data Retention Rules [slashdot.org]
(2005) EU Approves Data Retention [slashdot.org]

oblig. (-1, Offtopic)

EveryNickIsTaken (1054794) | more than 7 years ago | (#18954981)

09-F9-11-02-9D-74-E3-5B-D8-41-56-C5-63-56-88-C0

Smells like dirvish (2, Interesting)

Zekat (596172) | more than 7 years ago | (#18954983)

This sounds like http://www.dirvish.org/ [dirvish.org] , which is nearly as nice as the automatic file snapshots done by the "Network Appliance" fileserver boxes I've used at the last 2 out of 3 workplaces.

Performance? (1, Interesting)

Anonymous Coward | more than 7 years ago | (#18955003)

So how does the mechanism affect performance? Aren't the files going to be very fragmented after a while? How long does it take to make those snapshots?

Re:Performance? (1)

Loconut1389 (455297) | more than 7 years ago | (#18955221)

almost no time, it just freezes the inode table and starts creating new inodes for changes instead of modifying old ones.

Ze First Step (ZFS) (1)

udippel (562132) | more than 7 years ago | (#18955145)

Done it, been there.
Guess, this is the first step to approach ZFS, which for some stupid licence reason doesn't seem to have an easy path into the Linux kernel.
ZFS does a few, actually a lot, more. But why not write a different solution, for a plurality of choice.
May the best win !

Re:Ze First Step (ZFS) (1)

OverlordQ (264228) | more than 7 years ago | (#18955267)

IIRC the main reason ZFS wont make it into the kernel is that a non-trivial amount of the filesystem kernel code would need to be re-written.

Re:Ze First Step (ZFS) (1)

anilg (961244) | more than 7 years ago | (#18955495)

That would be ZFS [opensolaris.org] , built from the scratch to avoid the problems faced by traditional filesystems. It is, in fact, a volume-manager+file system, with RAID built into it. Not to mention the very easy to use commands.

Am I missing something here?? (1)

MarsBar (6605) | more than 7 years ago | (#18955151)

Looks to me (having read the paper [znjp.com] ) like you need to manually snapshot a file every time you might want to (later) revert back to it.

Now I don't know about anyone else but that's not what I want from a system like this: I want a system that keeps transaction logs, essentially, so that I can literally ask for any file as it was at any time.

Re:Am I missing something here?? (1)

wild_berry (448019) | more than 7 years ago | (#18955577)

My suggested solution would be to write a daemon that monitors atime or file writes and snapshots each time. Or configure a mount option or kernel tunable that enables or disables automatic snapshots -- you don't want to fill your /var/log with multiple copies of files that only get appended...

Apple (0, Troll)

jlebrech (810586) | more than 7 years ago | (#18955235)

I felt a great disturbance in the Force, as if millions of Apple fanboys suddenly cried out in terror and were suddenly silenced. I fear something terrible^H^H^H^H^H has happened.

some background (4, Informative)

pikine (771084) | more than 7 years ago | (#18955281)

I'm answering questions that people posted so far altogether.

Is it a file system or a file manager?

It is a file system. You access old snapshot by appending '@timestamp' to your file name. You have to first instruct ext3cow to take a snapshot first before you can retrieve old copies, otherwise it simply behaves like ext3. It appears that snapshot is always performed on a directory and applies to all inodes (files and subdirectories) under it.

My complaint is its use of '@' to access snapshot. Why not use '?' and make it look like a url query? Better yet, use a special prefix '.snapshot/' like NetApp file servers.

Does it store many copies of each file? or only the differences between the old and the new version?

How far off is it to use these filesystems as a revision control system replacement?

ext3cow takes it's name from "copy on write," and it does this on the block level. When you modify a file, it appears to the file system that you're modifying a block of e.g. 4096 bytes. COW preserves the old block while constructing a new file using the blocks you modified plus the blocks you didn't modify.

You can think about it as block-level version control. However, when you save a file, most programs simply write a whole new file (I'm only aware of mailbox programs that try to append or modify in-place). Block-level copy on write is unlikely to buy you anything in practical use.

Does it provide undelete?

Only when you remember to make a snapshot of your whole directory. An hourly cron-job would do, maybe. There is always the possibility you delete a file before a snapshot is made.

Compatibility and copy on write... (1)

argent (18001) | more than 7 years ago | (#18955545)

My first thought was the same as yours, why not use the ".snapshot" prefix from netapp, so that scriopts and tools written for Netapp servers will continue to work.

Second, I have hundreds of mail folders saved in files with names like "user@example.com". Oops.

Block-level copy on write is unlikely to buy you anything in practical use.

For binary files (eg, databases) it will. And it's pretty cheap to implement... for a whole-file write operation where the file is first truncated the cost is the same as if they didn't bother to COW, and it keeps lots of complete copies of log files from being created.

No Data (1)

wild_berry (448019) | more than 7 years ago | (#18955455)

I can't see anything linked from the ext3cow.com site, save for the near-silent mailing lists. I'm tagging this 'slashdotted'. There's not even a huge amount on the Wayback Machine: http://web.archive.org/web/*/http://ext3cow.com [archive.org]

I guess that this is a fork of the ext3 code with Copy On Write functionality and userland tools to make snapshots and time-travel the snapshots. Wikipedia's article on Ext3cow [wikipedia.org] names Zachary Peterson, the submitter of the article, and links to an ACM Transactions on Storage paper at http://hssl.cs.jhu.edu/papers/peterson-tos05.pdf [jhu.edu] .

Linux is catching up to BSD... (1)

mi (197448) | more than 7 years ago | (#18955543)

BSD operating systems had filesystem snapshots [wikipedia.org] functionality for several years now... Linux is catching up — in a usual Linux way with patches, which one has to collect from all over...

Or am I misreading the write-up and this new ext3cow thingy is much more than that?

Load More Comments
Slashdot Account

Need an Account?

Forgot your password?

Don't worry, we never post anything without your permission.

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>
Create a Slashdot Account

Loading...