×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

OpenZFS Project Launches, Uniting ZFS Developers

Soulskill posted about 7 months ago | from the putting-the-band-together dept.

Data Storage 297

Damek writes "The OpenZFS project launched today, the truly open source successor to the ZFS project. ZFS is an advanced filesystem in active development for over a decade. Recent development has continued in the open, and OpenZFS is the new formal name for this community of developers, users, and companies improving, using, and building on ZFS. Founded by members of the Linux, FreeBSD, Mac OS X, and illumos communities, including Matt Ahrens, one of the two original authors of ZFS, the OpenZFS community brings together over a hundred software developers from these platforms."

cancel ×
This is a preview of your comment

No Comment Title Entered

Anonymous Coward 1 minute ago

No Comment Entered

297 comments

I'm addicted (4, Interesting)

MightyYar (622222) | about 7 months ago | (#44879395)

I love ZFS, if one can love a file system. Even for home use. It requires a little bit nicer hardware than a typical NAS, but the data integrity is worth it. I'm old enough to have been burned by random disk corruption, flaky disk controllers, and bad cables.

Re:I'm addicted (5, Funny)

Anonymous Coward | about 7 months ago | (#44879469)

I love ZFS too, but I'd fucking kill for and open ReiserFS...

Re:I'm addicted (4, Funny)

Virtucon (127420) | about 7 months ago | (#44879635)

I think that anything having to do with ReiserFS is a dead end.

Re:I'm addicted (0)

Anonymous Coward | about 7 months ago | (#44879979)

On the other hand, the project got a captured developer who has lot of time to do a lot of thinking for the next n number of years uninterrupted.

Re:I'm addicted (1)

Anonymous Coward | about 7 months ago | (#44880303)

On the other hand, the project got a captured developer who has lot of time to do a lot of thinking for the next n number of years uninterrupted.

Yeah, I'm sure more than any other filesystem it will be devoted to maximum security...

Re:I'm addicted (1)

mysidia (191772) | about 7 months ago | (#44879825)

I love ZFS too, but I'd fucking kill for and open ReiserFS...

I heard that the act of using ReiserFS might be a criminal offense.

Something about making oneself an accomplice after the fact... I don't know; it's a bit murky

Re:I'm addicted (0, Troll)

Anonymous Coward | about 7 months ago | (#44879641)

That's because you're using Linux, you dumb fucking retard.

Re: Data integrity (-1)

Anonymous Coward | about 7 months ago | (#44880107)

but the data integrity is worth it. I'm old enough to have been burned by random disk corruption, flaky disk controllers, and bad cables.

Except ZFS doesn't have any actual ECC and uses one to one data mirroring halving the disk space. I would kill for a filesystem that had real intelligent error correction tuned to a more realistic level of data corruption on a drive (say 100 bad clusters per GB).

Re: Data integrity (4, Informative)

MightyYar (622222) | about 7 months ago | (#44880403)

Not sure what you mean. You certainly can set up a mirrored pair (or triplet or quadruplet), but you can also set up what's referred to as raidz, where it stripes the redundancy across multiple disks. You can configure how much redundancy... 1, 2, or more disks if you like. You can also tell ZFS to keep multiple copies of blocks, and it will spread those copies out among the disks. You can set that policy per sub-volume (file system in zfs-speak), so that if you decide that some of your data deserves more redundancy, you can set up a folder that will keep 2 copies of everything, but leave all the other folders at 1 copy. It's super geeky. I've had it detect (and correct) corruption in a failing disk, detect corruption because of a flaky disk controller that would otherwise pretend to work fine, and detect corruption when a SATA cable came loose. Combined with the ECC RAM in the server, I feel more comfortable about the integrity of my data than I ever have. I've lost family photos before to random drive corruption, so I'm sensitive to this stuff :)

Re: Data integrity (4, Informative)

saleenS281 (859657) | about 7 months ago | (#44880625)

One point to be extremely clear on however - when you set copies = 2 on a folder level, it does NOT guarantee those copies end up on different physical spindles. Early on there were many people who lost files because they skipped RAID thinking that copies=X would protect their data. It is NOT meant as a means to protect against hardware failures.

all i want is BP-rewrite (5, Informative)

Anonymous Coward | about 7 months ago | (#44879423)

If this gets us BP-rewrite, the holy grail of ZFS i'll be a happy man.

For those who don't know what it is - BP-rewrite is block pointer rewrite, a feature promised for many years now but has never come. It's a lot like cold fusion is that its always X years away from us.

BP-rewrite would allow implementation of the following features
- Defrag
- Shrinking vdevs
- Removing vdevs from pools
- Evacuating data from a vdev (say you wanted to destroy you're old 10 disk vdev and add it back to the pool as a different numbered disk vdev)

Re:all i want is BP-rewrite (-1, Offtopic)

larry bagina (561269) | about 7 months ago | (#44879899)

- Ejaculating data from a vdev (say you wanted to destroy you're old 10 disk vdev and add it back to the pool as a different numbered disk vdev)

ok...

Re:all i want is BP-rewrite (1)

BitZtream (692029) | about 7 months ago | (#44880251)

And adding those things would make it ... well pretty much perfect.

Throw in background dedup without dedup tables hogging massive amounts of RAM too!

Re:all i want is BP-rewrite (0)

Anonymous Coward | about 7 months ago | (#44880259)

Does ZFS really not support defrag?

Re:all i want is BP-rewrite (1)

hedwards (940851) | about 7 months ago | (#44880575)

Why would ZFS need defrag support? UFS never had defrag support and the only times that ever became a problem was when the disk was running out of room. Which is bad for performance reasons anyways.

Re:all i want is BP-rewrite (5, Informative)

saleenS281 (859657) | about 7 months ago | (#44880643)

Because a COW filesystem will become fragmented over time simply by the way it works. As you delete files, you're only free-ing up small segments of contiguous blocks. Over time, this leads to fragmentation because writes are sometimes forced into non-optimal disk placement due to lack of free space. Granted - if you never fill the pool beyond 50%, it won't be a problem. For everyone else, it's a matter of when, not if it will become fragmented.

Re:all i want is BP-rewrite (2)

saleenS281 (859657) | about 7 months ago | (#44880653)

This will have little to no effect on the bp-rewrite situation. The only people with the skill and intimate knowledge of ZFS to do the bp-rewrite coding have stated both that it's extremely difficult, and that the companies they work for/with have no interest in implementing the feature/paying them to work on the problem. I haven't heard any of them volunteering their free time to focus on it either. This is more or less a marketing campaign IMO.

Still CDDL... (4, Informative)

volkerdi (9854) | about 7 months ago | (#44879437)

Oh well. I'd somehow hoped "truly open source" meant BSD license, or LGPL.

Re:Still CDDL... (0)

Anonymous Coward | about 7 months ago | (#44879695)

If you really expected Larry to turn over the crown jewels to the Stallman-Cult, you need to stop smoking crack.

Re:Still CDDL... (3, Informative)

larry bagina (561269) | about 7 months ago | (#44879905)

CDDL is basically LGPL on a per-file basis.

Re:Still CDDL... (2)

volkerdi (9854) | about 7 months ago | (#44880587)

CDDL is basically LGPL on a per-file basis.

Perhaps the intent of the licenses is similar, but there's more to a license than that. Unfortunately, being licensed under the CDDL causes a lot more license incompatibility restrictions than either the LGPL or BSD license do. If it were under one of those, there'd be hope for seeing it as an included filesystem in the Linux kernel. But since it's under the CDDL, that can't happen.

The developers are, of course, welcome to use whatever license they like. Just pointing out that the CDDL is *not* basically the LGPL under "per-file" or any other basis.

Patents? (3, Insightful)

Danathar (267989) | about 7 months ago | (#44879475)

Not to rain on anybody's parade,but will the commercial holders of ZFS allow this? Or will they unleash some unholy patent suit to keep it from happening?

Re:Patents? (2)

gagol (583737) | about 7 months ago | (#44879517)

Same licence, new name. Its more about uniting dev efforts under one roof.

Re:Patents? (0)

Anonymous Coward | about 7 months ago | (#44879609)

Just having a source tree (!!) and a contribution licensing policy would be a nice start, but no signs of those on their wiki.

Re:Patents? (4, Informative)

utkonos (2104836) | about 7 months ago | (#44879693)

FAQ much? There is no central source repository for OpenZFS. Each supported operating system has it's own repository. [open-zfs.org] The previous also has a link to the source tree for each of the supported projects under the umbrella.

Re:Patents? (0)

Anonymous Coward | about 7 months ago | (#44879787)

From the FAQ, on a novel kind of "joined forces":

There is no central source repository for OpenZFS. Each supported operating system has it's own repository.

Re:Patents? (0)

Anonymous Coward | about 7 months ago | (#44879699)

https://github.com/zfsonlinux

If you're successful, Larry will come a callin' (2, Funny)

YesIAmAScript (886271) | about 7 months ago | (#44879521)

As long as Oracle's patents are valid, can anyone seriously believe this will go anywhere?

His fleet of boats isn't going to pay for itself.

Re:If you're successful, Larry will come a callin' (0)

Anonymous Coward | about 7 months ago | (#44879591)

Collecting money from opensource-companys? Daryl McBride will turn in his grave if Larry is even stupid enough to try it...

Re:If you're successful, Larry will come a callin' (4, Funny)

stoploss (2842505) | about 7 months ago | (#44879791)

Collecting money from opensource-companys? Daryl McBride will turn in his grave if Larry is even stupid enough to try it...

Eh? I don't think that the Mormons bury their living, no matter how ghoulish are the corporations that they helm.

I'm afraid Daryl McBride will be quite operational when your friends' commits arrive...

Re:If you're successful, Larry will come a callin' (2)

Virtucon (127420) | about 7 months ago | (#44879675)

You mean that fleet of losing boats? Last time I checked it was 7-1 NZ with first to 9 winning.

Re:If you're successful, Larry will come a callin' (1)

Anonymous Coward | about 7 months ago | (#44879801)

As long as Oracle's patents are valid, can anyone seriously believe this will go anywhere?

hmm. i thought the cddl covered patents sufficiently? and wasn't that one of the reasons why sun chose to create cddl instead of using bsd or gpl or some other existing license in the first place?

Re:If you're successful, Larry will come a callin' (5, Informative)

Bengie (1121981) | about 7 months ago | (#44879837)

Oracle released ZFS under a BSD compatible license. Anyone is allowed to do whatever to the opensource code. Going forward, Oracle has not opened an code after v28, which is the last OpenSource version to be compatible with Oracle ZFS.

Fantastic news! (0)

Anonymous Coward | about 7 months ago | (#44879529)

I wish them best of luck. ZFS is the best FS out there.

ZFS for Windows? (0)

Anonymous Coward | about 7 months ago | (#44879549)

Does this mean we might finally get ZFS for Windows?

Re:ZFS for Windows? (-1, Troll)

guruevi (827432) | about 7 months ago | (#44879619)

Windows is the only OS out there that isn't fully POSIX compatible, so no, you can't develop on a broken platform.

Either way, why would you want to pay for licensing in order to run open source software?

Re:ZFS for Windows? (2)

Virtucon (127420) | about 7 months ago | (#44879779)

It doesn't have to be POSIX compliant to have it ported to it and it doesn't require somebody to pay for licensing. With the Features of ZFS one could argue that a port to at least Windows Server would be great and it would garnish quite a following from those who've had to put up with the way NTFS views disk volumes and storage. There are applications that run well on Windows, especially on the Server side of things so I wouldn't call it dead quite yet. Besides, with Server 2012 we now have Storage Spaces and ReFS which brings some ZFS features to the table, but it's nowhere as sophisticated ad ZFS. There's already been one attempt [google.com] but it doesn't appear to be actively maintained and it's read only. Oracle has software for Windows Server that interfaces to the Sun ZFS Storage Server (SAN) that works at the VSS level [oracle.com]. It's not exposing a ZFS filesystem to windows either, but ZFS is configurable in the SAN. That's a hefty uplift if you're already in deep with EMC or NetApp.

Re:ZFS for Windows? (5, Informative)

tlambert (566799) | about 7 months ago | (#44880189)

It doesn't have to be POSIX compliant to have it ported to it and it doesn't require somebody to pay for licensing. With the Features of ZFS one could argue that a port to at least Windows Server would be great and it would garnish quite a following from those who've had to put up with the way NTFS views disk volumes and storage.

Windows isn't a very friendly development platform for Open Source, starting with the licensing requirements for tools and distribution restrictions on binaries derived from those tools when using header files containing substantial code, or runtime libraries. Part of this is an intentional legal defense against WINE and CrossOver Office, and part of it is just scale management by limiting the support community requirements to "serious developers".

In addition, a lot of the installable filesystem and similar code, as well as a lot of the necessary VM internals (memory mapped files and paging/swapping from filesystems) are not adequately explained (i.e. they involve locking text regions with level 0 locks, which require a level 3 lock then a level 0 lock, and to do this to get the offsets on the physical media for the blocks in question. This used to not work on removable media in NT as of 4.0.1; not sure if it's supported yet, but it was the reason you couldn't install it in JAZZ drives or even regular hard drives in removable carriers.

Having developed a filesystem for Windows95 IFSMgr, and reverse engineered all this crap, and having done it again for NT3.51, I would not look forward to having to repeat the process for Windows 7 or Windows 8, which are the only useful versions to target for by the time the code ends up functional.

So unless someone wanted to seriously underwrite the effort (read: it's have to be done by Oracle, or by a startup who had a monetization strategy that Microsoft wouldn't preempt, like they did when my team, at a previous employer, ported UFS + Soft Updates to Windows 95, and they announced Longhorn-which-never-happened, and then put together a lawsuit about "deep reverse engineering" which would have precluded using it as a bootable FS... no thanks.

Re:ZFS for Windows? (2)

BitZtream (692029) | about 7 months ago | (#44880285)

Windows isn't a very friendly development platform for Open Source, starting with the licensing requirements for tools and distribution restrictions on binaries derived from those tools when using header files containing substantial code, or runtime libraries.

Well, the tools are free and there isn't a redistribution problem, never has been.

Now, you could argue that ZFS and Windows won't work unless MS does it because ZFS is the whole disk I/O stack rolled into one, and no driver is going to work with the kernel to allow the ZFS system to work in windows, but thats another story entirely. Theres no way to bypass the disk cache for instance, not in a way ZFS would be compatible with. ZFS must use its own cache, and directly access the raw devices, and provide the filesystem driver all rolled into one ... but spread all across the kernel, in order to get proper performance.

Could get pretty close with some good hacks though, such as FUSE.

Re: ZFS for Windows? (0)

Anonymous Coward | about 7 months ago | (#44879917)

why would you wanna argue with an idiot.

Re: ZFS for Windows? (0)

Anonymous Coward | about 7 months ago | (#44880509)

Playing with people is fun!

-Dogbert

Re:ZFS for Windows? (1)

_merlin (160982) | about 7 months ago | (#44879927)

WTF are you smoking. POSIX compatibility is easy to achieve, and you can get it on Windows by installing the optional SFU package. Too bad POSIX says nothing about file system driver interfaces - that's entirely kernel-dependent, and even varies between BSDs.

FINALLY. (1)

djhaskin987 (2147470) | about 7 months ago | (#44879559)

It's about time ZFS went open. I feel like the only reason btrfs got any traction was ZFS licensing issues.

Re:FINALLY. (1)

aliquis (678370) | about 7 months ago | (#44879649)

Don't cheer too soon.

It's just as open as before.

Btrfs got a reason to exist for the same reason, and isn't that quite good to? I don't know whatever it's as stable / safe to use yet but if not now then some day.

Re:FINALLY. (3, Informative)

Anonymous Coward | about 7 months ago | (#44879721)

Been using btrfs for several non-essential file systems. Working great so far, and have even done several successful bedup runs. Has worked great for minimizing disk usage on some Maven repositories with lots of duplicate files between Jenkins and Nexus. Maybe not tested enough for your server that you need to stay up all the time, but great for the home desktop (provided you're sane and are keeping backups, which you should be doing already anyway). The more testing it gets, the sooner it becomes "tested enough" for the needs-to-always-be-available server.

Re:FINALLY. (-1)

Bengie (1121981) | about 7 months ago | (#44879849)

Let us know how that goes on a multi petabyte array where you have mission critical data and all kinds of departments using it.

Re: FINALLY. (1, Insightful)

Anonymous Coward | about 7 months ago | (#44879971)

He did say home use you dumb obnoxious cunt

Re:FINALLY. (3, Informative)

Virtucon (127420) | about 7 months ago | (#44879827)

licensing or patent issues [sys-con.com]?
What you also forget is that Oracle was the leading proponent of BTRFS and yes it had to do with licensing and patents from Sun. Once they acquired Sun that all went out the window. If I were the CEO at Oracle I'd ask "Why two file systems that essentially do the same thing? One's mature and the other, not so much" That's why BTRFS still survives but now with less Oracle support. Wait, is that a bad thing?

Cool, but.. (-1)

Anonymous Coward | about 7 months ago | (#44879597)

ZFS is neat, but if something like dm-csum got into the mainline so that you could have data integrity on all filesystems, I wonder how many people would still care about ZFS. That's really the one ZFS feature that matters to me. Everything else is already handled with LVM and software RAID.

Re:Cool, but.. (4, Insightful)

Bengie (1121981) | about 7 months ago | (#44879861)

Everything else is already handled with LVM and software RAID.

You have a great sense of humor, keep it up.

Re:Cool, but.. (3, Informative)

smash (1351) | about 7 months ago | (#44879961)

That. Those who don't understand ZFS are condemned to reinvent it, poorly.

Re:Cool, but.. (0)

Anonymous Coward | about 7 months ago | (#44880557)

Geek card please.

Advatages of ZFS over BTRFS? (2, Insightful)

TheGoodNamesWereGone (1844118) | about 7 months ago | (#44879643)

I'm sure I'll be corrected if I'm wrong, but does it offer any advantage over BTRFS? I'm not trying to start a flame war; I'm honestly asking.

Re:Advatages of ZFS over BTRFS? (0)

Anonymous Coward | about 7 months ago | (#44879719)

ZFS is used in production around the world by fortune 500 companies. BTRFS is the future default-fs of linux, and will probably always be (the future part that is).

Re:Advatages of ZFS over BTRFS? (1)

Anonymous Coward | about 7 months ago | (#44879731)

Re:Advatages of ZFS over BTRFS? (2)

deek (22697) | about 7 months ago | (#44880429)

I'm playing around with btrfs at the moment, and I've spotted some inconsistencies in the document you mentioned.

* Subvolumes can be moved and renamed under btrfs. I do this on a daily basis.
* btrfs can do read-only snapshots. Mind you, it does have to be specified.
* As far as I can tell, "df" does work fine with btrfs. The document implies it does not.

I am still quite new to btrfs, so I'm learning much at the moment. There may be more points that I've missed.

It seems, though, your document is a bit out of date, and btrfs has improved since then.

Re:Advatages of ZFS over BTRFS? (-1)

Anonymous Coward | about 7 months ago | (#44879749)

No benefits I've heard of.

LVM - Need to remove a disk with data on it? Easy. Just pvmove, vgreduce, pvremove, done.

ZFS - Need to remove a disk with data on it? Fuck you.

Now, that ZFS bit may be outdated information or entirely false, but with the dearth of available documentation, I have no way to find out.

So, there's a drawback.

I'm sticking with LVM and btrfs.

Re:Advatages of ZFS over BTRFS? (2)

hedwards (940851) | about 7 months ago | (#44880605)

That's never been true, you always had the option of detaching it or outright deleting just one disk, you just had to make sure you did it in a careful manner so as not to delete things you didn't want to delete.

Also resizing a volume on a disk is a risky operation to engage in. If it's something that you really need to do, the correct way is to back up the data to a separate disk and restore it to a new volume. Resizing volumes is not exactly in keeping with the philosophy that led to ZFS being created.

Re:Advatages of ZFS over BTRFS? (1)

silas_moeckel (234313) | about 7 months ago | (#44879753)

btrfs is still considered experimental by the devs zfs is used in production.

Past that btrfs does not seem to support any sort of ssd caching wich is realy a requirement for any modern fs.

Re:Advatages of ZFS over BTRFS? (1)

TheGoodNamesWereGone (1844118) | about 7 months ago | (#44879929)

I'm using it right now on an SSD, but then I've turned off the swapfile and 'discard' in FSTAB, with no trouble. I'll admit I was put off initially by its experimental nature. This is the first time I've used it; prior to now I always used ext2, 3, or 4. Thanks to everyone who commented.

Re:Advatages of ZFS over BTRFS? (0)

Anonymous Coward | about 7 months ago | (#44880125)

No, he's referring to the features in ZFS that allow you to use an SSD as either a read cache or a write cache. Basically you can have a huge array of spinning metal disks and a couple of SSDs for read and write caching.

aka bcache + any filesystem you want (3, Informative)

raymorris (2726007) | about 7 months ago | (#44880239)

Using a small, fast SSD as a cache for large, slow disks can be awesome for some workloads, mostly servers with many concurrent users.

To do that with ANY filesystem, bcache is now part of the mainline kernel . dmcache does the same thing, and there is another one that Facebook uses.

Re: Advatages of ZFS over BTRFS? (1)

UnknownSoldier (67820) | about 7 months ago | (#44880541)

It is called ZIL - zero insertion log IIRC

Re: Advatages of ZFS over BTRFS? (1)

devman (1163205) | about 7 months ago | (#44880621)

ZFS Intent Log. It soaks up sync writes to a fast drive so they can be written out later to the slower drives.

Re: Advatages of ZFS over BTRFS? (0)

Anonymous Coward | about 7 months ago | (#44880651)

ZIL is ZFS Intent Log

Re:Advatages of ZFS over BTRFS? (5, Informative)

Vesvvi (1501135) | about 7 months ago | (#44879755)

I don't have any practical experience with BTRFS, but I use ZFS heavily at work.

The advantage of ZFS is that it's tested, and it just works. When I started with our first ZFS testbed, I abused that thing in scary ways trying to get it to fail: hotplugging RAID controller cards, etc. Nothing really scratched it. Over the years I've made additional bad decisions such as upgrading filesystem versions while in a degraded state, missing logs, etc, but nothing has ever caused me to lose data, ever.

The one negative to ZFS (if you can call it that) is that it makes you aware of inevitable failures (scrubs catch them). I'll lose about 1 or 2 files per year (out of many many terrabytes) just due to lousy luck, unless I store redundant high-level copies of data and/or metadata. Right now I use use stripes over many sets of mirrored drives, but it's not enough when you read or write huge quantities of data. I've ran the numbers and our losses are reasonable, but it's sobering to see the harsh reality that "good enough" efforts just aren't good enough for 100% at scale.

Re:Advatages of ZFS over BTRFS? (1)

Anonymous Coward | about 7 months ago | (#44879881)

So you never lose data ever... yet you lose one or two files per year...
Not that your losses are big... but if the file lost was 100GB of weather data... that could be called a BIG loss.

Re:Advatages of ZFS over BTRFS? (1)

mysidia (191772) | about 7 months ago | (#44879963)

Apparently "never lost data" must mean never lost an entire filesystem -- that's not my definition. Usually file loss is user error.

ZFS does support snapshots, and Nexenta / FreeNAS / etc have snapshot options, and replication options (zfs send | zfs recv) available, for sure.

It's a highly resilient filesystem, but owning and using a highly resilient filesystem is not a replacement for having the proper backups.

Re:Advatages of ZFS over BTRFS? (3, Informative)

Anonymous Coward | about 7 months ago | (#44880005)

You don't understand. ZFS didn't lose that data -- ZFS detected that the underlying disk drives lost that data. You can run ZFS in a highly redundant modes that allow it to reconstruct lost data, but it sounds like OP's redundancy is such that sufficient drives may lose bytes as to cause lost files.

Re:Advatages of ZFS over BTRFS? (0)

Anonymous Coward | about 7 months ago | (#44880493)

The point is you'd lose way more files using other file systems, hell you'd probably lose more just by silent bit flips alone, and no, those $8k raid cards won't protect you from silent file corruption.

If the parent poster consider those 2 files important, he'd probably just set copies=x so ZFS will fix it automatically, at the cost of some storage space.

Re:Advatages of ZFS over BTRFS? (1)

hedwards (940851) | about 7 months ago | (#44880615)

It means the files were lost from the filesystem, and he was notified and recovered them from the backups. Which is a hell of a lot more than what other filesystems would do for you. One of the benefits of ZFS is that it makes it a lot easier to monitor for bit rot.

Re:Advatages of ZFS over BTRFS? (0)

Anonymous Coward | about 7 months ago | (#44879931)

"nothing has ever caused me to lose data, ever."

"I'll lose about 1 or 2 files per year"

??? Reconcile, please?

Re:Advatages of ZFS over BTRFS? (2)

Bengie (1121981) | about 7 months ago | (#44880011)

"Unexpectedly" lost data. The things he's mentioned would have hosed other Fes' completely, but losing some data because his lack of redundancy is fine.

Re:Advatages of ZFS over BTRFS? (1)

mysidia (191772) | about 7 months ago | (#44879941)

The one negative to ZFS (if you can call it that) is that it makes you aware of inevitable failures (scrubs catch them). I'll lose about 1 or 2 files per year (out of many many terrabytes) just due to lousy luck

What? Interesting.... I never lost a file on ZFS... ever; and I was doing 12TB arrays, for VMDK storage; these were generally RAIDZ2 with 5 SATA disks, running ~50 VMs. Then in ~2011, concatenated mirrored sets of drives; large number of Ultra320 SCSI spindles in a direct attach SCSI Chasis --- which is the hardware I got, when requesting SAS direct-attach JBOD and SAS HBAs; the SAS hardware wasn't available for less than $1000 on eBay, so management had a "better" idea, even after being persuaded (with great difficulty) that moving forward with 3 Exchange server VMs >1000 users, 10 web servers, spam filters, countless other stuff, on a RAID5/RAIDZ2 array with 6x 2TB SATA drives was not a great idea --- thankfully all that parallel SCSI junk has since been scrapped.

Which was done on NFS and also on iSCSI. The average virtual disk is approximately 100gb.

No matter how you put it "losing 1 or 2 files" of 100gb, 200gb, or 1TB in size is a big deal.

Choosing the storage solution that loses a VMDK file or VMX file; if the wrong VMDK/VMX file..... equates to an entire server disappearing into oblivion: if a mission-critical VM. Can be very harmful to continued employment.....

Re:Advatages of ZFS over BTRFS? (1)

Bengie (1121981) | about 7 months ago | (#44880021)

It sounds like he disabled/reduced ZFS's default to keep extra copies of meta-data.

Re:Advatages of ZFS over BTRFS? (1)

mysidia (191772) | about 7 months ago | (#44880131)

It sounds like he disabled/reduced ZFS's default to keep extra copies of meta-data.

That would seem to require altering the source code. At least in the Solaris X86 ZFS implementation; there is no zpool or zfs dataset option to turn off metadata redundancy.... of course it would be a bad idea.

Re:Advatages of ZFS over BTRFS? (2)

BitZtream (692029) | about 7 months ago | (#44880419)

I corrupted some files by the following:

This is a home setup, all parts are generic cheapo desktop grade components, except slightly upgraded rocket raid cards in dumb mode for additional sata ports:

4 HDDs, 2 vdevs that 2 drive mirrors (RAID 1+0 with 4 drives essentially)

1 drive in a 2 drive mirror fails, no hot spare.
When inserting a replacement drive for the failed drive, the SATA cable to the remaining drive in the mirror was jiggled and the controller considered it disconnected.

The pool instantly went offline. When the drive reconnected, and the new drive was added to the mirror, during the resilvering process, 2 files were detected with invalid checksums. There were files that were being written at that moment the VDEV was yanked out from under ZFS.

Scrub found additional correctable errors and repaired them, but the files it marked as irreparable were clearly irreparable.

Simply deleting the corrupted files cleared the pool errors after the next scrub. Since I was copying those files anyway when the failure occurred, I just recopied them and nothing was actually lost .

Of course, I really can't expect anything else to have happened. I'm EXTREMELY grateful that it didn't take the entire pool down, so while there was 'data loss' it performed exactly as I would have hoped it to.

You can't expect much better than what it did considering an entire vddv (both drives in the mirror) went off line as data was being written to them.

Redundant metadata can't solve the problem of large amounts of the HDD becoming unreadable, which given enough terabytes is going to happen, and possibly often when you get into big data sets (think LHC size data sets). You can of course, zfs set copies=5 on the pool, or whatever number of copies you want to get additional protection, but then you might as well just put more drives in the same vdev and benefit from increased read speeds. Copies=1 by default, making it entirely possible to lose data.

Re:Advatages of ZFS over BTRFS? (0)

Anonymous Coward | about 7 months ago | (#44880165)

If ZFS is giving you checksum errors, you probably have some bad hardware in there. Drives themselves have their own error correction mechanisms, so that means either you have a bad data path (i.e. a bad or too long SATA or SAS cable) or you have something else wrong like a bad PSU, flaky mobo, bad RAM, etc...

Re:Advatages of ZFS over BTRFS? (0)

Anonymous Coward | about 7 months ago | (#44880457)

That.

We've been running large RaidZ3 arrays replicated across multiple racks, it takes 4 disk failures to kill a vdev, and the exact same thing has to happen to 3 different servers to make recovery impossible, with another cold backup server the thing just won't go down.

Fuck expensive raid cards, ZFS is the future.

Re:Advatages of ZFS over BTRFS? (2)

mysidia (191772) | about 7 months ago | (#44879879)

I'm sure I'll be corrected if I'm wrong, but does it offer any advantage over BTRFS? I'm not trying to start a flame war; I'm honestly asking.

BTRFS is still highly experimental. I had production ZFS systems back in 2008. A mature ZFS implementation is a lot less likely to lose your data with filesystem code at fault (assuming you choose appropriate hardware and appropriate RAIDZ levels with redundancy).

Re:Advatages of ZFS over BTRFS? (-1)

Anonymous Coward | about 7 months ago | (#44879965)

None. It was built in Larry's house, you'd be a fool to support it. BTRFS is actually no-strings-attached GPL (Yes, being able to re-close the source is a restriction I find unreasonable), and it'll be as good / better than ZFS soon.

That's the cost of freedom, ZFS may be a bit more shiny now but it's propping up a monster. Let it die.

Re:Advatages of ZFS over BTRFS? (1)

Anonymous Coward | about 7 months ago | (#44880147)

None. It was built in Larry's house, you'd be a fool to support it. BTRFS is actually no-strings-attached GPL (Yes, being able to re-close the source is a restriction I find unreasonable), and it'll be as good / better than ZFS soon.

That's the cost of freedom, ZFS may be a bit more shiny now but it's propping up a monster. Let it die.

"No-strings-attached GPL" is clearly an oxymoron. But besides pointless debates over licensing, we have a bulletproof file system in ZFS now (er, well, we've had it for 5+ years actually). BTRFS may catch up some day, but wake me up when it's actually ready for critical systems deployment.

Re:Advatages of ZFS over BTRFS? (0)

TopSpin (753) | about 7 months ago | (#44880635)

and it'll be as good / better than ZFS soon

No. Sorry.

There hasn't been a commit to the official BTRFS tree [kernel.org] in over two months. There have only been five distinct contributors during the entire third quarter. The second quarter saw only 70 commits.

That pace is way too slow for a file system with so many 'to be implemented' features. While not dead, at this rate BTRFS will never surpass ZFS in any notable way.

I'm sincerely sorry about that. Linux contributors just aren't getting it done wrt BTRFS, and that's a crying shame; other operating systems should look on in envy at marvelous Linux file systems.

And yes, I should be in there plugging away at it. So should you. But we're not.

That's not Oracle's fault, either. People just don't care enough to put in the effort. We're just here griping about Oracle and the ZFS license issue and poasting about BTRFS being the answer, waiting for someone to do all that brutally hard work.

We're deluding ourselves.

Re:Advatages of ZFS over BTRFS? (0)

Anonymous Coward | about 7 months ago | (#44880103)

btrfs advantage = support on multiple architectures (nearly anything that Linux will run on), and small memory/low-end cpu. In-tree, on linux, so likely supported even with a reasonably current distribution kernel.

zfs advantage = stability, additional features, support on several kernels.

I've been running btrfs for a few years on non-essential data (upstream mirrors might get annoyed, but any data I store on btrfs can just be re-rsync'd from upstream if a failure). I've had one very bad fs corruption issue in btrfs a couple years ago, but nothing else. I use the snapshot feature extensively, but not much else (considered the offline dedup hack, but that didn't get much love). But, even with only this single failure, I still wouldn't trust it for anything that I mind losing.

As for ssd caching (mentioned in another reply), I have never done it, but there is a filesystem agnostic ssd caching layer in newer linux kernels (probably too new for any stable distribution kernels; you'll probably have to compile your own). It should work for any fs, but not quite the same thing as l2arc in zfs (more like the local disk cache for NFS that Solaris had all those years ago).

If you have lots of hardware to throw at it and care about your data, ZFS wins (even on Linux). If you are running a smaller system e.g., a 32bit arm board with 1G +/- ram, and can afford to lose the data, btrfs needs testing.

Re:Advatages of ZFS over BTRFS? (2)

sl3xd (111641) | about 7 months ago | (#44880167)

BTRFS has a large number of features that are still in the "being implemented", or "planning" stages. In contrast, those features are already present, well tested, and in production for half a decade on ZFS. Many touted "future" features (such as encryption) of BTRFS are documented as "maybe in the future, if the planets are right, we'll implement this. But not anytime soon"

Comparing the two is like making up an imaginary timeline where ReiserFS 3 was 4-5 years old and in wide deployment while ext2 was being developed, with plans to implement journaling (ie. ext3) and extents (ie. ext4) still in the "TODO" stage.

My own BTRFS system is appallingly slow compared to running ext4 on the same hardware; in contrast zfsonlinux is amazing.

What is ZFS and why should I care? (0)

JDOHERTY (323140) | about 7 months ago | (#44879991)

It would be nice if the posted article or even the OpenZFS project home page provided some sort of summary of the benefits and objectives of this effort.

What's the difference? (0)

Anonymous Coward | about 7 months ago | (#44880051)

What is the difference between this ZFS and Oracle's ZFS? If I have to patch the kernel either way, why should I choose one or the other?

Re:What's the difference? (1)

Bengie (1121981) | about 7 months ago | (#44880185)

Oracle stopped supplying code after v28. The open source community decided it's been too long and ZFS in open source needs new features. So they're parting ways with being compatible with Solaris ZFS. Up until now, Open Source ZFS was able to be mounted by Sun and visa versa, but only up to v28.

Re:What's the difference? (1)

Marillion (33728) | about 7 months ago | (#44880347)

The feature I'm waiting for is the v30 feature for filesystem encryption. Full disk encryption is the current fad, but selective encryption just seems cleaner. I see no point of encrypting operating system files only to unencrypt them every time you boot.

Re:What's the difference? (1)

hedwards (940851) | about 7 months ago | (#44880649)

Selective encryption means that you have to be incredibly careful that sensitive data never hits a non-encrypted portion of the disk. So, I'd say that the full disk encryption is the cleaner option.

Still no encryption... *sigh* (2)

the_B0fh (208483) | about 7 months ago | (#44880315)

I wish they had encryption... *sigh*

No, I don't want workarounds, I want it to be built in to ZFS like in Solaris 11.

Good (0)

Anonymous Coward | about 7 months ago | (#44880405)

Hope they'll implement encryption feature ASAP, fuck Oracle for locking in the good stuff from v29 and on.

I am sick and tired of these clueless greedy fucks who kept fighting against the tide at the cost of the community, it's not like they can keep it out of the open for ever, eventually someone will make something better for free, all they did was piss people off.

Load More Comments
Slashdot Account

Need an Account?

Forgot your password?

Don't worry, we never post anything without your permission.

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>
Sign up for Slashdot Newsletters
Create a Slashdot Account

Loading...