Beta

×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Running ZFS Natively On Linux Slower Than Btrfs

kdawson posted more than 3 years ago | from the early-days dept.

Sun Microsystems 235

An anonymous reader writes "It's been known that ZFS is coming to Linux in the form of a native kernel module done by the Lawrence Livermore National Laboratory and KQ Infotech. The ZFS module is still in closed testing on KQ infotech's side (but LLNL's ZFS code is publicly available), and now Phoronix has tried out the ZFS file-system on Linux and carried out some tests. ZFS on Linux via this native module is much faster than using ZFS-FUSE, but the Solaris file-system in most areas is not nearly as fast as EXT4, Btrfs, or XFS."

cancel ×

235 comments

Sorry! There are no comments related to the filter you selected.

First post! (5, Funny)

halfaperson (1885704) | more than 3 years ago | (#34306646)

Using BTRFS :)

They Why ZFS? (1)

BoRegardless (721219) | more than 3 years ago | (#34306684)

If 3 other file systems are "faster", then is ZFS somehow "better"?

Re:They Why ZFS? (5, Insightful)

klingens (147173) | more than 3 years ago | (#34306714)

ext2 is faster than ext3, simply because it does less. ZFS has many, many features most other FS don't have but they do come at a price.

Re:They Why ZFS? (1, Insightful)

Anonymous Coward | more than 3 years ago | (#34306772)

Sooo, are any of those features I'd particularly care about?
Ext4 seems to do all my simple needs (and those of my services) require.

Re:They Why ZFS? (0, Troll)

Anonymous Coward | more than 3 years ago | (#34306822)

do you require random corruption?

because ext4 it does comes in a fast variant, but that means the random complete disk erasing failure.

Re:They Why ZFS? (0)

Anonymous Coward | more than 3 years ago | (#34307006)

Huh. Referring to:

http://en.wikipedia.org/wiki/Ext4#Delayed_allocation_and_potential_data_loss [wikipedia.org] ?

If so, that seems a rather biased and inaccurate way to describe it...
"Other Linux file systems like XFS have never offered ext3-like behavior."

And certainy is not random.

Re:They Why ZFS? (0, Flamebait)

HogGeek (456673) | more than 3 years ago | (#34306826)

I'm going to answer your question with a question.

  What exactly is it you care about?

Since ext4 meets 'your' needs - why continue developeing anything. Close up shop, we're done here...

Re:They Why ZFS? (0)

Anonymous Coward | more than 3 years ago | (#34306860)

Ok, let me rephrase.
Which of the ZFS features most impact its performance? Since OP says features come at a price.

And, are they shiny and cool? Some details at least...

Re:They Why ZFS? (5, Informative)

daha (1699052) | more than 3 years ago | (#34307054)

Which of the ZFS features most impact its performance?

Compression enabled by default can't help (available in btrfs).

Checksum for all blocks probably doesn't help, but definitely helps detect corrupt data/corruption (available in btrfs).

Forcing any file that requires more than a single block to use a tree of block pointers probably doesn't help. The dnode only has one block pointer and the block pointer can only point to a single block (no extents). On the plus side, the block size can vary between 512 bytes and 64 KiB per object, so slack space is kept low. If more than a single block is necessary it creates a tree of block pointers. Each block pointer is 128 bytes in size, so the tree can get deep fairly quick.

Three copies of almost all file system structures (such as inodes, but called dnodes in ZFS) by default can't help (which are compressed of course).

Re:They Why ZFS? (1)

DJProtoss (589443) | more than 3 years ago | (#34307964)

don't forget the intent log -being able to recover from failed power issues is great, but unless you use a separate flash zil device, it ain't quick ('course, that assumes they are using sync'd writes).

Re:They Why ZFS? (5, Insightful)

Anonymous Coward | more than 3 years ago | (#34307076)

Snapshots.
And I don't just mean any snapshots.
Done right, like in ZFS, they are fast.
Faster than BSD's UFS snapshots, faster than using LVM's fs-agnostic snapshots. For people who need them, they're great.

Re:They Why ZFS? (0)

Anonymous Coward | more than 3 years ago | (#34308312)

built ARC caching too.. also scaling data sets into infinity and beyond. There's no need to use a partition ever again. In a year or 2 when solid state drives drop in price, the 'performance' difference between ZFS and other filesystems will not even be noticed. Deduplication. And more.

ZFS > *

Re:They Why ZFS? (3, Insightful)

Cwix (1671282) | more than 3 years ago | (#34306888)

What features does ZFS have that ext4 doesnt? Its a simple question, but you had to act like an ass. Good job.

If I have a bicycle that I ride everywhere, and never seen nor heard of a car. I would not know what a car could do for me, would I? SO if someone comes along and says, Hey cars are cool, they are just a little more expensive. I would ask something like.. What features does a car have over a bicycle.

Re:They Why ZFS? (1)

bigredradio (631970) | more than 3 years ago | (#34307744)

ZFS is not only a filesystem but also contains volume management. It's a filesystem that could replace LVM.

Re:They Why ZFS? (3, Insightful)

Jeff DeMaagd (2015) | more than 3 years ago | (#34307776)

Thanks for replying like a jerk, that really helps us all out. Nobody is going to simply transition to a new way of doing things just because it's new, they need to know what they'll get from the new way that makes the transition worthwhile.

Checksums - 1 feature ZFS has that Ext4 doesn't (4, Informative)

yup2000 (182755) | more than 3 years ago | (#34307088)

hmmm, well the most obvious feature that ZFS has that Ext4 does not is check summing.

That feature is one reason why ZFS is better (it will tell you if your disk is going bad, and if you have a raid setup, it will go get the good data for you). However, this is also one reason why ZFS is slower... it spends time making sure your data is safe and that it always gives you the correct bits from your disk.

That single feature is why I run FreeBSD (looking forward to kFreeBSD/debian!) on my file server in a mirrored raid configuration. Yes, it is "slower", but I still pull data off that server at over 50MB/sec on my home gigabit lan! The specs on that server aren't great either... 2GB ram, and an old 1.6GHZ single core sempron.

Re:Checksums - 1 feature ZFS has that Ext4 doesn't (0)

Anonymous Coward | more than 3 years ago | (#34307160)

Hm. daha, in a response just a little ways over in the comment tree, said btrfs has this as well.
He also gave some other potential problem areas.

I wonder if this btrfs feature was enabled in the test mentioned in the topic.

Re:Checksums - 1 feature ZFS has that Ext4 doesn't (1)

Bengie (1121981) | more than 3 years ago | (#34308096)

I want to make a file server with FreeBSD 9.0 and ZFS, but I want full gigabit speeds. After I got my new Win7 machine, I can SMB copy 114MB/sec between my computer and my wife's with only 1.5% cpu. I'm addicted to speed. 10gb card/switches have been coming down in price to. Looking into those for a File Server.

By the time I get this going(prob in 2 years), DDR4 will be out, 22nm 4core low power server cpus will be out. ZFS + SSD + lots of memory = ftw.

Re:They Why ZFS? (2, Interesting)

afidel (530433) | more than 3 years ago | (#34307272)

L2ARC is a HUGE performance improvement for many workloads, it essentially allows you to use faster disks to cache the most frequently used data. If they had combined the SSD and the 7200 RPM SATA drive and benchmarked a real world workload the ZFS implementation would have probably stomped the others because it would have used the SSD for the 'hot' data, the best you can do with btrfs is to place the metadata on the SSD.

Re:They Why ZFS? (1, Informative)

Anonymous Coward | more than 3 years ago | (#34307694)

L2ARC is a HUGE performance improvement for many workloads, it essentially allows you to use faster disks to cache the most frequently used data. If they had combined the SSD and the 7200 RPM SATA drive and benchmarked a real world workload the ZFS implementation would have probably stomped the others because it would have used the SSD for the 'hot' data, the best you can do with btrfs is to place the metadata on the SSD.

L2ARC is just another cache. The ultimate IO limits of the filesystem are still set by limitations of the final backing store.

So if you're moving lots and lots of data, the L2ARC is pretty useless.

Set yourself up a ZFS file system, then start benchmarking it. If you're running on Solaris, run something like "iostat -sndxz 1" so you can see actual IO to your physical LUNs every second. Under heavy write load, you'll see ZFS go for extended periods without writing anything, then it'll hang your box badly as it flushes to disk. That's bad for two reasons - the relatively long periods of time ZFS isn't writing are IO opportunities lost, and the hanging of the box is horrible.

ZFS's IO pattern gives away available bandwidth, and then ZFS hammers your system to its knees.

Re:They Why ZFS? (1)

clang_jangle (975789) | more than 3 years ago | (#34308224)

ZFS's IO pattern gives away available bandwidth, and then ZFS hammers your system to its knees.

That's where the tuning foo comes in handy. But you're right, it takes a pretty high level of competence to successfully run ZFS, at least on FreeBSD.

Re:They Why ZFS? (1)

afidel (530433) | more than 3 years ago | (#34308460)

Like I said, under most real world workloads the L2ARC will have significant impact. There will always be edge cases and artificial benchmarks that can swamp any cache, but I run a midsized enterprise on an array with only 8GB of cache and it absorbs 99.5% of the write workload and a fair percentage of the non-database read workload so a 64GB+ SSD would just be that much better. With L2ARC you can achieve a high 95-99% IOPS watermark with a small dollar investment, and because the cache is servicing most of the hot data it also means that the available backend IOPS are more available to service the hard cases.

Let me put it this way. (1, Flamebait)

jotaeleemeese (303437) | more than 3 years ago | (#34307778)

If you need to ask most likely you would not care about them.

Mentioning on the same breath ZFS and Ext4 says it all about your expertise on this field really.

Re:They Why ZFS? (0)

Anonymous Coward | more than 3 years ago | (#34306948)

ZFS has many, many features most other FS don't have

Many of which are unnecessary within the filesystem itself on any well designed and modern OS which has proper separation of various functional layers. On Linux LVM2 and the md layers do the job of ZFS zones and RAIDZ, and they do it without jamming everything into a giant blob of code.

Re:They Why ZFS? (4, Insightful)

Rakshasa Taisab (244699) | more than 3 years ago | (#34306732)

I can write the fastest file system around, assuming you don't put much weight on the whole 'being able to read the data back' thingie.

Re:They Why ZFS? (2, Funny)

Anonymous Coward | more than 3 years ago | (#34307520)

I can write the fastest file system around, assuming you don't put much weight on the whole 'being able to read the data back' thingie.

You mean "> /dev/null"?

Re:They Why ZFS? (2, Funny)

guruevi (827432) | more than 3 years ago | (#34308368)

Try a RAID-10 array of /dev/null's - it's even faster.

Re:They Why ZFS? (3, Insightful)

outZider (165286) | more than 3 years ago | (#34306838)

So, because ext3 implementations on other OSes are slow, that means ext3 is slow? Got it.

Try running ZFS on FreeBSD, or better yet, on the original OS: Solaris.

Re:They Why ZFS? (1)

hedwards (940851) | more than 3 years ago | (#34307012)

Indeed. The main reason to use ZFS over the other ones, even in cases where the features are the same is that ZFS is more widely available. Admittedly, it's far from universal, but right now it's officially supported in more than one OS. I'm not aware of a filesystem that provides similar functionality to ZFS which is more widely available.

And it's hardly fair to compare a filesystem that's being run in such a convoluted way to one that's able to be much more tightly integrated, especially considering that it's a licensing issue not a technical one that mandates the approach.

And yes, I've personally used ZFS on both FreeBSD and Solaris, and I haven't had any complaints about speed. Resource utilization yes, but that's been greatly improved.

I'm sure that Hammer and Btrfs are both great filesystems, but like EXT4FS, they aren't particularly useful in cross platform computing at present, and while servers aren't going to be doing that, it is something to consider when you've got massive arrays of disks that if you can't take it directly that you'll be stuck with some sort of really annoying migration process for the disks as well as the rest of it.

Re:They Why ZFS? (4, Interesting)

tlhIngan (30335) | more than 3 years ago | (#34307118)

The main reason to use ZFS over the other ones, even in cases where the features are the same is that ZFS is more widely available. Admittedly, it's far from universal, but right now it's officially supported in more than one OS. I'm not aware of a filesystem that provides similar functionality to ZFS which is more widely available.

Actually, I've run into this problem, not with ZFS (haven't used it), but with other filesystems, on Linux only. It seems not all filesystems are truly endian-aware, so moving a USB disk created on a big-endian system and moving it to a little endian system results in a non-working filesystem. Had to actually go and use that system to mount the disk.

Somewhat annoying if you want to pull a RAID array our of a Linux-running big-endian system in the hopes that you can recover the data... only to find out it was using XFS or other non-endian-friendly FS and basically not be able to get at the data...

Re:They Why ZFS? (1)

icebraining (1313345) | more than 3 years ago | (#34307106)

Yeah, especially since:

The SPL packages provide the Solaris Porting Layer modules for emulating some Solaris primitives in the Linux kernel, as such, this ZFS implementation is not ported to purely take advantage of the Linux kernel design.

Re:They Why ZFS? (1)

windcask (1795642) | more than 3 years ago | (#34306882)

Does RAID-Z ring a bell?

Re:They Why ZFS? (5, Interesting)

caseih (160668) | more than 3 years ago | (#34306946)

ZFS is, until BtrFS hits truly enterprise stable, the only FS for large disks, in my opinion. I currently run ZFS on about 10 TB. I never worry about a corrupt file system, never have to fsck it. And snapshots are cheap and fast. I shapshot the entire 10 TB array in about 30 minutes (about 2000 file systems). Then I back up from the snapshot. In other areas of the disk I do hourly snapshotting. Indeed snapshots are the kill feature for me for ZFS. LVM has snapshots, true, but they are not quick or convenient compared to ZFS. In LVM I can only snapshot to unused space in the volume set. With ZFS you can snapshot as long as you have free space. The integration of volume management and the file system may break a lot of people's ideas of clear separation between layers, but from the admin's point of view it is really nice.

We'll ditch ZFS and Solaris once BtrFS is ready. BtrFS is close, though; should work well for things like home servers, so try it out if you have a large MythTV system.

Re:They Why ZFS? (2, Informative)

Anonymous Coward | more than 3 years ago | (#34307042)

ZFS is...the only FS for large disks

XFS

I shapshot the entire 10 TB array in about 30 minutes (about 2000 file systems)...LVM has snapshots, true, but they are not quick or convenient compared to ZFS.

30 minutes? That's insane. An LVM2 snapshot would take seconds. I fail to see how that's not quick, and how "lvcreate -s" is less convenient.

In LVM I can only snapshot to unused space in the volume set. With ZFS you can snapshot as long as you have free space.

I can't even make sense of these two sentences. What you're saying is, an LVM snapshot requires free space, and er, a ZFS snapshot requires free space?

Re:They Why ZFS? (1, Informative)

Anonymous Coward | more than 3 years ago | (#34307336)

If you evenly split your 1TB disk into 2 LVM volume sets and volume1 is full, you can't make a snapshot of it. volume2 is sitting there empty, but the snapshot can't use it.

Re:They Why ZFS? (0)

Anonymous Coward | more than 3 years ago | (#34307396)

I've never actually backed myself into this situation so I have no idea, but surely if you just continue to add physical volumes to the existing volume group you can create a logical volume snapshot anywhere within that volume group without having to worry about which physical disk the snapshot is on?

Re:They Why ZFS? (1)

caseih (160668) | more than 3 years ago | (#34307982)

In an enterprise you're typically dealing with SAN. Just simply "adding physical volumes" isn't quite so simple. What if your disk array is full? Just tack a USB disk on the server? For us, all our SANs are hardware RAID (we don't use RAID-Z), so adding new volumes, as you suggest, involves buying at least 4 disk (RAID-6), sticking them in the chassis and creating a hardware volume set. It's quite an undertaking to expand storage. LVM can certainly accommodate our hardware, but would certainly not be efficient use of our disks. LVM's need to have unallocated space for snapshots has always been a weak spot. LVM snapshots are actually writable, though (BtrFS and ZFS snapshots are read-only). ZFS snapshots may be slower, but they are easier and more flexible. Perhaps some of the speed issues with ZFS snapshots come from the idea of thousands of sub-file-systems. Some overhead there, but the flexibility is totally worth it.

Re:They Why ZFS? (0)

Anonymous Coward | more than 3 years ago | (#34308510)

In an enterprise you're typically dealing with SAN.

Yes I am, you're right. However the rest of argument is total bunkum: if you are an enterprise user to hope to hell you monitor your disk usage and perform capacity planning. No one waits for their SAN to fill up and then go "Oh shit, now I have to add more disk!". I'm really at a loss as to why people think having to have a bit of space available for an LVM snapshot is a big deal. It's basic capacity planning.

Re:They Why ZFS? (1)

ranulf (182665) | more than 3 years ago | (#34307362)

He's saying that LVM can only snapshot to unallocated space, whereas ZFS can snapshot to space that is allocated to a partition not isn't currently being used.

This is simply because LVM works at a layer above the FS, whereas ZFS is the filesystem.

Re:They Why ZFS? (2, Interesting)

caseih (160668) | more than 3 years ago | (#34307906)

XFS

Wrong answer. XFS is extremely prone to data corruption if the system goes down uncleanly for any reason. We may strive for nine nines, but stuff still happens. A power failure on a large XFS volume is almost guaranteed to lead to truncated files and general lost data. Not so on ZFS.

30 minutes? That's insane. An LVM2 snapshot would take seconds. I fail to see how that's not quick, and how "lvcreate -s" is less convenient.

Glad to know LVM is faster though. However, as I stated before it's not convenient. With ZFS I do the following things:
- snapshot the works every night, and keep 7 days worth of snapshots.
- some directories are snapshotted every night, but I keep 365 snapshots (one year). For example the directories that our financial folk use.
- snapshot important directories every hour, keep 24 hours worth

You simply cannot do that with LVM. Sorry. How would I know how much free volume space to plan for? If I have a 10 TB disk, do I plan to use 6 TB of it and leave 4 TB for snapshots? Snapshots consume as much space as subsequent changes. For the 365 say snapshots, this could be a lot or very little depending on what has been touched.

I can't even make sense of these two sentences. What you're saying is, an LVM snapshot requires free space, and er, a ZFS snapshot requires free space?

It's very simple. LVM snapshots require free volume set space. If your volume group is 10 TB, then you must leave unallocated space on it for the snapshots to consume. On ZFS you don't need to do this. Any free space on the file system can be used for either files or snapshots; it's all the same pool. To do snapshots with LVM the way I do with ZFS would require me to set aside a lot of space. Very unefficient and wasteful.

As far as I can tell, BtrFS will work in a similar way to ZFS, bypassing the need for LVM. Which I'm totally okay with.

Re:They Why ZFS? (0)

Anonymous Coward | more than 3 years ago | (#34308346)

XFS

Wrong answer. XFS is extremely prone to data corruption if the system goes down uncleanly for any reason. We may strive for nine nines, but stuff still happens. A power failure on a large XFS volume is almost guaranteed to lead to truncated files and general lost data. ...

How much of that historically has been due to running XFS on LVM? LVM ignored file system barriers until very recently - kernel 2.6.30 or so, IIRC.

Re:They Why ZFS? (1)

DJProtoss (589443) | more than 3 years ago | (#34308036)

zfs snapshots are much more akin to a block level version of rsnapshot. lvm snapshots are more like zfs clones (although not quite, as even they are done Copy on Write (CoW).

Re:They Why ZFS? (2, Interesting)

TheLink (130905) | more than 3 years ago | (#34307126)

Question about ZFS, say I have a bunch of ZFS filesystems on a bunch of physical drives or drive arrays on Solaris/OpenSolaris/OpenIndiana.

How do I figure out which physical drives/devices a particular ZFS filesystem depends on?

And if a physical drive is faulty, how would I know which actual physical drive it is? e.g. get its serial number or physical slot/bay/position or whatever.

Re:They Why ZFS? (5, Informative)

Maquis196 (535256) | more than 3 years ago | (#34307196)

zpool status

That's the command you are looking for. The zfs-fuse lists disks by id which means if you go into /dev/disks/by-id/ and do a ls -al you'll see which devices they are linked to.

It is done this way to make it easier in Linux, in BSD/Solaris the disks are by gpt name (well they were for me) so this keeps it sane.

Hope it helps.
Maq

Re:They Why ZFS? (0)

Anonymous Coward | more than 3 years ago | (#34307228)

zpool status

Re:They Why ZFS? (0)

Anonymous Coward | more than 3 years ago | (#34307522)

ZFS is, until BtrFS hits truly enterprise stable, the only FS for large disks, ...

Do you even KNOW what a "large disk" is? 10 TB is child's play.

Go use something like Sun/Oracle SAM/QFS or IBM's GPFS or SGI's XFS (as another poster noted - but only on "native" Irix because in Linux XFS is crappy crippleware).

ZFS is stink-slow compared to those. Hell, IMO ZFS is stink-slow compared to just about any competent file system. Anybody using ZFS and having performance requirements has to toss way too much hardware at their problems. ZFS has some good features, but speed sure ain't one of them.

And also note that there's no FOSS alternative to any of those. Sorry, folks, but if you want a file system that can take petabytes worth of 8 GB/sec fiber arrays and run them at line speed transferring data over actual shared file systems for hours on end, FOSS fails and fails badly.

Re:They Why ZFS? (1)

LWATCDR (28044) | more than 3 years ago | (#34307194)

Well they tested on a single SSD.
I have not used ZFS or Btrfs but I have read a lot about ZFS.
This is not really the use case for ZFS. ZFS has many features for things like using an SSD to cache for the HDDs , RAID like functions, data compression and so on.
The idea that a simpler less full featured file system is faster is no big shock.
I would like to see tests with maybe two wan servers each with say 12 HHDs and an SSD for cacheing. That is more the use case for ZFS than a workstation with a single SSD.

Re:They Why ZFS? (1)

Bengie (1121981) | more than 3 years ago | (#34307972)

People like my cousin who run a data center with 10,000+ hard drives and by requirement must have a File System that has been considered stable for at least 5 years. Any data loss is unacceptable. Unless God targets you with His wrath, you have no excuse for any data loss or corruption.

Using a first beta slower than stable? Wha?!?!? (5, Insightful)

tysonedwards (969693) | more than 3 years ago | (#34306704)

Who would have thought that a first-release Beta kernel module would not run as fast or be as reliable as the stable implementation for other operating systems, or the stables on Linux?

Re:Using a first beta slower than stable? Wha?!?!? (2, Informative)

chrb (1083577) | more than 3 years ago | (#34307406)

Who would have thought that a first-release Beta kernel module would not run as fast or be as reliable as the stable implementation for other operating systems, or the stables on Linux?

The full release is supposed to be coming out in the first week of January. Given the short time frame, it would seem like this is probably closer to the final release than the words " first beta" imply.

Surprises:

  • Native ZFS beat XFS on several of the benchmarks - XFS is usually a good performer in these kind of tests
  • Native ZFS does very well on the Threaded IO Test, where it ties for first place.
  • Btrfs is really bad on the SQLite test, taking 5 times longer than XFS on both 2.6.32 and 2.6.37 (bug?)
  • XFS IOzone write performance increased by 45% going from 2.6.32 to 2.6.37 (!) XFS increased on FS-Mark by 37%. I thought XFS would be pretty much at the point where there would be no such great improvements.
  • "Real" Solaris+ZFS gets absolutely slaughtered on the Threaded IO Test and the PostMark Test, with ext4 pushing almost 10x more transactions per second.
  • Tests were done on a SSD, apparently there was no difference in relative performance of the filesystems on SSD versus HD

Notes:

  • "Real" Solaris+ZFS results are not shown for most tests
  • Would be nice to know how many replicates they did of each test
  • This is an interesting set of results that will get people talking/arguing :-) Thanks to Phoronix for starting the discussion.

Re:Using a first beta slower than stable? Wha?!?!? (0)

Anonymous Coward | more than 3 years ago | (#34307758)

Tests were done on a SSD, apparently there was no difference in relative performance of the filesystems on SSD versus HD

Ah, but here's a nice trick: with ZFS you can use plain disks for raw storage, but slot in an SSDs transparently so that you get performance boosts in read and writes. You don't have to spend a lot of cash on pricey SSDs, just enough to caching your working set:

http://blogs.sun.com/studler/entry/zfs_and_the_hybrid_storage

Re:Using a first beta slower than stable? Wha?!?!? (0)

Anonymous Coward | more than 3 years ago | (#34308122)

A guy from LLNL gave a talk at Supercomputing 2010 last week about the status ZFS on Linux; he said they have done zero performance tuning. Zero. As in, "don't expect ZFS on Linux to be fast."

http://sc10.supercomputing.org/schedule/event_detail.php?evid=bof186

how about versus ZFS on Solaris or FreeBSD? (2, Insightful)

Anonymous Coward | more than 3 years ago | (#34306738)

On similar hardware of course.

It occurs to me that ZFS does a lot more than EXT4 and Btrfs too.

Re:how about versus ZFS on Solaris or FreeBSD? (1)

mcelrath (8027) | more than 3 years ago | (#34306872)

If you had read TFA, you'd know they did benchmark on Solaris (OpenIndiana).

Re:how about versus ZFS on Solaris or FreeBSD? (0)

Anonymous Coward | more than 3 years ago | (#34306906)

read TFA -- Are you crazy?

Re:how about versus ZFS on Solaris or FreeBSD? (1)

hedwards (940851) | more than 3 years ago | (#34307072)

But that's not particularly helpful. I don't believe that Btrfs is supported beyond Linux at the moment and neither FreeBSD nor Open Solaris support both. Meaning that you're comparing a filesystem that's been grafted onto Linux via fuse with one that can ultimately be integrated into the Linux kernel.

Re:how about versus ZFS on Solaris or FreeBSD? (1, Informative)

Anonymous Coward | more than 3 years ago | (#34307576)

If you read TFA (or perhaps even the slashdot submission text) you should know that both fuse and native ports for linux are being discussed.

That's not a solaris filesystem (1, Insightful)

Anonymous Coward | more than 3 years ago | (#34306806)

You can't call it "the Solaris file-system". You can say that the Linux native implementation of ZFS (a Linux file-system) is slower than BTRFS, though.

And, what does it matter it to be fast if it's not reliable? You can save your stuff in /dev/null quite fast too!

http://www.spinics.net/lists/linux-fsdevel/msg35235.html

Re:That's not a solaris filesystem (3, Funny)

datapharmer (1099455) | more than 3 years ago | (#34306892)

You can save your stuff in /dev/null quite fast too!

I know! It is friggin crazy fast. I've been using it for backups for years. Even with terrabytes of data I've never managed to fill it up or slow it down!

Re:That's not a solaris filesystem (2, Insightful)

hedwards (940851) | more than 3 years ago | (#34307098)

Well, don't forget to use that magic rewinding tape that mysteriously never fills no matter how many backups you use it for. Better safe than sorry I always say.

Doomed to failure by license conflict (4, Interesting)

mattdm (1931) | more than 3 years ago | (#34306858)

OpenAFS, which still today provides features unavailable in any other production-ready network filesystem, is a nightmare to use in the real world because of its lack of integration with the mainline kernel. It's licensed under the "IPL", which like the CDDL is free-software/open source but not GPL compatible.

ZFS is very cool, but this approach is doomed to fail. It's much better to devote resources to getting our native filesystems up to speed -- or, ha, into convincing Oracle to relicense.

Personally, I was pretty sure Sun was going to go with relicensing under the GPLv3, which gives strong patent protection and would have put them in the hilarious position of being more-FSF free software than Linux. But with Oracle trying to squeeze the monetary blood from every last shred of good that came from Sun, who knows what's gonna happen.

Re:Doomed to failure by license conflict (1)

wonkavader (605434) | more than 3 years ago | (#34306952)

"But with Oracle trying to squeeze the monetary blood from every last shred of good that came from Sun, who knows what's gonna happen."

Well, in the short term, we know what's not going to happen.

Re:Doomed to failure by license conflict (1)

caseih (160668) | more than 3 years ago | (#34307346)

You mean like how the Nvidia GPU driver has failed because of licensing conflict? I see no reason why the ZFS module can't be distributed in a similar manner to the nvidia driver. I'm sure that rpmfusion could host binary RPMs without problem. They wouldn't be violating the GPL because it would be you the user who taints the kernel.

Of course ZFS on Linux probably isn't aimed at normal users anyway. It's far more likely to be used by people with existing ZFS infrastructure (large fiber-channel arrays, etc). In my opinion, ZFS on linux gives a smoother migration path away from Oracle Solaris and ZFS.

Re:Doomed to failure by license conflict (1)

mattdm (1931) | more than 3 years ago | (#34308356)

It differs from the Nvidia driver because the Nvidia module until recently was needed to make very common PC hardware work at all, and even with the new free software Nouveau drivers, still needed for game-level performance. ZFS has neat features, but you don't need it in order to have storage on Linux.

There's clearly a niche market for out-of-tree ZFS modules, or else this wouldn't have gotten funding. But if you're not already committed, it adds significant overhead. As someone who was dependent on OpenAFS for years for legacy reasons, I strongly caution people that the overhead is unlikely to be worth it.

Re:Doomed to failure by license conflict (0)

Anonymous Coward | more than 3 years ago | (#34308266)

Yet a troll complaining about Oracle/Sun. Not original, true or worth listening too

Re:Doomed to failure by license conflict (1)

QuantumRiff (120817) | more than 3 years ago | (#34308294)

ZFS is very cool, but this approach is doomed to fail. It's much better to devote resources to getting our native filesystems up to speed -- or, ha, into convincing Oracle to relicense.

Personally, I was pretty sure Sun was going to go with relicensing under the GPLv3, which gives strong patent protection and would have put them in the hilarious position of being more-FSF free software than Linux. But with Oracle trying to squeeze the monetary blood from every last shred of good that came from Sun, who knows what's gonna happen.

Um, just who do you think is writing BTRFS? http://en.wikipedia.org/wiki/Btrfs [wikipedia.org] I know its fashionable to knock Oracle every chance you get... but Look at the line:

Btrfs, when complete, is expected to offer a feature set comparable to ZFS.[16] btrfs was considered to be a competitor to ZFS. However, Oracle acquired ZFS as part of the Sun Microsystem's merger and this did not change their plans for developing btrfs.[17]

Different ZFS distros (3, Informative)

hoggoth (414195) | more than 3 years ago | (#34306870)

I was confused as to what versions of ZFS were available on which distros so I made a chart that lists the different distros and which version of ZFS they support:

http://petertheobald.blogspot.com/2010/11/101-zfs-capable-operating-systems.html [blogspot.com]

Hope it's helpful...

Btrfs naming convention (3, Funny)

digitaldc (879047) | more than 3 years ago | (#34306898)

Couldn't they name the file system something better than butterface?

Re:Btrfs naming convention (2, Funny)

timeOday (582209) | more than 3 years ago | (#34307046)

What are you complaining about? I always thought it was "bitter farts."

Re:Btrfs naming convention (0)

Anonymous Coward | more than 3 years ago | (#34307064)

The 'r' is silent, it's actually buttface

Re:Btrfs naming convention (1)

spud603 (832173) | more than 3 years ago | (#34307144)

no, they could not.

Re:Btrfs naming convention (2, Funny)

Abstrackt (609015) | more than 3 years ago | (#34307240)

Unfortunately Gimp was already taken.

Re:Btrfs naming convention (0)

Anonymous Coward | more than 3 years ago | (#34307658)

I always thought it was "bitter face", like the old Keystone Beer "bitter beer face" commercials.

Re:Btrfs naming convention (1)

Tehrasha (624164) | more than 3 years ago | (#34307664)

I keep reading that as 'Bit Torrent File System'...

Re:Btrfs naming convention (1)

tangent3 (449222) | more than 3 years ago | (#34308026)

Can't read my, can't read my, no he can't read my btfs

As an end user... (1)

soupforare (542403) | more than 3 years ago | (#34306904)

I've been through a few filesystem war^Wdramas and stuck with ext?fs the whole time. I liked the addition of journaling but I'm not sure that I've noticed any of the other "backstage" improvements in day to day functioning.
Is there really a reason to jump ship as a single-workstation user?

Re:As an end user... (1)

Etherized (1038092) | more than 3 years ago | (#34307094)

Snapshotting is probably the most compelling feature of either FS for workstation use. Both BTRFS and ZFS are copy-on-write, and they both feature very low overhead, very straightforward snapshotting. That's a feature that almost anybody can utilize.

Aside from that, ZFS features a lot of datacenter-centric goodies that might have some utility on a workstation as well. Realtime (low overhead) compression, realtime (high overhead) deduplication, realtime encryption, easy and fast creation/destruction of filesystems and virtual block devices, and a ton of other odds and ends.

Re:As an end user... (1)

afidel (530433) | more than 3 years ago | (#34307556)

L2ARC! Use that small SSD to improve average performance for almost all your files.

Re:As an end user... (1)

hedwards (940851) | more than 3 years ago | (#34307218)

The ext?fs work well unless they don't. In my, admittedly limited experience, I've lost more files on ext2fs than on all other filesystems I've dabbled in combined. Admittedly, I had backups, but any fs that depends upon you having backups to that extent should not be trusted. And while I'm sure the newer ones are better, I'm not sure that I personally trust them as ext2fs shouldn't have been that easy to corrupt. IIRC that was only a couple years ago, and it should've been both robust and well undestood by then.

Re:As an end user... (1)

icebraining (1313345) | more than 3 years ago | (#34307268)

Probably not, especially considering they're still less tested. Ext3 + LVM already provide everything I need for now.

Re:As an end user... (1)

mlts (1038732) | more than 3 years ago | (#34307682)

For me, journaling was the reason to move from ext2 to ext3. However, for an end user, ZFS has a few cool features that are significant:

1: Deduplication by blocks. For end users, it should save some disk space, not sure how much.
2: File CRCs. This means file corruption is at least detected.
3: RAID-Z. 'Nuff said. No worry about the LVM layer.
4: Filesystem encryption.

Re:As an end user... (1)

Hatta (162192) | more than 3 years ago | (#34307804)

If you need the features provided by an advanced filesystem, you'll know. If you're not hitting your head on the limits of EXT4/LVM/RAID, then you don't really need ZFS or Btrfs.

WTF? RU2B4 REAL? OMFG! 90210! IBM! GOP! GIMP! !! (-1, Offtopic)

Anonymous Coward | more than 3 years ago | (#34306922)

Right then. On with what you were doing, sailor.

I'm using btrfs on my home partition. (1)

Seth Kriticos (1227934) | more than 3 years ago | (#34306960)

It's OK, runs fairly stable, but it also locks up once in a while and does some aggressive disk I/O. No idea what exactly, probably housekeeping, but it's somewhat irksome, could use some more fine tuning.

The main problem with btrfs right now is that it lacks fsck tools, so in case of havoc there is little chance to recuperate, which is not good for server like systems.

As for ZFS, it's not the tech that's keeping it from Linux but the restrictive licensing. Unless that gets fixed (probably won't happen), it is off limits, and Linux folks will do their own thing, like the always do.

Re:I'm using btrfs on my home partition. (1, Informative)

larkost (79011) | more than 3 years ago | (#34307634)

"As for ZFS, it's not the tech that's keeping it from Linux but the restrictive licensing."

Just to be clear: between CDDL (ZFS) and GPL (BTRFS), GPL is clearly the more restrictive license. BTRFS can probably never be shipped with any other major OS other than linux (at least not in kernel mode), while ZFS has already shipped with a few.

The license restriction is one of linuxes making, not ZFS's. There are arguments for that restricion, but calling the problem one of CDDL being restrictive is a completly distorted view.

Re:I'm using btrfs on my home partition. (3, Insightful)

Hatta (162192) | more than 3 years ago | (#34307872)

BTRFS can probably never be shipped with any other major OS other than linux

It's not BTRFS's fault that other operating systems use licenses with more restrictions than Linux.

Re:I'm using btrfs on my home partition. (0)

Anonymous Coward | more than 3 years ago | (#34308042)

Just to be clear: between CDDL (ZFS) and GPL (BTRFS), GPL is clearly the more restrictive license.

We want freedom for the little guy (end users), not freedom for corporations.

Re:I'm using btrfs on my home partition. (1)

tao (10867) | more than 3 years ago | (#34308582)

Hmmm, so the btrfsck I have installed on my system is imaginary? On a Debian system the relevant package is btrfs-tools, but you might be running a different distro where it's called something else.

Not bad news (4, Interesting)

wonkavader (605434) | more than 3 years ago | (#34307108)

It's still under development. But it's already pretty competitive, doing reasonably well in many tests.

And then there's this (on the last page) "Ending out our tests we had the PostMark test where the performance of the ZFS Linux kernel module done by KQ Infotech and the Lawrence Livermore National Laboratories was slaughtered. The disk transaction performance for ZFS on this native Linux kernel module was even worse than using ZFS-FUSE and was almost at half the speed of this test when run under the OpenSolaris-based OpenIndiana distribution."

Ok, maybe someone can disabuse me of a misconception that I have, but: There's no reason that ZFS in the kernel should be slower than a FUSE version. That means there's something wrong. If they figure out what's wrong and fix it, that could very likely affect the results in some or all of the other tests.

ZFS isn't done yet, and it already looks like it might be worth the trade-off for the features ZFS provides. And performance might get somewhat better. This article is good news (though that final benchmark is distressing, especially when you look at the ZFS running on OpenSolaris).

It says: "When KQ Infotech releases these ZFS packages to the public in January and rebases them against a later version of ZFS/Zpool, we will publish more benchmarks."

and I'm looking forward to that new article.

Re:Not bad news (0)

Anonymous Coward | more than 3 years ago | (#34308420)

ZFS in the kernel ISN'T slower than the FUSE extension. RTFS.

Why I use ZFS/Solaris in production for Postgres (1)

Pengo (28814) | more than 3 years ago | (#34307296)

The throughput for large data sorts are just faster, period.

A lot of it has to do with the reading of compressed data, and the huge ram-buffer that ZFS uses on the OS, optional commit on writes, block sizes that match the database pages.

The system scans 3 megs of index data, what it's actually reading to get that off is say 1 meg, which it decompresses on the fly on one of the many cores the database server has. In the end throughput destroys what i get running non-compressed volumes on EXT4 or XFS on Linux. For "MY" database, it runs nearly 2-3x faster than the same hardware running on Linux. (RHEL5 is what I ran the db on for a long time).

I have not been able to get Linux/Postgres to run even partially as fast as I have been able to get Solaris/ZFS running Postgres 8.3.

Btrfs isn't even near production states yet, but i am really hoping that it will give me an option to get off of Solaris.

On that note, one thing i haven't tried yet with our DB is Solid State Drives. The sheer throughput might more than make up for the benefits i get on compressed ZFS volumes.

I for one am VERY VERY hopeful that BTRFS can get stable, and fast. Oracle's fiasco has me and a few other people at our small business very nervous. I'm not planning on replacing our Sol10 (free) distribution , and could care less about the support Oracle offers. I'm playing with Solaris Express 11 now, but not sure I want to pay the $1k a year for production use, though if it offers me the performance gains over linux that I'm currently seeing, it will probably be worth it for our Database system alone.

Has anyone here had experience tuning Postgres on Linux versus Solaris/ZFS ? We're not a huge shop, 8 people running large data-warehouse type applications. We run on a shoestring and don't have a bunch of money to throw at the problem and would be very grateful for any ideas on how to make our database run with comparable performance on Linux. I'm hoping that I'm missing something obvious.

Re:Why I use ZFS/Solaris in production for Postgre (1)

jimicus (737525) | more than 3 years ago | (#34307410)

Has anyone here had experience tuning Postgres on Linux versus Solaris/ZFS ? We're not a huge shop, 8 people running large data-warehouse type applications. We run on a shoestring and don't have a bunch of money to throw at the problem and would be very grateful for any ideas on how to make our database run with comparable performance on Linux. I'm hoping that I'm missing something obvious.

What have you done so far and how are you using Postgres? Mostly reads, mostly writes or some combination of the two? Postgres as it ships is notorious for slow configuration, and many Linux distributions are consistently one major version behind the curve (which is a little annoying as much of the focus of the Postgres people for some time has been improving performance).

Re:Why I use ZFS/Solaris in production for Postgre (0)

Anonymous Coward | more than 3 years ago | (#34307566)

So, you're not a fan of Oracle I take it? Have a look at who's developing btrfs.

Re:Why I use ZFS/Solaris in production for Postgre (1)

Ash-Fox (726320) | more than 3 years ago | (#34308694)

Have a look at who's developing btrfs.

For the curious, it's a single person called Chris Mason who happened to work also on ReiserFS (the killer filesystem).

Apples vs Oranges (0)

Anonymous Coward | more than 3 years ago | (#34307364)

Holy crap, they only tested on a single SSD. This is akin to running DOS on a 64 core system.

ZFS is a full volume management /file system with the ability to partition filesystems (on the fly I might add), assign different attributes (compression, deduping, max size, caching, data replication etc), add space on the fly, set storage/redudancy parameters (RAID 0,1,10 and their own raidz1, raidz2), spare assignment, live disk replacement, etc, etc.

Try throwing 48 x 3TB disks into a chassis and configuring them (use SSDs for the ZIL/ARC, and set up the zpools correctly). Try doing the same thing with BTRFS. Actually, you can't.

I love the fact that a native Linux driver is coming, but this review is completely useless.

It's just not yet there (1)

Artem Tashkinov (764309) | more than 3 years ago | (#34307602)

When XFS was introduced in Linux it also sucked performance wise, so, I think for ZFS on Linux there's certainly a room for improvement.

And even in this early age ZFS shows very remarkable results, so let's just wait and see.

For ZFS, speed is a secondary goal (3, Insightful)

pedantic bore (740196) | more than 3 years ago | (#34307718)

Picking on ZFS for being slow when ported to a different OS and running on atypical hardware is like criticizing Stephen Hawking for being a poor juggler. It's focussing on the wrong thing. The goals of ZFS are, in no particular order:
- Scalability to enormous numbers of devices
- Highly assured data integrity via checksumming
- Fault tolerance via redundancy
- Manageability/usability features (i.e., snapshots) that conventional file systems simply don't have
Oh, and if it's fast, well, that's gravy.

Re:For ZFS, speed is a secondary goal (1)

guruevi (827432) | more than 3 years ago | (#34308588)

The fast can be achieved by more/better hardware. A filesystem shouldn't have 'fast' or 'faster than ye' as it's primary focus anyway. If it's very fast but not 100% trustworthy it's not a good file system (eg. ReiserFS).

Some features that make ZFS a bit slower are thought up by people that have years of experience in large SAN and other storage solutions. Writing metadata multiple times over different spindles might seem overkill for most but that is until you lose a N+1 spindles (or just get r/w errors on the N+1'th spindle while the others are recovering) and in a typical situation this means the whole file system is hosed but ZFS can sometimes recover a lot of and will be able to tell which files it could not fix which is nice when your system has many TB's and takes days to restore from a full backup.

ZFS solves the fast by allowing frequently accessed data to be in memory or faster disks (like SSD's) and have small sync writes hit an intent log while optimizing async writes before putting them on-disk. Give ZFS more memory than your average desktop and you'll be a lot faster in reads, give it a small SLC SSD and see it hit 10k stable write IOPS.

Re:For ZFS, speed is a secondary goal (1)

Ash-Fox (726320) | more than 3 years ago | (#34308634)

Picking on ZFS for being slow when ported to a different OS and running on atypical hardware

How is he picking? He's just measuring the file system performance compared to others on a specific OS.

It's focussing on the wrong thing.

I don't think it is, this person wanted to measure performance on Linux, not compare features and he got what he was testing. I would imagine there are plenty of people who want to know how well it performs - regardless of features - in comparison to other filesystems.

Benchmarking ZFS on a single disk is misleading (1)

Guy Smiley (9219) | more than 3 years ago | (#34308466)

Since ZFS is doing metadata replication, running the tests on a single disk is going to punish ZFS performance much more than other filesystems. It would be much more interesting to run a benchmark with an array of 6 or 8 disks with RAID-Z2, with ZFS managing the disks directly, and XFS/btrfs/ext4 running on MD RAID-6 + LVM. Next, run a test that creates a snapshot in the middle of running some long benchmark and see what the performance difference is before/after.

Load More Comments
Slashdot Login

Need an Account?

Forgot your password?
or Connect with...

Don't worry, we never post anything without your permission.

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>