Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Software SSD Cache Implementation For Linux?

timothy posted more than 4 years ago | from the only-obscure-if-you-think-it-is dept.

Data Storage 297

Annirak writes "With the bottom dropping out of the magnetic disk market and SSD prices still over $3/GB, I want to know if there is a way to to get the best of both worlds. Ideally, a caching algorithm would store frequently used sectors, or sectors used during boot or application launches (hot sectors), to the SSD. Adaptec has a firmware implementation of this concept, called MaxIQ, but this is only for use on their RAID controllers and only works with their special, even more expensive, SSD. Silverstone recently released a device which does this for a single disk, but it is limited: it caches the first part of the magnetic disk, up to the size of the SSD, rather than caching frequently used sectors. The FS-Cache implementation in recent Linux kernels seems to be primarily intended for use in NFS and AFS, without much provision for speeding up local filesystems. Is there a way to use an SSD to act as a hot sector cache for a magnetic disk under Linux?"

cancel ×

297 comments

Sorry! There are no comments related to the filter you selected.

I don't get it (1)

microbee (682094) | more than 4 years ago | (#31946240)

Linux caches data from any disks all the same, SSD or not.

Re:I don't get it (1)

Wesley Felter (138342) | more than 4 years ago | (#31946280)

Linux caches disk data in memory. The author wants to cache disk data in an SSD.

Re:I don't get it (1)

Korin43 (881732) | more than 4 years ago | (#31946436)

Would using an SSD as a swap device have the effect they want?

Re:I don't get it (1, Informative)

Unit3 (10444) | more than 4 years ago | (#31946478)

No. Swap is not a cache. Swap holds things that don't fit in RAM. I/O cache will never hit swap, it limits itself to physical RAM.

Re:I don't get it (1, Informative)

Anonymous Coward | more than 4 years ago | (#31946614)

http://leaf.dragonflybsd.org/cgi/web-man?command=swapcache&section=ANY

Re:I don't get it (1, Offtopic)

Penguinisto (415985) | more than 4 years ago | (#31946518)

...only if you want to blow out the SSD wear-limits.

What the author wants (I believe) is to have Linux figure down which sectors are read most frequently, and have those mapped/linked/whatever to the SSD for speed reasons.

If that's indeed the case, then why not simply put the MBR, /boot, /bin, and /usr on the SSD, then mount stuff like /home, /tmp, swap, and the like onto a spindle disk? No algorithm needed, thus no overhead needed to run it, etc.

Re:I don't get it (1)

Annirak (181684) | more than 4 years ago | (#31946676)

The issue is that there are lots of frequently read data blocks in /home, /tmp, /swap, etc. and there are lots of infrequently accessed data blocks in /boot, /bin, and /usr. Storing infrequently read data on a fast device and frequently read data on a slow device is inefficient. I want a system which puts the frequently read data on a fast device, and the infrequently read data on a slow device.

Re:I don't get it (1)

pak9rabid (1011935) | more than 4 years ago | (#31946864)

Correct me if I'm wrong, but isn't /tmp usually mapped to a ramdisk?

Re:I don't get it (1)

the_one(2) (1117139) | more than 4 years ago | (#31947040)

Probably not. That would be bad if for example you wanted to burn a DVD and the burner program put a lot of stuff in /tmp. I'm not a linux pro or anything so I don't know how different distros do it but I don't think that's the default.

Re:I don't get it (1)

raynet (51803) | more than 4 years ago | (#31947200)

Well, if you use tmpfs and not ramdisk for /tmp, then pages will be swapped to disk if needed, thus you can burn you DVD as long as you have enough swap available and damons like swapd or swapspace allow you to have reasonable size swap partition and then will create swapfiles by demand.

Re:I don't get it (1)

rwa2 (4391) | more than 4 years ago | (#31947118)

Correct me if I'm wrong, but isn't /tmp usually mapped to a ramdisk?

Depending on the distribution, but sometimes.

On servers /tmp can get pretty big with random crap, though, so generally you want to be able to put it on a disk or allow it to swap out and use your RAM for something more useful.

But on thin clients, netbooks, etc. without too much going on it might be better to put it on tmpfs to reduce SSD wear.

Re:I don't get it (1)

dissy (172727) | more than 4 years ago | (#31946872)

If that's indeed the case, then why not simply put the MBR, /boot, /bin, and /usr on the SSD, then mount stuff like /home, /tmp, swap, and the like onto a spindle disk? No algorithm needed, thus no overhead needed to run it, etc.

Unfortunately, it usually works out that some of the most volatile places on disk (/home /tmp swap) are the very places one would see the best result in speeding up.

Also unfortunately those are the worst uses currently for a SSD

Then again, for anyone who really wants to speed up things like swap and /tmp, the best way is to simply quadruple your ram and get rid of swap, and use tmpfs in ram for /tmp.

The usual reason for not doing that is ram is expensive, and on top of that motherboards to handle a ton of ram are also expensive (At least compared to their max 4-8gb 4 slot lower end versions)
However since SSDs are in the equation here, I'm guessing penny pinching is probably not that much of a concern.

Re:I don't get it (1)

countach (534280) | more than 4 years ago | (#31946926)

Because all of /bin is hardly going to be your most used stuff, and there's probably a ton of stuff frequently used that isn't in /bin, /usr.

Sure, you can try and mount your most used stuff on SSD, but that's (a) a pain in the neck to fiddle around with (b) something ideally better left to an algorithm. (c) doesn't actually work that well, since you have to divide all your most used stuff into separate file systems.

Re:I don't get it (1, Informative)

Jezza (39441) | more than 4 years ago | (#31946880)

Assuming the SSD was faster at both read and write - it should speed things up. Hell just moving the swap onto a different physical disk helps. But don't. SSD have a limited life, in a different sense to spinning disks. SSD wear with writing, so if you constantly write to the same "sectors" they will fail. If you think about what's happening when the system is swapping - that's exactly what's going on. So yes, it'll help (a bit) but it's really expensive given what will happen to the SSD. Better is add RAM, so the system won't need to swap (with enough RAM you don't need swap at all).

Re:I don't get it (2, Informative)

Anonymous Coward | more than 4 years ago | (#31947224)

SSD wear with writing, so if you constantly write to the same "sectors" they will fail.

2006 called, they want their FUD back. While it's true that erase blocks in flash memory wear out with use, the whole battle between SSD manufacturers for the last couple years has been in mapping algorithms that ensure you don't hit the same erase block very often. By now, SSDs have longer lifetimes than HDDs. Of course that applies to real SSDs, not makeshift IDE-to-CompactFlash adapters.

Re:I don't get it (1)

owlstead (636356) | more than 4 years ago | (#31946324)

This is about doing double caching: cache to fast but limited RAM (L1) first and then have a much larger but slower cache, that being the SSD (L2). Difference being with other caching systems that the SSD of course holds state if power is down (so often use sectors may never be written do disk).

Re:I don't get it (4, Insightful)

Colin Smith (2679) | more than 4 years ago | (#31946506)

so

CPU L1
CPU L2
CPU L3
RAM
SSD
DISK
NETWORK
Internet

I estimate SSDs would be closer to Level 5 cache.

 

Re:I don't get it (1)

Znork (31774) | more than 4 years ago | (#31946982)

I'd argue it's better to implement as HSM (Hierarchial Storage Management), with least recently used things getting delegated to more archival storage. It would be nice with a device-mapper-hsm layer that would let you simply stack one device upon the other and obtain the best distribution of desireable characteristics they could offer.

IIRC, there was an intern at IBM who did a project like that some years ago, but I don't think much became of it.

Re:I don't get it (1)

MobyDisk (75490) | more than 4 years ago | (#31947162)

<facepalm
If you want to play Mr. Pedantic, you skipped registers. And CPU cache may not necessarily cache disk data, so those don't count for the same reason registers don't count. Networks don't cache the internet. And don't forget newspapers - the Internet caches those. And newspapers cache events, which cache time. :-)

The point is that most of the layers you listed are implementation dependent or not relevant to the discussion. For this purpose, the CPU is a black box - it could have different # of levels, and they might not always be used for that purpose. So RAM and SSD are caching disk. It stops there.

Re:I don't get it (1)

Jezza (39441) | more than 4 years ago | (#31947060)

Yeah, especially if those sectors don't change much - an SSD isn't suitable for data that's rapidly changing.

Re:I don't get it (0)

Anonymous Coward | more than 4 years ago | (#31946394)

I think the submission is talking about inserting the SSD as a (huge) caching layer between the buffers in RAM and the hard drives, something that's much better than normal RAID, but not quite as good as using a metric fuck-ton of RAM and memcached.

Re:I don't get it (1)

vikingpower (768921) | more than 4 years ago | (#31947226)

That is how I understood it.

Re:I don't get it (1)

JessGras (953965) | more than 4 years ago | (#31946404)

Linux caches disk accesses to RAM. The OP asks about caching disk accesses to SSD. SSDs are much more expensive than magnetic disk, but in turn RAM is much more expensive than SSDs. So at a fixed price point you could cache a great deal more of your HD w/ SSD than w/ RAM.

Plus, RAM is wiped on reboot. With an SSD cache in front of your HD, you would benefit from the SSD performance, say, on your next reboot - something a cache in RAM could not offer. Or perhaps you launch Gimp several times a day: it would be nice to see it fire up 100x faster!

Re:I don't get it (5, Informative)

Anonymous Coward | more than 4 years ago | (#31946466)

The idea is to use the SSD as a second-level disk cache. So instead of simply discarding cached data under memory pressure, it's written to the SSD. It's still way slower than RAM, but it's got much better random-access performance characteristics than spinning rust and it's large compared to RAM.

As for how to do it in Linux, I'm not aware of a way. If you are open to the possibility of using other operating systems, this functionality is part of OpenSolaris (google for "zfs l2arc" for more information).

Cache Devices
          Devices can be added to a storage pool as "cache devices."
          These devices provide an additional layer of caching between
          main memory and disk. For read-heavy workloads, where the
          working set size is much larger than what can be cached in
          main memory, using cache devices allow much more of this
          working set to be served from low latency media. Using cache
          devices provides the greatest performance improvement for
          random read-workloads of mostly static content.

          To create a pool with cache devices, specify a "cache" vdev
          with any number of devices. For example:

              # zpool create pool c0d0 c1d0 cache c2d0 c3d0

          The content of the cache devices is considered volatile, as
          is the case with other system caches.

You can also use it as an intent log, which can dramatically improve write performance:

Intent Log
          The ZFS Intent Log (ZIL) satisfies POSIX requirements for
          synchronous transactions. For instance, databases often
          require their transactions to be on stable storage devices
          when returning from a system call. NFS and other applica-
          tions can also use fsync() to ensure data stability. By
          default, the intent log is allocated from blocks within the
          main pool. However, it might be possible to get better per-
          formance using separate intent log devices such as NVRAM or
          a dedicated disk. For example:

              # zpool create pool c0d0 c1d0 log c2d0

          Multiple log devices can also be specified, and they can be
          mirrored. See the EXAMPLES section for an example of mirror-
          ing multiple log devices.

          Log devices can be added, replaced, attached, detached, and
          imported and exported as part of the larger pool. Mirrored
          log devices can be removed by specifying the top-level mir-
          ror for the log.

Re:I don't get it (1)

jibjibjib (889679) | more than 4 years ago | (#31946616)

Yes, Linux caches data from disks in RAM. But what we're talking about here is not caching in RAM, but using a fast disk (SSD) as cache for a slow disk.

Re:I don't get it (1)

InlawBiker (1124825) | more than 4 years ago | (#31946918)

What about this: the SSD Ram Disk (SSDRD). It's exactly like a normal RAM disk, but it simulates an SSD. It would be supremely faster to write to an imaginary SSD rather than an imaginary HD.

Patent!

Re:I don't get it (4, Informative)

TheRaven64 (641858) | more than 4 years ago | (#31946988)

The submitter wants something like ZFS's L2ARC, which uses the flash as an intermediate cache between the RAM cache and the disk. This works very well [sun.com] for a lot of workloads. Since Linux users appear to be allowed to say 'switch to Linux' as an answer to questions about Windows, it only seems fair that 'switch to Solaris of FreeBSD' would be a valid solution to this problem.

Re:I don't get it (0)

GuruBuckaroo (833982) | more than 4 years ago | (#31947002)

What the author wants is what Windows 7 calls ReadyBoost [wikipedia.org] - except using SSDs instead of USB Flash drives. I'd love to see it too.

I'd like to do this in Windoze (0)

Anonymous Coward | more than 4 years ago | (#31946262)

I can't find a commercial SSD / Platter Hybrid anywhere...

I wouldn't mind doing some of the work myself but I can't even find those mythical ssd / hard drive kits.

Re:I'd like to do this in Windoze (1)

GuruBuckaroo (833982) | more than 4 years ago | (#31947150)

Throw the SSD in a USB enclosure and try ReadyBoost [wikipedia.org] if you're using Windows Vista SP1, 7, or 2008 Server. Not sure if that will work or not, but you can try it. Works with Flash drives.

isn't 40 GB enough for applications? (3, Interesting)

owlstead (636356) | more than 4 years ago | (#31946264)

Is there really a need for this? Intel 40 GB SSD still has a read speed of 170 MB/s and costs about 100 euro here in NL. Why have some kind of experimental configuration while prices are like that? OK, 35 MB/s write speed is not that high, but with the high IOPS and seek times you still have most of the benefits.

I can see why you would want something like this, but I doubt the benefits are that large over a normal SSD + HDD configuration.

Re:isn't 40 GB enough for applications? (4, Informative)

Unit3 (10444) | more than 4 years ago | (#31946502)

They are huge for larger applications. Database servers, for instance, can see performance increases in the magnitude of 10-20x the number of transactions per second when using a scheme like this for datasets that are too large to fit in RAM.

Re:isn't 40 GB enough for applications? (2, Insightful)

kgo (1741558) | more than 4 years ago | (#31946946)

Yeah, but if you've got some 'enterprise-level database' with those sort of transaction requirements, you can probably justify the purchase of SSDs. It's not exactly like you're building that system from craigslist parts...

Re:isn't 40 GB enough for applications? (1)

Amouth (879122) | more than 4 years ago | (#31947092)

Speak for your self.. some companies do not want to spend the money required to do it right.. but rather would have you spend more time than the equipment cost putting something crazy together to make it work.

Re:isn't 40 GB enough for applications? (1)

Annirak (181684) | more than 4 years ago | (#31946594)

The idea is that I want to automate the management of which data are stored on the SSD. Entire files need not be cached, only their hot sectors. By using a SSD directly, you lose the benefits of keeping infrequently accessed bits of data out of the SSD, reserving it only for the most commonly accessed data.

Doing it this way makes the majority of the filesystem perform better, whereas using a normal SSD+HDD configuration requires you to actively manage where your most frequently used data are.

Re:isn't 40 GB enough for applications? (1)

MobyDisk (75490) | more than 4 years ago | (#31946716)

t I doubt the benefits are that large over a normal SSD + HDD configuration.

Which doesn't work for laptops. :-(

Most laptops can only fit a single drive. I would love to have an SSD for faster build times, but a 40GB SSD is useless in my laptop since the second drive would have to be an external. But a 300GB drive with 16GB of integrated flash might give me a single drive with the performance boost that I am looking for.

Re:isn't 40 GB enough for applications? (1)

mickwd (196449) | more than 4 years ago | (#31946810)

I'd agree with this. Get that Intel SSD and stick /usr on it, together with any other read-mainly filesystems (maybe the root filesystem too, if you have stuff like /var on separate partitions).

As well as faster reads, the biggest gains are in seek times, so it'd be helpful to have your home directory and all it's "dot" config files on there too (especially when starting up something like Gnome or KDE). However, if you're gonna fill your home directory with tons of stuff, then stick your home directory itself on the SSD, but mount a hard-drive-based partition into it and keep your space-using stuff on here. For example:

/            SSD
/var         Magnetic
  :
[Paging]     Magnetic
/usr         SSD
/home        SSD
/homestorage Magnetic

For user X, create /homestorage/X, then create a symlink /home/X/stuff -> /homestorage/X.

Yes it's a slight pain to keep most of your personal stuff in a subdirectory below your home directory, but it's hardly going to kill you.

Holy fuck, just use partitions. (0)

Anonymous Coward | more than 4 years ago | (#31946278)

Get one drive of each type. Stick them in your computer. Create partitions for the main directory hierarchies. Put /, /boot, /bin, /etc, /usr, and other relatively-static hierarchies on the SSD drive. Put /home, /var, and other frequently-modified directories on the magnetic disk drive. There, you've got the caching you want.

Re:Holy fuck, just use partitions. (1)

Unit3 (10444) | more than 4 years ago | (#31946512)

Err, no you don't. That's not caching at all, and doesn't help with datasets that don't fit on the SSD.

This is a shortsighted kludge with limited uses, and not at all the elegant solution the poster was asking for.

Sure it's caching. And it's not a "kludge" at all. (0)

Anonymous Coward | more than 4 years ago | (#31946700)

Sure it's caching. You're storing frequently-used data on a faster medium.

It's actually better than caching, because in this case you don't have to write back the data to the spinning-disk drive when there are changes, since the SSD drive itself is non-volatile. An added benefit of that is that there's no need to keep the SSD-stored directories on the spinning media, freeing up more space there.

Call it a "shortsighted kludge" all you want, but this general technique has been used very effectively for decades by people and organizations working with absolutely huge data sets. It's a proven technique that dates back to mainframes and microcomputers.

Hell, that's why this is so easy to do with UNIX-like systems. Disk drives then had a smaller capacity but faster access times, and were used to store frequently-accessed data (like system files and applications). User data was stored on tape, since it was comparatively plentiful, even if somewhat slower. The very earliest UNIX implementations split their filesystem hierarchy over multiple storage devices with differing capabilities.

Re:Sure it's caching. And it's not a "kludge" at a (1)

TheRaven64 (641858) | more than 4 years ago | (#31947024)

Sure it's caching. You're storing frequently-used data on a faster medium.

That's not what caching means. It comes from the French word for 'hidden' and the fact that it is not directly addressable is the important part of the definition. A cache is not just a faster medium, it's a faster medium that is hidden from the user / programmer and is used to accelerate access to the slower medium.

Re:Sure it's caching. And it's not a "kludge" at a (1)

EvanED (569694) | more than 4 years ago | (#31947072)

Sure it's caching. You're storing frequently-used data on a faster medium.

You're also storing a lot of infrequently-used data on a faster medium, and a lot of frequently-used data on the slower medium. How is that better than using the faster medium only for frequently used data?

As it happens, for many of my workloads anyway, I'd expect to see bigger benefits by storing some of my data directories on SSDs than storing /usr. Which directories? Well, they're sort of scattered all around. Why should I have to go through and figure out what things I use all the time? How can I even determine that (I don't know what files programs are opening behind my back). Figuring out that sort of thing is exactly what computers are good at.

I's actually better than caching, because in this case you don't have to write back the data to the spinning-disk drive when there are changes...

That's not a problem. Writes are already delayed because they can be buffered; if you cache in a SSD, they can be delayed even further before you write them back to the magnetic drive.

An added benefit of that is that there's no need to keep the SSD-stored directories on the spinning media, freeing up more space there.

Whoopde do. Even if you got a 100 GB SSD, duplicating that space on, say, a 1 TB hard drive would cost under $10. Considering that the SSD would be in the vicinity of $400, that extra $10 in lost space isn't exactly something to cry about.

Re:Holy fuck, just use partitions. (1)

dow (7718) | more than 4 years ago | (#31946696)

Seems rather obvious to me... I've always had my / on the first part of the fastest disk, and had /usr, /home and /var elsewhere. It makes it far easier to upgrade or mess around with multiple distributions when its partitioned like this, but Linux distributions often dont encourage it as much as they should.

You probably just have one or two directories that are huge, such as your Southpark collection and maybe some porn. Just put everything on the SSD, and have the few large directories of rarely used sequentialy accessed files on a cheap green drive. If you don't already have all these sorts of files grouped together, changing your way of working won't kill you.

I'm currently using Windows7 off an SSD, and won't ever be without one in the future. There is no way that 200 quid spent on a processor or ram or graphics card can make a system so responsive as only 100 quid spent on an SSD. My other drive in this system is a Velociraptor, but in hindsight I'm thinking perhaps I should have bought a slower larger drive. Then again, I guess that is what NAS is for :)

ZFS (5, Informative)

Anonymous Coward | more than 4 years ago | (#31946288)

ZFS can do this (http://www.solarisinternals.com/wiki/index.php/ZFS_Best_Practices_Guide#Separate_Cache_Devices) but I don't know about zfs-fuse

Buffers? (1)

DoofusOfDeath (636671) | more than 4 years ago | (#31946292)

I hate to sound dumb, but isn't what you're describing basically file system buffering that OS's have been doing for many decades now?

Re:Buffers? (0)

Anonymous Coward | more than 4 years ago | (#31946432)

Yes, but I guess this is supposed to be persistent across power cycles.

I.e. recently/often accessed files/sectors/blocks are kept on the fast SSD, whereas data that is accessed more seldom is stored on an ordinary HD. Assuming the caching system does the right thing (it probably needs to be somewhat smarter than just a LRU), you would benefit from the SSD-cache mainly during boot and directly afterward when data is not yet cached in RAM.

Re:Buffers? (0)

Anonymous Coward | more than 4 years ago | (#31946490)

Almost. Typically files are cached from the spinning disk straight to RAM. The author wants to cache the spinning disk to the SSD, then to RAM. In other words, think of the SSD as a very large cache for the spinning disk. The advantage is that the author wouldn't have to decide which files to write to the small SSD or the large disk, the caching algorithm would do that automatically.

Re:Buffers? (2, Informative)

MobyDisk (75490) | more than 4 years ago | (#31946766)

No.

You would buffer on an SSD differently than your would do it in memory. Memory is volatile, so you write-back to disk as fast as possible. And whenever you cache something, you trade valuable physical memory for cache memory. With an SSD, you could cache 10 times as much data (Flash is much cheaper than DRAM), you would not have to write it back immediately (since it is not volatile), and the cache would survive a reboot so it could also speed the boot time.

Re:Buffers? (1)

ras (84108) | more than 4 years ago | (#31946978)

Not really. There are a couple of applications for SSD's. One is to speed up boot times. Obviously a RAM cache is useless in that application.

Another is if you want to speed up a transaction server (one that is writing as much as it is reading), then the answer is again no. Think of the battery backed up RAM cache RAID arrays have. Those caches are there for a reason. RAM can do read caching, but it can't do write caching and still be secure across power failure.

My interest is in the second application, and like the guy who wrote this article I have been patiently waiting for someone to produce a nice local file system cache for Linux. So far I haven't seen anything that comes close.

ZFS L2ARC (5, Informative)

jdong (1378773) | more than 4 years ago | (#31946320)

Not Linux per se, but the same idea is implemented nicely on ZFS through its L2ARC: http://blogs.sun.com/brendan/entry/test [sun.com]

Re:ZFS L2ARC (2, Interesting)

Anonymous Coward | more than 4 years ago | (#31946492)

Not Linux per se, but the same idea is implemented nicely on ZFS through its L2ARC: http://blogs.sun.com/brendan/entry/test [sun.com]

Swapcache on DragonFly BSD 2.6.x was implemented for this very reason IIRC.

http://leaf.dragonflybsd.org/cgi/web-man?command=swapcache&section=ANY

You don't need it (0)

Anonymous Coward | more than 4 years ago | (#31946330)

Install the operating system and key applications to the SSD and use your standard harddrive for all the data storage

Re:You don't need it (1)

MobyDisk (75490) | more than 4 years ago | (#31946750)

Laptops can only fit a single drive.

Re:You don't need it (1)

countach (534280) | more than 4 years ago | (#31946974)

A lot of people replace the DVD drive with a 2nd drive. There are kits available.

Re:You don't need it (1)

MobyDisk (75490) | more than 4 years ago | (#31947258)

For some laptops, yes, that hack would work. But that isn't a general purpose solution like the SSD + HDD combo that the submitter proposed.

Counter-Productive (1)

Bucaro (758451) | more than 4 years ago | (#31946338)

By having the SSD act this way, you will lower the lifespan of the drive by unnecessarily depleting the write-cycles with continued optimization

Re:Counter-Productive (1)

sourcerror (1718066) | more than 4 years ago | (#31946408)

If you want to use it to boost booting, you will mostly read it.

Re:Counter-Productive (1)

JessGras (953965) | more than 4 years ago | (#31946430)

But what is the point of accelerating an access which happens only rarely?

Re:Counter-Productive (0)

Anonymous Coward | more than 4 years ago | (#31946650)

To you but mabye there are peole that's boot often. mabye people that whis to save energie and reboots the computer.
people doing development that brings the system down every now and then and lots of other people for varius reasonshttps://help.ubuntu.com/community/WifiDocs/Driver/bcm43xx/Feisty_No-Fluff

Re:Counter-Productive (0)

Anonymous Coward | more than 4 years ago | (#31946528)

So what if he wants to blast his drive... The idea is sound. It should be generic though and be able to use any drive.

So you could have say a cheap usb drive and a slow HD...

MS did this with their 'readyboost' and USB thumb drives. Doesnt work worth a damn because they reset the cache everytime you reboot. So you spend 10 mins waiting for it to fill 4 gig with junk.

Re:Counter-Productive (2, Insightful)

Unit3 (10444) | more than 4 years ago | (#31946534)

Define "unnecessarily". Given current SSD costs and depletion rates, it's probably completely acceptable to replace an SSD used as an intermediary cache in front of a large spindle-based array every couple of years.

Just because it's not useful to you, doesn't mean it's not useful.

Re:Counter-Productive (1)

MobyDisk (75490) | more than 4 years ago | (#31946672)

I don't understand what you mean. How does caching data on an SSD lower the lifespan on the magnetic drive? Or did you mean it lowers the lifespan of the SSD? Which it would do, but it shouldn't be any more than any other use of an SSD. SSD lifespan is really not an issue any longer. (Intel claims over 10 years on their drives. Other manufacturers are claiming similar timespans).

Could you clarify?

Re:Counter-Productive (3, Informative)

pwnies (1034518) | more than 4 years ago | (#31946838)

Sadly, nowadays this is a myth. Current MLC and SLC SSD's have (on average) 10,000 and 100,000 writes (respectively) before any bitwear will occur. While this number is small, remember that all modern mainstream SSD's have wear leveling algorithms built into the controller. Intel rates their drives' minimum useful life at 5 years [pdf link - page 10] [intel.com] , with an estimated life of 20 years. Note that this number is based on 20GB of writes per day, every day. SSD's nowadays will have no problems with acting as a cache for the system.

Re:Counter-Productive (1)

TheRaven64 (641858) | more than 4 years ago | (#31947052)

And if you're only using it for cache, who cares if it wares out in a few years? Your data is all safely on the other disk(s) and in a few years the SSD won't be worth much anyway.

Dear Slashdot : (0)

Anonymous Coward | more than 4 years ago | (#31946356)

How much was Slashdot paid for this plug for Silverstone?

Hackers want to know.

Thanks in advance.

Nick Haflinger.

Re:Dear Slashdot : (0)

Anonymous Coward | more than 4 years ago | (#31946396)

Actually $0.0000. If you want to post your own plug, please visit http://slashdot.org/submission [slashdot.org]

And thanks for asking.

Slashdot

I get it (0)

Anonymous Coward | more than 4 years ago | (#31946372)

He wants the OS to intelligently (and automatically) use an SSD to store the frequently used files from his larger spinning hard disk. It's a great idea and surely Windows will do it soon enough (as much as I hate to say it).

Re:I get it (1)

drsmithy (35869) | more than 4 years ago | (#31946802)

He wants the OS to intelligently (and automatically) use an SSD to store the frequently used files from his larger spinning hard disk. It's a great idea and surely Windows will do it soon enough (as much as I hate to say it).

It basically does already (ReadyBoost). I can't imagine there would be much work involved in modifying it to use arbitrary disks like SSDs instead of just thumbdrives. Indeed, I wouldn't be at all surprised if there's a simple Registry setting that decides what devices can and can't be used for ReadyBoost.

Hybrid drives were the way to go (1)

engrstephens (1714758) | more than 4 years ago | (#31946400)

I think SSD will always be behind Magnetic. Ive dreamed of building an SSD controller that had a magnetic backend. Popin SD cards to expand the cache. OS doesnt notice that your drive is really 2 drives. I'm going to get flamed but I do think M$ had it right when they backed hybrid drives!

Re:Hybrid drives were the way to go (1)

MobyDisk (75490) | more than 4 years ago | (#31946556)

I agree, and I've been thinking along these same lines. Existing caching algorithms are not sufficient for this purpose. Iin-memory caches, they attempt to write-back the data as soon as possible since the memory is volatile. You would want an algorithm specifically made for non-volatile caching.

I imagine a 500GB hard drive with 32GB of SSD. The caching algorithm would be smart enough to keep 2 cache areas: 16GB reserved for long-term read-only things like OS files. No matter what disk thrashing goes on, don't purge these. This keeps boot times fast. The second 16GB would be a normal write-back cache. Browser cache, temp directory, pagefile, etc. So if I edit a video, compile some code, or play a game - I get SSD performance. The drive might decide to write that back to the magnetic disk immediately, or in 5 minutes, or tomorrow afternoon. Yes - I could even power-down the computer and it could finish the write-back another day. There is no need to write-back unless the rive is idle, or if it is out of cache space.

The good thing is that this could be done in software. I had hoped that Windows ReadyBoost would do this. But from what I've read, even though it is improved in Windows 7 it still is only useful if you have low memory. I am not sure if that is because USB is slow, or because ReadyBoost still operates like an in-memory cache (since the USB flash drive could be removed). I would love to have a check-box that says "I promise I won't remove this drive - treat it as non-volatile" to get it to do that. I bet someone could write a driver to do it.

Good idea, lousy implementation. (1)

SanityInAnarchy (655584) | more than 4 years ago | (#31947048)

Kind of like the current idea of pushing the wear-leveling back to the drives. This is something the OS can do, and it's a case where flexibility matters -- it's not something I want in a black box inside a drive controller.

OSDI (1)

jameson (54982) | more than 4 years ago | (#31946416)

The OSDI deadline is in August; plenty of time to implement this, write it up, and get a publication at a top research conference out of it!

bcache (5, Informative)

Wesley Felter (138342) | more than 4 years ago | (#31946420)

http://lkml.org/lkml/2010/4/5/41 [lkml.org]

I'm a little surprised at the lack of response on linux-kernel.

Solaris and DragonFly have already implemented this feature; I'm surprised that Linux is so far behind.

Re:bcache (5, Informative)

Kento (36001) | more than 4 years ago | (#31946558)

Hey, at least someone noticed :)

That version was pretty raw. The current one is a lot farther along than that, but it's still got a ways to go - I'm hoping to have it ready for inclusion in a few months, if I can keep working on it full time. Anyone want to fund me? :D

Re:bcache (1)

CyprusBlue113 (1294000) | more than 4 years ago | (#31946832)

If it were a block device wrapper along the lines of md, I'd be interested.

Have a project at the moment where I'd *love* to be able to specify tiers of storage (say md volumes), and have writes go to the highest priority, and blocks trickle down to the lowest based on usage.

Sort of like a specialized CoW.

Re:bcache (1)

Kento (36001) | more than 4 years ago | (#31947080)

This should do what you want - with the caveat that I haven't thought about multiple tiers, but I can't imagine that being that hard to add.

It's currently written so there's a systemwide pool of block devices that are used for cache, and all cached data is spread around them, regardless of where it came from. It wouldn't take much work at all to change that though, if there was something that'd benefit.

Re:bcache (1)

rayvd (155635) | more than 4 years ago | (#31946980)

Any plans to add in write cache support? I'm thinking along the lines of putting ZFS's ZIL on SSD's. Really makes NFS in sync mode much quicker.

Re:bcache (1)

Kento (36001) | more than 4 years ago | (#31947106)

Yep, that's on the list.

Re:bcache (1)

pydev (1683904) | more than 4 years ago | (#31946732)

You shouldn't be surprised that "Linux is so far behind"; we like it that way. If we thought that what the Solaris or DragonFly engineers are doing was important, we'd be using their systems instead.

Re:bcache (1)

TheRaven64 (641858) | more than 4 years ago | (#31947082)

I'm surprised that Linux is so far behind

Obviously you are either unfamiliar with Linux, or unfamiliar with all non-Linux operating systems except perhaps Windows and maybe Darwin.

Waste of time (5, Informative)

onefriedrice (1171917) | more than 4 years ago | (#31946442)

What a waste of time. Just put /home on a magnetic disk and everything else on the SSD. This way, you can get away with a small (very affordable) SSD for your binaries, libraries, config files, and app data, and use tried and true magnetic for your important files. Your own personal files don't need to be on a super fast disk anyway because they don't get as much access as you would think, but your binaries and config files get accessed a lot (unless you have a lot of RAM to cache that, which I also recommend). I've been doing this for over a year and enjoying 10 second boots, and instant program access coldstarts (including openoffice and firefox).

I personally fit all my partitions except /home in only 12.7GB (the SSD is 30GB). Seriously, best upgrade ever. I will never put my root partition on a magnetic drive ever again.

Some ideas, from "The world of Windows" though (0)

Anonymous Coward | more than 4 years ago | (#31946760)

I stash these items on my SSD (4gb GIGABYTE IRAM SSD #1 - this can actually BOOT AN OS though, so others know):

====

1.) WebBrowser caches

2.) %Temp% + %Tmp% ops

3.) Event Logs

4.) Print Spooler

5.) cmd.exe %Comspec%

----

And, on my 3gb CENATEK RocketDrive:

1.) Pagefile.sys (for nearly the ENTIRE SSD in size of it)

====

Seems to all work out well, for better performance... how so?

Well - not just because those items benefit by mostly being smallish files inside folders, which tend to "increase speed" (less latency in seeks mostly & NO head movements either as in mechanical disks) but, also because I am "offloading" my main C: drive (bootdrive in Windows for those "not in the know" on Windows, & yes, there are those folks out there @ times, albeit rarely), making it do LESS WORK also!

APK

P.S.=> Your ideas aren't 1/2 bad either for Linux though... good job! apk

Re:Waste of time (1)

MobyDisk (75490) | more than 4 years ago | (#31946826)

Just put /home on a magnetic disk and everything else on the SSD

Try jamming two hard drives into a laptop. :-(

Re:Waste of time (1)

Logic (4864) | more than 4 years ago | (#31946858)

My primary Linux laptop is an Inspiron 1721, with two mirrored drives.

Re:Waste of time (1)

MobyDisk (75490) | more than 4 years ago | (#31946906)

Can I have it? :-)

I think my next laptop will have space for 2 drives. I might even be willing to have my optical drive be external. As a developer, I need the disk space and a single SSD just can't cut it unless I want to spend close to $1000 on the drive.

Re:Waste of time (0)

Anonymous Coward | more than 4 years ago | (#31946994)

take out the CD/DVD drive and you can fit a hard drive in there instead.

Re:Waste of time (0)

Anonymous Coward | more than 4 years ago | (#31947136)

A nice idea (as far as caching). I've often thought that using two drives for my system is a pretty big waste of one of the drives. I used two drives because when installing a new version of Linux, I didn't want to hose all of my user preferences, email, configuration, etc. So what I would do is keep /home on a second drive, then when I install a new version of Linux, I blow away the drive with the root partition, install a new version. After installation, I toast the /home partition on the / partition, then ln -s /seconddrive/home /home and one link brings all data back (really it never moved), yet its there. You never have to move email, background pictures, fonts or other user configuration again. Extra binaries and system data files are also kept on the second drive, while system files (libraries, updates, graphics, etc) are all kept on the root partition. For a 'fully loaded' Linux setup, 20GB is lots (since extra binaries and data files are kept on drive 2). Go ahead and apt-get to your hearts content, and you will have a hard time filling 20GB. Very slick. You always get a new version whenever you upgrade your system (which on Ubuntu for me is every 6 months) all new software, and yet all of your stock configuration is fast and easy (about 4 key presses for everything), and you never lose any email, even decades later.

Configurability (0)

Anonymous Coward | more than 4 years ago | (#31946454)

Set the root and swap partitions on the SSD, set up the kernel so the hibernation image is stored on the SSD, and set the /home partition on the HDD.

Not as elegant as what your asking for, and your solution could easily be implemented, but the solution outlined above should get most people the same results. Any directories you don't want on the SSD you can just assign to the HDD.

Ahh, configurability - don't you love Linux? :D

Go for Hardware implemented Caching (1, Informative)

iammani (1392285) | more than 4 years ago | (#31946464)

Hardware implemented caching is the way to go. There are 'hybrid drives' available now, which automatically cache disk access to SSD. These are very specific for the task and way more efficient than any software implementation.

Re:Go for Hardware implemented Caching (1)

Wesley Felter (138342) | more than 4 years ago | (#31946686)

I see that you haven't actually used hybrid hard drives, because they're nothing like what you describe. AFAIK only Samsung ever made them, and they've now been discontinued. The HHD itself didn't perform caching; it relied on Vista to manage the flash, which didn't really work out when no one bought Vista. HHDs also included laughably small (256MB) and slow flash that would get totally owned by the smallest slowest SSD today.

Re:Go for Hardware implemented Caching (1)

iammani (1392285) | more than 4 years ago | (#31947262)

Mmmm, interesting. I guess I my understanding about how these 'hybrid drives' work is wrong. That makes me wonder why isnt a completely hardware implemented ssd cache available? Is there a technical reason why this is not possible? Wouldnt this be faster than a software implemented one?

Working on this (1)

pmjordan (745016) | more than 4 years ago | (#31946580)

I've actually been working on this off-and-on for a while, I'm hoping we can release some beta code soon. Currently developing it on Linux, but planning to release OSX and Windows versions, too. We're caching reads and writes, and only the blocks that are most frequently used, plus various other SSD-relevant optimisations. The block allocation logic is pretty complex (and I'm too busy with work), which is why it's been taking so long.

But why? (1)

Eudial (590661) | more than 4 years ago | (#31946642)

The oldest and simplest solution is to mount partitions from a small fast disk where you want fast read/write speeds, and partitions from slower disks everywhere else. Works quite well, too.

Do it yourself. (0)

Anonymous Coward | more than 4 years ago | (#31946646)

This wouldn't be hard to do with a little smart configuring.
When you install linux, make sure that /home /boot and /bin end up on the SSD (probably / would be fine), and mound the spinning HDD somewhere else. I call mine /data.
When you rip your DVDs, or whatever, put them on /data.
When you 'sudo apt-get' a new application, it goes onto the SSD, without you having to do anything special.

No cache implementation rewrite necessary.

People forgot the low-level Linux stuff quickly. (2, Informative)

guruevi (827432) | more than 4 years ago | (#31946740)

First of all, you can do this with ZFS which is newer tech and works quite well but is not (ever going to be) implemented in the Linux kernel

For lower tech, you can do it the same way we used to do back when hard drives were small. In order to prevent people from filling up the whole hard drive we used to have partitions (now we just pop in more/larger drives in the array). /boot and /var would be in the first parts of the hard drive where the drive was fastest. /home could even be on another drive.

You could do the same, put /boot and /usr on your SSD (or whatever you want to be fastest - if you have a X25-E or another fast writing SSD you could put /var on there (for log, tmp etc. if you have a server) or if you have shortage of RAM make it a swap drive. If you have small home folders, you could even put /home on there and leave your mp3's in /opt or so.

dm-cache (3, Informative)

Gyver_lb (455546) | more than 4 years ago | (#31946782)

google dm-cache. Not updated since 2.6.29 though.

I was just wondering the same thing (1)

RelliK (4466) | more than 4 years ago | (#31946844)

Windows 7 (and I think XP) has ReadyBoost. I haven't been able to find anything similar for Linux. It is also not clear how much difference ReadyBoost makes. The only benchmark I was able to find uses a crappy USB flash drive [pcstats.com] . I was wondering how much difference something like the 80GB x-25m would make. There is clearly potential for huge gains as MaxIQ benchmarks show.

This would be an awesome speedup if it was supported: just add a 40-80GB SSD for swap & file cache, and gain a massive performance boost over a standard cheap 7200RPM drive. Given that the price per GB for SSD is likely to stay very high for the foreseeable future, this seems like the best way to go. If only there was OS support for it...

The alternative (that I'm waiting for) is for SSD prices to drop.

Re:I was just wondering the same thing (1)

MobyDisk (75490) | more than 4 years ago | (#31947232)

It is also not clear how much difference ReadyBoost makes

Keep searching. There's lots of other benchmarks, and they all same the same thing. It helps if you have an old machine with insufficient memory.

ReadyBoost doesn't really do what the author wants. Windows treats ReadyBoost as a write-through cache like it treats memory. It assumes you might unplug the drive at any moment. It won't speed up boot time, and it won't speed up writes. It won't place the swap file on there either. I'm not sure if you could tell Windows to use a regular SATA drive for ReadyBoost - that might be an interesting experiment, but it still won't use it effectively.

Thread summary (2, Funny)

Goaway (82658) | more than 4 years ago | (#31946904)

"If Linux doesn't already do it, you don't need it anyway!"

Puppy Linux (1)

technosaurus (1704630) | more than 4 years ago | (#31947242)

Puppy Linux implements periodic syncing to its save file (ext2,3,& 4) and the times can be adjusted. This was initially implemented when flash drives were less reliable, but still useful if you want to reuse old equipment or if you need to do a lot of read/write intesive operations.

Load More Comments
Slashdot Login

Need an Account?

Forgot your password?