Beta

Slashdot: News for Nerds

×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Best Solutions For Massive Home Hard Drive Storage?

timothy posted more than 4 years ago | from the watch-and-release dept.

Data Storage 609

i_ate_god writes "I download a lot of 720/1080p videos, and I also produce a lot of raw uncompressed video. I have run out of slots to put in hard drives across two computers. I need (read: want) access to my files at all times (over a network is fine), especially since I maintain a library of what I've got on the TV computer. I don't want to have swappable USB drives, I want all hard drives available all the time on my network. I'm assuming that, since it's on a network, I won't need 16,000 RPM drives and thus I'm hoping a solution exists that can be moderately quiet and/or hidden away somewhere and still keep somewhat cool. So Slashdot, what have you done?"

cancel ×

609 comments

Define "massive" (4, Insightful)

jschen (1249578) | more than 4 years ago | (#32215108)

How much data constitutes "massive"?

Re:Define "massive" (1, Insightful)

Anonymous Coward | more than 4 years ago | (#32215184)

Since he's maxing out two machines worth of bays, if we assume three bays per machine and 3 TB disks, then massive probably means something more than 10 TB.

Re:Define "massive" (0)

sopssa (1498795) | more than 4 years ago | (#32215428)

I think 1 TB disks are more realistic. Price/Size ratio is a lot better with 1 TB disks. On the other hand I'm sure he can fill more than three bays per machine, so I think he means >10 TB too.

Re:Define "massive" (4, Informative)

svirre (39068) | more than 4 years ago | (#32215622)

My pricing indicate 2TB disks are slightly cheaper/GB than 1TB

Re:Define "massive" (2, Interesting)

BLKMGK (34057) | more than 4 years ago | (#32215664)

I've got room for 30TB of data storage in two machines for a total of 60TB. However I have only populated them to around 12TB right now, I don't add drives till I'm out of space! :-) Not what I would call massive yet but getting there!

Re:Define "massive" (0)

Anonymous Coward | more than 4 years ago | (#32215204)

I don't know, I didn't write "massive". But I can see 20 - 40tb within the next year. I modded a comment so I'm posting anonymous...

Something like this (5, Interesting)

fotbr (855184) | more than 4 years ago | (#32215128)

Do something like this. Put it in a case / box / cabinet of your own design since you don't need the rackmount capability.

http://blog.backblaze.com/2009/09/01/petabytes-on-a-budget-how-to-build-cheap-cloud-storage/ [backblaze.com]

Re:Something like this (4, Informative)

Anonymous Coward | more than 4 years ago | (#32215360)

Do something like this. Put it in a case / box / cabinet of your own design since you don't need the rackmount capability.

http://blog.backblaze.com/2009/09/01/petabytes-on-a-budget-how-to-build-cheap-cloud-storage/ [backblaze.com]

If possible use something like ZFS (or btrfs if you feel confident about it) so that you get checksumming data protection.

If you're going to put all your eggs in one basket, you better watch that basket very carefully.

The creators of that kit don't use any kind of redundancy with-in the box because their custom software stack handles replication (kind of like Google FS / Hadoop FS).

Re:Something like this (0)

Anonymous Coward | more than 4 years ago | (#32215408)

A linux box?
Many hd in your case and nfs, samba, sftp and so on to access everyrhing with one clik.
Nfs is the best way...
HUjuice

Re:Something like this (3, Insightful)

EdIII (1114411) | more than 4 years ago | (#32215500)

The only caveat about that particular solution is the lack of redundant power, poor serviceability in the rack (may not apply like you said), and slow speed.

Their solution achieves the density it does because they are using SATA multiplexers, but that effectively creates bottlenecks and lowers overall speed. It works for BlackBlaze's application requirements, but YMMV.

Protocase.com makes the enclosure and will sell it to you for a pretty reasonable price. Getting all the parts is not such a big issue. I think we estimated we could build one without drives for less than $3k.

If you don't have it in a rack, then serviceability will be a lot better for sure. Rackmount solutions require cable management and heavy duty slide rails, and wide aisles, in order to gain access to the drives. The backplanes are parallel to the ground, facing up, and require taking the top off to access. Not exactly IT friendly.

Since the person in the article is not using this in a datacenter, cooling is going to be an issue. I suspect BackBlaze survives due to hot-cold aisles and plenty of airflow. Sticking one of those enclosures in a closet without ventilation/cooling is a recipe for disaster.

Re:Something like this (0)

Anonymous Coward | more than 4 years ago | (#32215586)

Nice CAD renderings.. but have they built it ?

Look at the DroboPro (0)

Anonymous Coward | more than 4 years ago | (#32215134)

I really like the DroboPro. It's pricey but you can install up to 8 drives, upgrade drives as necessary to increase storage, and it's quiet/fast/low-power. You'll still need a file server since it's a SAN device, not a NAS device, but I'm happy with my 16TB configuration.

Re:Look at the DroboPro (1)

idobi (820896) | more than 4 years ago | (#32215174)

I second the drobopro solution. Add a droboshare for network access

Re:Look at the DroboPro (0)

Anonymous Coward | more than 4 years ago | (#32215252)

If the drobopro is outside your price range they came out with the drobo FS www.drobo.com

Re:Look at the DroboPro (3, Interesting)

binarylarry (1338699) | more than 4 years ago | (#32215284)

Good fucking god, $700 for the Drobo FS?

You could build a capable home server box AND buy some of the drives for that much.

Re:Look at the DroboPro (5, Informative)

Firehed (942385) | more than 4 years ago | (#32215426)

Having done that in the past, I'll say that buying a Drobo was worth the cost. Granted, I hunted around a bit to get a good sale price (it's not too difficult... though the FS is brand new so maybe not on that model yet), but unless you really enjoy tinkering with getting samba shares set up and working properly, sometimes it's just easier to buy your sanity.

Don't get me wrong - I wish they were cheaper. But their system worked better and more reliably than anything I ever put together, and I'm by no means incompetent. And their BeyondRaid tech, while proprietary, is pretty damn cool and works incredibly well. Being able to mix drives and not waste tons of storage space is a huge advantage that (as far as I know) I'm not going to get anywhere else.

Just a happy customer, not an employee or anything like that.

Re:Look at the DroboPro (1)

DigiShaman (671371) | more than 4 years ago | (#32215626)

I second that.

Unless you're on the cheap and/or like tinkering with homebrew solutions, your better off using a professional product like Drobo. For most people that have simple storage requirements, the idea of "Set-it-and-forget-it" is the mantra they want to adhere to.

Re:Look at the DroboPro (1)

mindstrm (20013) | more than 4 years ago | (#32215646)

I'll second that - while I don't use a drobo - it's on the purchase list.

I'm highly technically competent - I use an opensolaris fileserver with ZFS and some terabyte drives - and sure, I can expand it and do all kinds of cool things - but in the end, I could get the same features I get there (the ones I actually use) out of a drobo, with less hassle.

With my fileserver I have to know a bunch of stuff about how to manipulate it to expand it... with a Drobo, you have to watch some blinkenlights and just pop in bigger drives when you need to grow. It locks dives when you shouldn't remove them, and is dead simple to use. From all my research, it's unparalleled in the ease of use department for joe average.

Re:Look at the DroboPro (1)

cynyr (703126) | more than 4 years ago | (#32215630)

www.qnap.com/pro_detail_hardware.asp?p_id=127 and you get 4 drive slots, for 600, with a better feature set than the Drobo. Dual GbE nics, 2 esata ports, 26W active w/ 4 drives vs 56W with the drobo. Anyways, thought i'd point out that the drobo is quite a bit overpriced like you mention.

Re:Look at the DroboPro (1)

BLKMGK (34057) | more than 4 years ago | (#32215650)

In fact a friend of mine DID! His Drobo fucked up so often he nearly threw it out a window! He pinged me one night raging about it yet again. I told him to head for Fry's and when he got back with $600 worth of hardware he was good to go. His hardware booted unRAID luckily and when he was done and the parity all setup and the disks built he had a solid system that replaced TWO Drobo for $600. Sold his Drobo to folks on Amazon and was ahead in the money dept! The software he's running will support 15 data drives same as mine and I've got two of them. I see zero point in owning the Drobo, too much money for the hardware and the headaches. Took me years to get him off that crap and he is way happier now!

Re:Look at the DroboPro (1)

mr.big_pig (521727) | more than 4 years ago | (#32215540)

I third this. We've sold a lot drobos to customers. Easy to setup, easy to expand.

Cheap NAS (3, Informative)

Bodhammer (559311) | more than 4 years ago | (#32215136)

DLink - DNS-323 with two WD 1 TB Green Drives. Quiet, works out of the box and is also Linux hackable if you feel the need.
Enjoy!

Re:Cheap NAS (1)

teh31337one (1590023) | more than 4 years ago | (#32215150)

Best option. Build a NAS server

Re:Cheap NAS (1)

Sandman1971 (516283) | more than 4 years ago | (#32215638)

I second this. I just bought my 2nd DNS 323 after the first one got full. (its cheaper to buy 2 DNS 323 which are 2 bays, compared to getting the 4 bay version). I shoot a lot of pics, and at 20-25 meg per RAW file, it adds up quickly. The first is stacked with 2x1.5 TB drives, the 2nd with 2x2TB drives. I've got them running in RAID 1 for hardware redundancy. Great little devices and cheap (150$ Canadian). With the latest firmware you can even run NFS, and a native bittorrent client.

My 2 cents (2, Informative)

ars vitae (1272902) | more than 4 years ago | (#32215146)

Personally I'm using a Synology solution at the moment, for my NAS. They offer a relatively low cost feature rich hardware, with low power and depending on the HDD's you use, lower power consumption than that of a always on PC.I've been thinking about later on upgrading, since a general rule of thumb, you can never have to much storage. For HD BluRay images I would recommend making sure the network isn't the bottleneck and use gigabit ethernet, as I'm finding on my aging 10/100 switches it's not cutting it. xvid's and MP3 streaming it seems to be fine.

Re:My 2 cents (1)

Low Ranked Craig (1327799) | more than 4 years ago | (#32215190)

Agreed on Synology. A little more expensive than build it yourself, but they have solutions that scale easily to 20TB. Synology + 1TB or 2TB WD Green drives + GB Ethernet and you can have 20+ TB of media on line very quickly.

Paranoia (5, Funny)

Anonymous Coward | more than 4 years ago | (#32215148)

So Slashdot, what have you done?

Why? What have you heard??

ZFS (2, Interesting)

mikael_j (106439) | more than 4 years ago | (#32215152)

My personal storage solution consists of a 4U rack case with a computer with a c2d CPU, gig-E NIC, a few gigs of ram, a bunch of 7200 RPM disks and FreeBSD on the system disk (I also have the system disk mirrored just in case). All the storage disks are then pooled using RAIDZ. Pretty simple yet powerful. Just don't expect too much in the way of performance.

Re:ZFS (1)

sanosuke001 (640243) | more than 4 years ago | (#32215208)

I did something similar. I made a 4U machine, 4GB ram, basic cpu, gigE, 6x1TB HDDs and an old 60GB system drive. No RAID (as I wasn't going to mirror and didn't want to lose storage) but set up samba and access my drives from my windows box.

Re:ZFS (1)

RocketRabbit (830691) | more than 4 years ago | (#32215582)

You'll easily saturate gigabit ethernet with such a setup.

With ZFS performance is largely independent of storage media speed.

I did a similar thing with laptop drives in an external sata cage, over firewire, with FreeBSD and the sustained speeds were still really good - about equal to our $10,000 Dell Storage Server machine with Windows Storage Server 2003.

The production machine runs Opensolaris and is faster than nipple whiz.

Re:ZFS (4, Interesting)

0100010001010011 (652467) | more than 4 years ago | (#32215648)

ZFS + Solaris.

I have a standard ATX case with 4-in-3 [newegg.com] adapter from Newegg. I didn't get the more expensive ones with trays because I didn't need to hot swap.

I have 2x1TB drives in ZFS mirror for boot. 5x1.5TB drives in RaidZ as a tank and 2x200GB drives in mirror with a virtual block device for Xen Debian and Windows 7.

OpenSolaris is amazingly simple to use, if you're just doing your home network.

At the most basic level:
zpool create tank c5t0d0s0 c5t1d0s0
zfs sharenfs=on tank
zfs sharesmb=on tank
zfs shareiscsi=on tank

Now your new drives are all shared over NFS, SMB and iSCSI.

I keep looking for the old school full height 'desktops' at a bargain store or so. Search newegg. 3.5" external works just as well as internal.

This [newegg.com] has 11 3.5" bays and 3 x 5.25" bays. With a 4 in 3 linked above you could have 15 hard drives in a case for $100. Or if you care about hot swappability This one [newegg.com] has 20 hot swap bays (at 3x the cost).

If you want more performance, get some SSDs to work as the ZFS "cache".

How much is a lot? (3, Insightful)

jimmyfrank (1106681) | more than 4 years ago | (#32215162)

..., are the movies you download compressed at all? You say you run out of slots, how big are the drives you're putting in the slots? Personally, I let Netflix do the storing for me. I have a few TB's but never come close to filling it up.

Re:How much is a lot? (-1, Flamebait)

Anonymous Coward | more than 4 years ago | (#32215450)

Fuck your netflix and the horse it rode in on, get aids and die upside down you fascist cunt.
The OP is obviously talking about pirated material so you and your smug netflix ass needn't bother commenting.

Cheap solution (4, Funny)

tomhudson (43916) | more than 4 years ago | (#32215166)

  1. create a million gmail accounts (or buy them from spammers in bulk at 3 for a penny
  2. link the accounts as one big drive [wikipedia.org]
  3. 640k petabytes ought to be enough ...

Re:Cheap solution (1)

so-logical (802444) | more than 4 years ago | (#32215322)

Just don't forget the overhead to split every file into 25 MB chunks before storing... (according to the wikipedia article linked, 25 MB cap on file size). 25 MB ought to be enough for anyone, especially people working with 1080p video.

Re:Cheap solution (1)

Dice (109560) | more than 4 years ago | (#32215430)

Also don't forget the bandwidth required to push and pull all those HD videos.

Re:Cheap solution (1)

pclminion (145572) | more than 4 years ago | (#32215576)

It occurred to me once that a person could write a FS driver that did something like that, but my immediate next thought was "Naaah.. You're insane, man." The fact that somebody has actually done it makes me giggle uncontrollably. I have to stop letting sanity get in the way of ambition -- I coulda been there first!

Hmmm. With a couple thousand "reflector bots" I wonder how much data you could store in the form of IRC messages flying back and forth.

Re:Cheap solution (1)

icebraining (1313345) | more than 4 years ago | (#32215668)

Install ii [suckless.org] from Suckless and then you just need to do 'cat $file > server_dir/channel_dir/out'.

Filesystems are awesome.

Why do you need them available at all times? (5, Informative)

VinylRecords (1292374) | more than 4 years ago | (#32215176)

I used to work for ABC news and we never kept archive footage always accessible like you want. If we wanted something that was really old we'd have to dig it off a tape, an unplugged hard drive or powered off computer, or we'd have to find another news agency that had the footage and grab it off of a satellite feed. And this was a 24/7 TV news station responsible for national news programming where we would be tracking stories for years. If we didn't need a system where everything was instantly accessible then you needing it on an individual level might be overkill in my opinion.

I have over 30TB of music, movies, and raw video footage on my home computers and I just keep everything on separate external hard drives. I label the drives, back them up twice each, and then keep an index in a .txt file that is easy to search through. So if I want a 1080p backup copy of Blade Runner I search 'Blade Runner' in the .txt file and I see it's on drive 'A' and then I plug in drive 'A' and dump the movie on my computer. I also keep an external drive that has backups of every TV show I own on DVD. So if I want to watch The Wire then I plug in the external drive labeled 'TV' and have at it.

Re:Why do you need them available at all times? (4, Informative)

KahabutDieDrake (1515139) | more than 4 years ago | (#32215642)

First off, judging personal wants and needs by the way a giant corporation acts is hardly reasonable. ABC has cost/benefit to consider when trying to keep data available, and it's probably easier/cheaper to do it the way you say they do, rather than implement a fully digital, fully available storage system.

That being said, the solution is SIMPLE. If you have a bunch of hard drives with data you want, you put together a low end PC, install it into a server case, and fill it with hard drives and SATA controllers. When it's full, you build another one. You have 30tb of data, mostly not accessible. I have 10tb of data accessible from any internet connected computer on earth, and it's twice as much storage as I actually use. It cost me about 500$ to build and deploy a personal storage server, and it doubles as an HTPC. ( I already had most of the drives, and some parts) It's likely most people here have enough hardware laying around to implement a basic storage server. There really isn't any reason not to do it. As a bonus, since it's not a machine you need to access directly most of the time, you can hide it in a closet and forget all about it.

Sure, you could buy a premade NAS/SAN or stand alone data box. However, they are costly and not any more suited for the job than an old machine, or low end new system. At least, not in a personal environment. If you actually require robust data storage, I'd suggest a NAS, from any number of sources. But now we are talking about 4k worth of hardware, and requiring proper power systems to be added if you really want longevity out of it. However, that's overkill for a home storage solution, no matter how much data you have. Simply because you don't need enterprise class data serving, when only one or two computers are accessing the data.

If you don't know how to build and deploy a system with lots of drives accessible over a network, then you probably started at the wrong website for help. You want DELL/HP/IBM small office sales line.

well... (0)

Anonymous Coward | more than 4 years ago | (#32215182)

lets say 5 GB per movie - I have 200 in my downloaded collection and that is just movies people have recommended. Add in some tv shows and probably some music and lets exaggerate a little. So 5 TB would be a great start.
Buy a mobo with a lot of sata slots (read: as many as possible). Then load it full of 1.5 TB drives. run an outlet and a cat5e (or cat6) to the towel closet and hook it up on the top shelf. problem solved.

Solution (5, Funny)

Anonymous Coward | more than 4 years ago | (#32215192)

Don't worry, we'll be right over and take care of everything. You'll never have to worry about it again.

MPAA

P.S. My sister, Riaa wants to know if you're into MP3s

SATA port multipliers (4, Informative)

the eric conspiracy (20178) | more than 4 years ago | (#32215196)

SATA port multipliers - 5 to 1 for about $50 + 5 2 TB gives you 10 TB off 1 SATA port.

Re:SATA port multipliers (1)

jd (1658) | more than 4 years ago | (#32215354)

I'm fairly sure you can tunnel SATA over IP. Not sure where you'd get an IP-to-SATA adapter, though, or how much such a device would cost. But it would give him fully networked storage (ANY box on the network would see all drives as though locally connected) without having to use a network filesystem and potentially be as extensible as an IPv4 private network range. But it's heavily dependent on price as to whether it's worth it.

Re:SATA port multipliers (1)

Wesley Felter (138342) | more than 4 years ago | (#32215424)

Do port multipliers actually work? Last I heard they were quite unreliable. Besides, there are SOHO NAS boxes with 8 real SATA ports which gives you 16TB without any headaches.

Re:SATA port multipliers (4, Informative)

EdIII (1114411) | more than 4 years ago | (#32215546)

They work, but will slow the system down considerably. If you connect 5 drives up to 1 multiplier, the total speed you will get is the same as 1 drive hooked in directly. In otherwords, a bottle neck.

So technically it is possible to hook up 250 SATA drives into a single SATA RAID card, but you are not going to be that impressed with the performance.

My Solution: (1)

AtomicOrange (1667101) | more than 4 years ago | (#32215198)

Not the cheapest, nor the most expensive but heres my solution:

Thecus 4100 Pro

4 x 1.5 TB Hard Drives in RAID for 4.5 TB of Storage

The Thecus 4100 pro has lots of capabilities that I just haven't taken the time to use/learn yet. But it's been very effective for me so far. This is my first NAS build. It's great to be able to access media all over the house and now serves as my primary storage rather than the 4 x 1 TB's I had in my desktop and other externals that I had spread out.

Re:My Solution: (1)

AtomicOrange (1667101) | more than 4 years ago | (#32215216)

Oh and where I put it in the house: I keep it on the entertainment center next to the TV. It's not too loud, though the blue light from the interface may annoy some.

Re:My Solution: (1)

logistic (717955) | more than 4 years ago | (#32215506)

I took an old PIII 933 mhz tablet that stank up the place runing xp tablet edition.. Xubuntu runs surprisingly well if you don't mind. I use bittorrent (transmission) client for download. VNC to admin without going to basement. (VINO to serve) Runs samba fine. I used to use enclosures and make my own but WD etc external drives have better drive spin down etc.
Currently about 2 TB of USB 2.0 external storage. Plug in more drives as needed. Use separate drives. Just mount and use Samba to make a share.

To playback video over network the cheap 5400 rpm vanillia externals deliver plenty of speed I have no trouble with playback on my lan. mostly MPEG 2 and H.264 stuff I've downloaded with VLC on the client machines. (Might be different if more than one person is accessing.)

I'm reasonably savy but no linux command line god (hence VNC ). And I wanted something I could set up and just work without tweaking for months like the last time I tried Mythtv. I got samba running and a share available on my network with about 2 hours worth of work. There's always room to tweak. Powermanagemnt on old laptop is not perfect, I can't get it to turn off the backlight on the screen. But for the cost of the drives I got a working file server and a separate machine to download torrents in a total of about 4 hours worth of work without haveing to mess with the command line too much. The main issue is that vnc is not xubuntu's strength. and I had to install samba via apt-get.

5400 RPM (3, Interesting)

Anonymous Coward | more than 4 years ago | (#32215200)

I personally set up a downloadserver that also functions as a media server to stream the content to other devices. I put in a couple 5400 RPM 1,5 TB drives, they use less power, generate less noise and heat than a regular 7200RPM drive but since you're not running any applications of them, you won't really notice the difference in performance. Prices have gone down a bit so the sweet spot for $/GB might be at the 2TB mark now. If you don't want to go for an entire computer, maybe a NAS solution would be best for you, with the same 5400RPM drives. A NAS will have less room for the disks if you really want *massive* amounts of storage, and also you usually must purchase one + the disks. The PC you can build from spare parts lying around. I personally put gentoo linux on mine, but you also don't exactly need top of the line equipment for a nice windows XP install. The NAS however will have outputs directly for your TV and will take up less room and power.

Still, the key is 5400 RPM + 1,5/2 TB.

Sounds like one hell of a porn collection (4, Funny)

Trailer Trash (60756) | more than 4 years ago | (#32215210)

/. is definitely the place to ask...

Re:Sounds like one hell of a porn collection (1)

mattack2 (1165421) | more than 4 years ago | (#32215434)

Wow, that was quite far in the discussion before someone brought up that old joke.

I'd use it for extra storage for Tivos, personally.

Software RAID (1)

TrevorB (57780) | more than 4 years ago | (#32215230)

Linux Software RAID5 has worked very well for me. Performance is decent (perfectly fine to play back and transfer 1080p video). I got one way back when 3x320GB was enormous and had a 1TB drive before they were remotely available.

Now I'm seriously considering 6x2TB for a 10TB RAID for my next server replacement. No need for an SSD for booting either, just set aside a tiny RAID1 partition (mirrored across all drives) for /boot and you're set. It boots and operates fast enough.

The one problem (as with any solution here) is that 10TB is nearly impossible to back up. Assume the data is lost in case of server hack/house fire, and back up that 50GB that's *really* important to removeable media offsite. I've got an external USB SATA drive desktop "plugin" that works well for larger file transfers. (And Hard Drives are now getting very close to DVDs in GB/$)

I'm sure someone will; pipe up about how SATA drives aren't stable enough for RAID. ZFS is the alternative, but I'm not sure what tools are available for Linux distributions.

Re:Software RAID (1)

BLKMGK (34057) | more than 4 years ago | (#32215522)

The only issue I see with that is in order to access any of your data all drives must be spinning - that's a waste of power. This is caused by data being striped. What happens to your data if two drives die at once? Or three? Do you lose it all or just those drives? Can you use standard recovery software on the drives?

IMO SATA is fine, ZFS looks interesting but I'm happy already.

WHS (4, Informative)

Barny (103770) | more than 4 years ago | (#32215234)

Windows home server, 1TB 7200rpm main drive with seagate LP 5900rpm drives, lock it away and never have to think about it till you need to drop another drive in.

The reason for the fast main drive is that with WHS when you copy data to it, it stores it on the main drive first, then schedules it to be distributed out to the storage drives the next time a "storage balance" is done.

Works fairly well, its based off windows server 2003 at the moment, but if you can wait till the end of the year they have a server 2008r2 version coming out soonish.

Re:WHS (3, Interesting)

Shados (741919) | more than 4 years ago | (#32215318)

This. Even ready-made resellers have pretty small devices built on Win Home Server that can take a LOT of drives. Mine supports 4, but there's many models that can take 8, 12, or more drives. The OS is rock solid and has a lot of neat features, like being able to access your network from an SSL secured web app (built in) from anywhere with indexed search, and its easy to develop plugins for (though there's a ton available already) to extend it.

Re:WHS (1)

EdIII (1114411) | more than 4 years ago | (#32215604)

The OS is rock solid

All the articles, complaints, and bugs regarding data corruption in Windows Home Server might lead one to think otherwise......

Re:WHS (2, Informative)

Barny (103770) | more than 4 years ago | (#32215666)

Yeah, because a bug (running vista on your client and using the server without the latest updates) over a year old and fixed is a problem...

I will also point out that the very first linux release wouldn't run on my 8088 cpu... ... ... ...

Please sir, if you are going to google for bugs, check your dates :)

Re:WHS (1)

BLKMGK (34057) | more than 4 years ago | (#32215490)

That's a waste of drive space. The system I use puts parity on one drive, the OS on a USB, and the rest of the drives are standard format drives that can be mounted under Linux should I need to try data recovery (ReiserFS). Data is not stored redundantly and I can use any size drive I want so long as the Parity drive is bigger or of equal size. With 16 2TB drives I get 30TB worth of storage per server...

unRAID - worth looking into at least. Won't have some of the ability of the Home Server to run apps on it but it's an appliance for storing video and doesn't have the fat OS install either.

Re:WHS (3, Informative)

Sabriel (134364) | more than 4 years ago | (#32215518)

Another happy WHS owner here. I do recall reading that one of the service packs (there have been three) fixed the requirement for a big first drive - files now copy directly to the storage drives.

That said, I still use a fast system drive, and the rest are a mix of 7200 and 5400 rpm drives (depending on what was cheapest at the time).

Bought the original Coolermaster Stacker case. The front of the chassis is solely 5.25" drive bays - eleven of them - technically twelve if you mod the case to move the power+usb front panel elsewhere. :)

Oh, and despite being based on Server 2003, one of the nice things about WHS is that unlike the former it doesn't cost an arm and a leg.

Re:WHS (1)

Barny (103770) | more than 4 years ago | (#32215618)

Hehe, I use a thermaltake armor full tower, with an extra 2 "icage" units installed, 10 drives in the front and 3 in the back, all cooled directly with fans (except for one of the front bays).

Currently only have 11 drives in it, but there is still room to grow.

Its survived 3 drive failures (data was redundantly stored) and a motherboard failure (24/7 constant operation over 3 years managed to kill the caps on the motherboard with 2 months to spare on the warranty), so it is indeed rock solid :)

Re:WHS (1)

ep32g79 (538056) | more than 4 years ago | (#32215636)

Lets be generous and give a 720/1080 dvd rip the size of 10 gigs. With format and overhead he's going to be lucky with stuffing 90 or so rips before the drive is stuffed to the gills.

Given that in the summary the author states

I have run out of slots to put in hard drives across two computers.

I would assume that he is looking for a solution that is about 20x more than what you are recommending.

"I won't need 16,000 RPM drives" (1)

sznupi (719324) | more than 4 years ago | (#32215242)

More than that, you might not need even 7200 RPM drives. There are large capacity "green line" drives from some manufacturers, 5400 RPM, that might be perfectly enough.

I'm sure other posters will have much better recommendations as to how the overall setup should look like, but for whatever it's worth from me - stay away from consumer NAS solutions, they have usually quite small transfers (and I guess its important to you, with files being rather big). Large tower with plenty of space inside + Atom motherboard should be enough, otoh (as long as that Atom board has enough SATA ports...)

Re:"I won't need 16,000 RPM drives" (1)

BLKMGK (34057) | more than 4 years ago | (#32215462)

Correct, 5400drives are fine for viewing 1080P video and actually so is 100meg ethernet but transferring data is slow so go GigE. Consumer NAS are indeed junk, a friend just emo-raged and pitched two Drobo onto Amazon's sales board. He stormed down to Fry's and bought $600 worth of hardware and drives to build an unRAID and is now quite happy with his new appliance that no longer needs care and feeding nor smokes interfaces. ATOM systems can be done but finding a board with enough slots and enough SATA is near impossible for a large system :-( Use a normal mobo and underclock a Celeron or AMD CPU and you'll be fine so long as you choose a reaosnable PSU that's "green". Big tower is a good idea and use the drive trays that lay drives on their side, I use SuperMicro units and can get 5 drives where 3 would normally fit.

You are correct. (3, Interesting)

ClintJCL (264898) | more than 4 years ago | (#32215498)

Even if you "need' a 16000 RPM drive, just make it for your local drive that you play your videos directly off of. Use 5400 for all the other ones. Just move your file before watching it. Sure, if you're an impatient baby and want to watch something within 5 seconds of it entering your mind, then you might have to wait 5 minutes if the file is 4.5G. Then again, it's the type of waiting you can go pee or make your snack during.

pervert (5, Funny)

Cyko_01 (1092499) | more than 4 years ago | (#32215264)

I download a lot porn, and I also record a lot of masturbating videos. I have filled two computers with porn already. I want access to my porn at all times, especially since I maintain a porn site. I don't want to have swappable USB drives, I want all my porn available all the time on my network. I'm assuming that, since it's on a network, I won't need 16,000 RPM drives and thus I'm hoping a solution exists that can be disguised or stashed away and not overheat. So Slashdot, what have you done?

Business opportunity (1)

Dr.Altaica (200819) | more than 4 years ago | (#32215628)

I download a lot porn, and I also record a lot of masturbating videos.

Upload you masturbation vids to get free porn!

It would be like the old MP3 trading FTP sites. then site operater puts sells the vids on a amatur pornsite. Where's my Business Method Patent?

NAS (3, Interesting)

DoofusOfDeath (636671) | more than 4 years ago | (#32215312)

I've tried a variety of approaches, but overall I've been happiest with just buying a NAS box.

I have a Synology DS209 [newegg.com] , and I've been very satisfied. It's a relatively cheap way to get 2 TB RAID 1 storage with really simple backup to an external USB drive. If you need more storage, you can buy NAS devices with more than just two bays.

More info please... (0)

Anonymous Coward | more than 4 years ago | (#32215344)

You don't want swappable USB drives, when then give us more details. How much do you want to spend? $100, 1000, 10000? 1TB USB drives are under $100 each. You can pay that or up to $500 for a NAS in a can for 2TB. The next step up is a group level rack mount SAN/NAS but expect to pay about $500-1000 per TB. Other than that, you will be piecing something together and will undoubtedly be much more complicated or time consuming but no one here can give you a solution for that because you have not given enough details. Search Google for various LaCie NAS devices, they seem pretty cheap. If you are the only one accessing this shared storage, USB 1TB for $100 really is a good solution.

Some of my experience... (0)

Anonymous Coward | more than 4 years ago | (#32215356)

Here are some of my experiences (I've published some more information at http://www.tummy.com/Community/Articles/ultimatestorage2008/):

If you just want super cheap storage, and really don't care about occasionally losing it, just use a collection of simple USB enclosures or network storage devices with individual file-systems on them. This way, data loss may be isolated to a single device, but you get little in the way of error correction.

If you care about the integrity of the data, use ZFS with RAID-Z. RAID-Z checksums the data and can recover from drive corruption. While it's rare, I *HAVE* over the last 3 years seen a few occasions where it corrected errors that normal RAID-5 probably wouldn't have corrected. If you don't particularly care about detecting bit-rot, this is optional.

If you care about the data, make sure you have backups. Again, for video you have downloaded, this may not be that important. In my case, I store all my old photos, music (which I *CAN* re-rip, but don't want to ever have to), so I built two systems.

If any of this data is private, use crypto. For example, we store scanned copies of our records. If someone steals this computer, I don't want them to also walk away with all this private data. Again, maybe not a concern for the OP.

Just get a case that will hold enough drives for your needs, don't try to be fancy with external storage, USB enclosures, etc. External storage sounds like a good idea, but these additional connections are places that can fail. External SATA is a nice idea, but I've had tons of problems with them. The connectors easily come undone, and 2 out of 3 of the 5 drive enclosures I got (see above URL) have been nothing but problems. Luckily, the one on my main storage server has been absolutely no problem. Doing it over in the future, I'd be tempted by something like the Antec twelve hundred which can hold 9 drives internal or up to 20 via the "5 in 3" internal enclosures (or some combination of the two).

Don't worry so much about a quiet machine, it doesn't need to be anywhere that it matters. Mine is in the furnace room.

Of course, set up regular RAID array verify runs and array alerting so that you find out when the first drive in the array fails, not the second.

Sean

ZFS (1)

DiSKiLLeR (17651) | more than 4 years ago | (#32215362)

Set up a nice OpenSolaris box with ZFS and export it with Samba/NFS/iSCSI, etc.

Budget? (3, Interesting)

HaloZero (610207) | more than 4 years ago | (#32215370)

You made no mention of a budget. I'd go with a Drobo - probably the DroboFS. http://www.drobo.com/products/drobo-fs.php [drobo.com]

This should be modded up (0)

ThousandStars (556222) | more than 4 years ago | (#32215476)

The parent post should be modded up -- I came to this comment thread specifically to mention a Drobo. I don't actually have one because I haven't needed the storage, but they've gotten stellar reviews online. They also appear to scale up relatively easily from the cheap 4-drive Drobos to the bigger 8-drive ones.

One other thing you should consider, especially with a lot of people recommending dedicated servers, is power consumption: the bigger and heavier the box, the more you're going to pay in monthly power bills. This is one reason why using an old computer that's sitting around and stuffing 6 HDs into it might not be an optimal solution: if it costs you another $10 - $15 a month in power, you can relatively quickly spend your way out of whatever savings you've nominally achieved.

Dedicated NAS (1)

h3 (27424) | more than 4 years ago | (#32215372)

I suppose if you like fiddling and want to tweak, then building your own is fun and all but if you just want something that works, is most likely quieter and uses less power than one you build yourself, then I say a standalone NAS unit.

I have a QNAP which I love - Synlogy, D-Link, Thecus, Buffalo, etc etc there's a lot of choices out there in 1/2/4/8+++ drive bay sizes. They will typically have various RAIDing options, spiffy web management interfaces, etc that make 'em pretty plug and play.

Just make sure to get one with DLNA support if you want to do streaming to entertainment systems.

Even if you don't get a drobo ... (1)

quizdog (796994) | more than 4 years ago | (#32215376)

... you should totally check out Cali Lewis's promo video at http://drobo.com/resources/drobodemo.php [drobo.com] She is soooooo cute. Plus I recommend the drobo as well with its cool "Beyond Raid" system that let's you just pull out the smallest disk in your array and plug in a new one. Anyone who's ever rebuild or updated a traditional RAID array knows what an improvement that is. Good Luck

Re:Even if you don't get a drobo ... (1)

wagnerrp (1305589) | more than 4 years ago | (#32215514)

Plus I recommend the drobo as well with its cool "Beyond Raid" system that let's you just pull out the smallest disk in your array and plug in a new one. Anyone who's ever rebuild or updated a traditional RAID array knows what an improvement that is.

Eh? That sounds like every other RAID system I've ever used.

The Black Dwarf (1)

war4peace (1628283) | more than 4 years ago | (#32215384)

Try The Black Dwarf technique: http://www.willudesign.com/BlackDwarfTop.html [willudesign.com]

Re:The Black Dwarf (1)

v1 (525388) | more than 4 years ago | (#32215488)

nice unit but entirely too much work for most people. this isn't a casemod, it's more like a "build your own car" kind of project.

I vote for the drobo elite. All that time and materials and tools that dwarf requires easily covers the cost.

unRAID from Lime Technology!! (2, Interesting)

BLKMGK (34057) | more than 4 years ago | (#32215420)

I have two of these servers now. Each server can hold as many as 16 disks (possibly more actually as the programmer keeps bumping that up) with one disk reserved for parity. Data is NOT striped and parity is ONLY stored on the one drive. If a disk fails I lose no data, if two fail I lose two disks of data but nothing else. No hot spares or any other crap. If a disk isn't being used it goes to sleep and saves me heat and power. Disks can be ANY size but the parity disk must be as big or bigger than any of the data disks. Runs on a pretty decent selection of hardware although keeping the list of what works and what doesn't up to date is apparently tough since hardware changes so fast. It's Linux based but pay for play, yes he's followed the GPL. It's not super expensive and it boots from a USB drive to be web administered. I use full tower cases with SuperMicro 5n1 trays, 2gig of memory, Celeron CPU, power saving PSU, and supported mobo that have onboard video and GigE which you WILL need.

Their forums are a big help and active, users are working to expand the capabilities of these NAS and the programmer is working on making that easier too. Check it out, I've not found anything better yet and with some of the newer versions of SAMBA in the code it's pretty fast too! Perfect for a HTPC but not so great for a big transactional database

http://www.lime-technology.com/ [lime-technology.com]

Big cases, PCI-E SATA Controllers (1)

evenmoreconfused (451154) | more than 4 years ago | (#32215432)

We make S3D movies. Lots of them, in HD, IMAX, 2K DCP, etc. Tons and tons of data, with lots of revisions, none of which any creator ever actually is willing to get rid of.

Our server array is (mostly) made up of machines each of which has a 14-bay case, a fairly basic mobo with 6 onboard SATA ports, two added 4-port PCI-E SATA controllers for a total of 14 x 2TB = 28TB per server for less than $1K not including drives. Our servers happen (for stupid red-tape reasons) to run W2K8 Server but any SAMBA equivalent would work just as well.

No redundancy or backup though: this is only for (easily reproducible) render output. Key stuff is kept on systems with RAID-5 arrays, nightly mirroring, etc.

My solution (1)

EnsilZah (575600) | more than 4 years ago | (#32215458)

Preferring Western Digital drives (for no particular reason) I have a pair of 1TB My Book Essential Edition external USB drives as well as a 2TB My Book World Edition network drive (which I got form a guy for like half the price).
Anyway, the World Edition has a USB port that allows me to connect the other two drives to it using a USB hub and it shows them as network shares in addition to its own folders.
Another nice thing about the World Edition is that it runs Linux so there's neat stuff you can do with it, mine is currently running a torrent client called Transmission which has a Windows (only a wannabe Linux geek) front end I can use to control it remotely.

Have you considered ATA Over Ethernet (AOE)? (3, Interesting)

jcwren (166164) | more than 4 years ago | (#32215464)

http://en.wikipedia.org/wiki/ATA_over_Ethernet [wikipedia.org]

This is something I've always wanted to play with. It's a little expensive (for a home user) to get into, but it's extremely scalable. If I moved all my DVDs and such to on-line storage, I think this is what I would opt for. It can be run in all sorts of RAID configurations, doesn't require matched sized hard drives, and it can all be racked up very nicely.

OWC Qx2 4-drive RAID array (2, Interesting)

david.emery (127135) | more than 4 years ago | (#32215480)

4 drive bay, USB, FW400/FW800 and eSATA. Will take 2tb drives, RAID 0, 1, 5 and 10. Comes pre-populated or unpopulated, the latter is what I got and added my own drives. http://www.macsales.com/ [macsales.com] No financial connection, just a satisfied customer (they have great tech support!)

This is obviously not a build-it-yourself storage array, but is a good option if you want a commercial out of the box solution.

DIY. Map-Drives, Dir, Grep (1)

ClintJCL (264898) | more than 4 years ago | (#32215484)

Really, that's all you need. A good map-drives script that maps all your drives to all your computers so everybody has access to everything. I'm almost out of drive letters, but I basically have your goal. No solution. Just using what's in the operating system already.

Have another script that you run to index things. Basically, a dir /s command [add filesizes to the end if you can]. There's your index of where everything is. Use grep to access it quickly, or load it all up in a text editor and find to access it slowly. I like grep. Regular expressions help.

Anonymous Coward. (0)

Anonymous Coward | more than 4 years ago | (#32215492)

drobo pro =)

Two Options (3, Informative)

Zarjazz (36278) | more than 4 years ago | (#32215502)

1) Cheap tower server + your favourite unix distro + software RAID + many, many cheap 2TB drives.

2) Standalone NAS device. Everyone so far seems to recommend different makes so I'll carry on the trend and suggest Thecus [thecus.com] . Just slot in the drives and you're ready. Install the SSH module and you also have a Linux server too.

N510 + 4GB + USB flash+ 7+ disks + full tower (0)

Anonymous Coward | more than 4 years ago | (#32215542)

An Atom N510 board with 4GB, a small USB flash drive, and FreeBSD 7.3 or OpenSolaris. You'll get an awesome amount of capacity with negligible setup or maintenance, all while being able to tell whether or not (with certainty) there's data corruption occurring. An old full tower with 3.5" bays going the full length of the front would be the ticket. Be sure to have a powerful enough PSU.

I'm still kicking myself over getting rid of an old Gateway 2000 case: it must've had room for 10 3.5" drives; more, if you'd use drive adapters for the 4 (5?) 5.25" bays.

You can't beat it for price/performance/storage, and it's the same basic thing you'll find in a higher-end SAN type device.

NAS Should be Obvious (1)

segedunum (883035) | more than 4 years ago | (#32215550)

It should be obvious that you need a NAS. Buffalo do devices called Linkstations that can house a single drive storing up to a terabyte (or put in your own disk), or they have Terestations I think they're called with network attached RAID storage. These are all extremely quiet and unobtrusive when compared to doing this with PC hardware. They can be accessed over SMB, NFS, FTP or with other file transfer protocols.

What you do after that depends on how geeky you want to be. I have Freelink running on my Linkstation (Debian for ARM basically) running a full Samba Windows domain with authentication, completely automatic printing support where you don't even have to do any manual printer driver installs and the kitchen sink.

The Delete key (1)

Doug52392 (1094585) | more than 4 years ago | (#32215572)

Do you really need to save all those Blu-ray rips of the latest Hollywood blockbusters? Just delete them after you watch them. That way you'll have plenty of room for all those raw uncompressed video.

Unraid! (1, Redundant)

BarryJacobsen (526926) | more than 4 years ago | (#32215578)

I'd highly suggest you check out unRaid [lime-technology.com] . It's an inline expandable raid system which allows up to one drive to fail without losing data. I've been using mine for quite a while and I love it!

Cheap n easy (0)

Anonymous Coward | more than 4 years ago | (#32215584)

1TB or 2TB external USB 2.0 drives are cheap.. just the other day LaClie had a 1TB external drive for approx $70
Get a 6 port USB hub... now you can scale your storage.
If you need a backup.. get another drive.

If you have a PS3 install PS3 Media server (free) and you have a cheap HTC solution.

FreeNAS (1, Informative)

Anonymous Coward | more than 4 years ago | (#32215590)

http://freenas.org/

Booting this off USB flash with 4 1.5 TB drives in a RAID 5 configuration =
It can even spin down the drives when they aren't in use.

Each 1.5 TB drive : $100
AMD Misc processor, motherboard with 4 SATA, GigE, RAM: $200
Case and Decent power supply: $100

Go All- Apple. (-1, Troll)

Anonymous Coward | more than 4 years ago | (#32215592)

Save yourself the hassle of all these inane linux/bsd solutions and go buy yourself an Apple X-serve [apple.com] 's. Each one can hold 6 terabytes of disks, which is plenty for most uses. Yes this will cost you more money up front, but the time you save versus fighting driver and configuration issues with Linux or BSD will end up saving you money, unless of course your time is worth nothing.

Think Different
Think Better
Think Apple

Forget NAS (2, Interesting)

Kjella (173770) | more than 4 years ago | (#32215632)

If you want cheap, affordable storage get:

A decent full tower case with a modular PSU
A motherboard with 8+ SATA ports (cheap)
A 4-port SATA expansion card
=
12 SATA slots + 12x SATA power for cheap

Get a cheap bunch of 1.5 TB drives for up to 18TB total. If you say home I assume you don't mean 99.9% redundancy. You can buy a new PSU or motherboard or whatever and have it delivered and that's okay. Softraid two drives in RAID1 for 1.5 TB less storage. If you need more protection then upload it to some offsite backup - any external disk or second machine is still vunerable to theft, fire and whatever. It works for me, though I only have ~10 TB due to due of old low-capacity disks.

Oracle Sun Fire X4540 (2, Funny)

xavi62028 (877425) | more than 4 years ago | (#32215634)

48 SATA drives onboard a 4U rack, dual six-core opterons, redundant power supply. This thing won't let you down. Go big and heavy with this and it'll cost $1/GB. http://www.oracle.com/us/products/servers-storage/servers/x86/031210.htm [oracle.com] Make sure to ask for a discount, only suckers pay full price for Oracle gear.

NAS devices (1)

svirre (39068) | more than 4 years ago | (#32215656)

You specified quiet and hidden (small), not cheap, so I'd go for a NAS device the synology DS1010 can do 5x2TB (8TB with redundancy), and if you need more it can be expanded with 5 bays more.
A cheaper option would be to take some old hardware and toss a NAS distro on it, but I'd expect more hassle and noise from that solution

speed? (1)

farble1670 (803356) | more than 4 years ago | (#32215658)

any sort of network accessible drive is going to be relatively slow. if you are copying large files that will be important. if you expect to use the large drive for your working sets, as opposed to just for storage, that will be crucial.

the truth is that you probably won't be happy with anything less than a eSATA interface.

Load More Comments
Slashdot Account

Need an Account?

Forgot your password?

Don't worry, we never post anything without your permission.

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>
Create a Slashdot Account

Loading...