Beta

Slashdot: News for Nerds

×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Ask Slashdot: Best Kit For a Home Media Server?

samzenpus posted more than 2 years ago | from the serving-your-collection dept.

Debian 355

First time accepted submitter parkejr writes "I started off building a media library a few years ago with an old PC running Ubuntu. Folders for photos, ogg vorbis music from my CD collection, and x264 encoded mkv movies. I have a high spec machine for encoding, but over the years I've moved the server to a bigger case, with 8 TB of disk capacity, and reverted back to Debian, but still running with the same AMD Sempron processor and 2GB RAM. It's working well, it's also the family mail server, and the kids are starting to use it for network storage, and it runs both link and twonkyserver, but my disks are almost full, and there are no more internal slots. The obvious option to me is to add in a couple of SATA PCI cards, to give me 4 more drives, and buy an externally powered enclosure, but that doesn't feel very elegant. I'm a bit of an amateur, so I'd like some advice. Should I start looking at a rack system? Something that can accommodate, say, 10 3.5" drives (I'm thinking long term, and some redundancy)? Also, what about location — I could run some cat6 to the garage and move it out of the house, in case noise is an issue. Finally, what about file format, file system, and OS/software? I'm currently running with ext3 and Debian Squeeze. Happy with my audio encoding choice, but not sure about x264 and mkv. I'd also consider different media server software, too. Any comments appreciated."

cancel ×

355 comments

No reason to change from H.264 (5, Insightful)

InterestingFella (2537066) | more than 2 years ago | (#38491694)

Why would you change away from x264 and mkv. They are the industry standards. Not just in computers, but every way in the distribution chain. Going about it for some FOSS reason is just stupid because they're only for your own use, not for distribution. You would be either spending double the space or get half the quality by going with something other than H.264, and on top of that you introduce yourself additional problems because they are not what everyone uses.

Mac mini or apple Tv (2, Funny)

noh8rz (2535268) | more than 2 years ago | (#38491816)

I would get a Mac mini, and use iTunes. Or you can get an apple tv and put everything in the cloud.

Re:Mac mini or apple Tv (4, Insightful)

Kagetsuki (1620613) | more than 2 years ago | (#38491838)

I'm sorry, did you read the summary? Do you honestly think a Mac mini is a step up from that or solves the problems presented?

Re:Mac mini or apple Tv (0, Funny)

Anonymous Coward | more than 2 years ago | (#38491876)

I didn't understand most of tfs - lots of gobblegook. But It sounds like he wants a htpc, which is a techie way of saying Mac mini. Just suggesting that an apple tv would work well too.

Re:Mac mini or apple Tv (2)

aaron552 (1621603) | more than 2 years ago | (#38491952)

He wants a media server not a HTPC.

Re:Mac mini or apple Tv (5, Interesting)

sortius_nod (1080919) | more than 2 years ago | (#38492238)

Just because you don't understand the request doesn't mean it's gobbledegooke, it means your knowledge is limited.

Personally, I don't trust any auto-encoding solutions as they easily go haywire. I'd suggest doing that all by hand.

Ext3 is fine, & a rack mount is a necessity. If you want smooth operation of the system, at least 2 network cards are a must (I run 3, 2 bonded for media/SMB, 1 for management & VPN). I'd suggest having a decent 16 port switch in the house & running the 2 (or more) cables to the box.

For DLNA I just run miniDLNA & for torrents I've just set up uTorrent with a web interface. There's very little my desktop actually does.

Re:Mac mini or apple Tv (1)

PNutts (199112) | more than 2 years ago | (#38491982)

I was going to suggest Apple as a joke, but rereading it his main issue is storage and he does ask about OS as part of his issue. I don't consider storage and front end coupled so the Mac mini and iTunes is a legitimate interface suggestion as long as he understands the implications to the content.

Re:Mac mini or apple Tv (4, Informative)

pcx (72024) | more than 2 years ago | (#38491996)

A mac mini is a perfectly valid (if expensive) media solution. I personally use 2 synology NASs feeding a Mac Mini which is a dedicated iTunes server feeding 4 apple TVs (When I win the lotto I might try out a promise pegasus but until then the synology NASs are growing the library nicely). It's a bit of a pain to remux mkvs to mp4s but it works and it's a really nice solution once you've gone through the headaches of setting it up (yes better than a boxee box -- tried it, now gathering dust, better than every DLNA solution I've tried). The Apple TVs are just really, really nice media end-points and iTunes is a perfectly good management system.

But yea iCloud has no place in any video solution (your purchased shows can be streamed -- movies can not, and this solution will introduce you to your ISP's bandwidth caps very quickly).

Re:Mac mini or apple Tv (0)

Anonymous Coward | more than 2 years ago | (#38492082)

If it's a new version of the mimi, I think it's not a bad option as you can thunderbolt (lightpeak) to additional external storage.

Re:Mac mini or apple Tv (5, Interesting)

hairyfeet (841228) | more than 2 years ago | (#38492124)

If he just wants to go cheap and its gonna be stuck in some garage anyway i'd say white trash it like we did at my last shop. Me and the boss took a couple of the biggest cheapo ATX cases we could get our hands on, took out the motherboard mount and cut the frame on one, and finally a couple of small weld stuck them together. We ended up with something like 16 SCSI drives in that sucker for a total of 2Tb when most folks were still getting 80Gb drives. with those babies loaded with every single driver for just about every piece of hardware up to that point it was quick work to reload a PC.

:As for the board I'd go with something like a cheap AMD board with a nice cheap Phenom low power quad. you can get a Phenom I quad for $55 at Starmicro [starmicroinc.net] and add a couple of Gb of DDR 2 for maybe $25, board for around $40 and finally a decent HSF for around $15.

So when you figure in the cases which with no PSU can be had for less than $20 a piece from many places like Geeks you are looking at a final total of around $185 for a quad with 2Gb of RAM with an add on card for adding more SATA slots. Sure it won't be the prettiest thing around but if its just gonna be serving files from a corner somewhere who cares? I'd add a little more and get a full size board with lots of SATA slots and 3 PCI slots for adding more SATA cards and you'd have a thing you could load to the brim and with the low power Phenom quad you'll have plenty of power for controlling the whole deal and maybe even it doing some of the transcoding work via scripts at night when its not serving files.

Re:No reason to change from H.264 (-1)

Anonymous Coward | more than 2 years ago | (#38491836)

Hey, CommanderPony, techla, cgeys, etc. etc. etc. Diaf

Re:No reason to change from H.264 (1, Interesting)

Telvin_3d (855514) | more than 2 years ago | (#38492132)

Why would you change away from x264 and mkv. They are the industry standards.

Stick with x264. It is open source and industry stanard. Not open licensed from a commercial point of view, but all the software side and standards are completely open.

Ditch .mkv as soon as possible. It's an almost completely unsupported container. Even among software that supposedly supports it there can be compatibility issues. It's popular in the ripping/pirate communities precisely because it's a pain to use. Just getting your videos to work on a regular basis is a mark of distinction.

Switch to a standard .mp4 container. Much better supported on hardware or software. Some day you will want to be able to stream from your server to a thin set-top box or load a file on your kid's phone. On that day .mkv will make you cry.

There are server remuxers floating around that repackage the video and audio streams into a new container. No re-compression or quality loss. You can fold any subtitle files or other extras into the file at the same time.

Re:No reason to change from H.264 (5, Informative)

Ltap (1572175) | more than 2 years ago | (#38492206)

Actually, Matroska has a number of good advantages:
1: More open than MP4 -- it has none of the ugly MPEG-LA overtones.
2: More codec support than MP4.
3: More consistency -- there aren't PS3-oriented versions or AppleTV versions of Matroska. MP4's device support might be wide, but when every player and device seems to have its own version with its own ridiculous, restrictive standards, it doesn't really qualify as supported at all.
4: Content management: Matroska is the easiest to mux and makes it great to work with alternate audio and video streams, subtitles, etc. You get the widest range of options of any container out there.

While Matroska might be used by some because it is perceived as more "elite", there are many very valid reasons for using it, mainly revolving around flexibility and openness. As someone who has worked with both containers, I can say that Matroska far more easily delivers what I want and the tools for working with it are generally free (in both senses), more usable, and more powerful.

Re:No reason to change from H.264 (4, Informative)

Stiletto (12066) | more than 2 years ago | (#38492212)

Ditch .mkv as soon as possible. It's an almost completely unsupported container. Even among software that supposedly supports it there can be compatibility issues. It's popular in the ripping/pirate communities precisely because it's a pain to use. Just getting your videos to work on a regular basis is a mark of distinction.

I disagree. Having just converted my entire collection over to mkv, I'm never looking back. There are some great reasons to use it:

1. I've found MKV to have better support for chapters
2. MKV has heaps better support for subtitles (I could never manage to get subtitles to properly work across players using MP4)
3. MKV can hold just about any video and audio encoding, not just H.263 and H.264
4. (If you care) MP4 has patent issues, whereas MKV does not

Re:No reason to change from H.264 (1, Informative)

kyrio (1091003) | more than 2 years ago | (#38492286)

MKV is used because it's currently the best container, there's nothing else to it. With the dumb shit you are saying, you obviously have no clue about the state of the technology.

Why? (1, Flamebait)

bonch (38532) | more than 2 years ago | (#38491696)

Should I start looking at a rack system?

Well, how should we know what price you're willing to spend? If you're just wanting to know what rack systems are available that can support 10 hard drives and how much they cost, can't you just Google that?

Also, what about location — I could run some cat6 to the garage and move it out of the house, in case noise is an issue.

How could we answer this? We don't know what location is most convenient for you. We don't know how much physical space you have available in your house or your garage. Are your family's file-serving needs so extreme that you can't even rely on wireless networking?

Finally, what about file format, file system, and OS/software? I'm currently running with ext3 and Debian Squeeze. Happy with my audio encoding choice, but not sure about x264 and mkv. I'd also consider different media server software, too. Any comments appreciated.

You're running out of disk space, yet you're encoding audio with a technically inferior format that uses more space for less quality than other formats.

Honestly, based on the rattling off of technical specs, it sounds like this project is more about tinkering and tweaking. There are plenty of pre-built Linux-based media server projects [lmgtfy.com] (I'd suggest other operating systems, but I know you'll only accept Linux), but you're not going to accept those solutions because you want to tweak and do it all yourself. For crying out loud, you're running your own family mail server and 8TB file server in an era of Facebook-based communication, web-based email, and cloud storage/streaming services.

Re:Why? (1)

Anonymous Coward | more than 2 years ago | (#38491770)

A little harsh. The question wasn't posed perfectly, but I'd say I got the idea of what he's looking for: great, contemporary ideas that he hasn't heard of in the home server space. I think he realizes that a rack system is going to be expensive, but that someone out there may be doing it on a tight budget and may have some advice, and since it's going to be long term and multi-user, the cost might be justified.

Personally, I can't help much. I run a cheap retail headless box with 4 drive slots on a shitty uplink. It sounds like this guy wants a permanent solution and doesn't mind getting his hands dirty. I can appreciate that, and hope he finds some constructive advice here, but I'll agree with you on this: we need more information about his actual needs, as his questions don't allow for much outside the box thinking.

My only comment would be on the additional pci comment. If you're going to stuff the gear in the garage, who gives a shit how it's all connected?

Re:Why? (3, Interesting)

InterestingFella (2537066) | more than 2 years ago | (#38491822)

Rack doesn't make any sense for home environments. It's best used in data-center environments because it saves space and is highly standardized, meaning it also has fast and easy ways to connect input devices and everything else in a environment that hosts thousands of servers. The guy isn't going to be hosting so many servers, so rack doesn't really help with anything. It just costs more and will actually restrict you with size limitations and customization. On top of that you need to get something to mount it to. With a standard box you can just put it anywhere.

Re:Why? (1)

smi.james.th (1706780) | more than 2 years ago | (#38491888)

His standard box is already full though. Unless he gets another one next to it. I've never done anything with more than 3 HDDs personally, so I'm afraid I'd be very little help here. Good luck though, it sounds like a fabulous project!

Re:Why? (1)

symbolset (646467) | more than 2 years ago | (#38491940)

You can get a 25-30U rolling rack and put it into the garage. I did. $100 at surplus.

Re:Why? (3, Informative)

sjames (1099) | more than 2 years ago | (#38492298)

Agreed. In addition, rack is generally designed with the premise that space is at a premium and that it's well worth cramming as much as possible into the smallest space possible and then compensating for the poor natural airflow with high speed fans everywhere. That's fine if space is that big a premium, but ion a home environment, it's rarely THAT tight. A big roomy tower will run quieter, have less problem with failing fans, and probably will run a bit cooler. It'll also be less of a pain to work on.

Re:Why? (2, Insightful)

mikael (484) | more than 2 years ago | (#38491962)

I'd say it would be better for him to get a separate system for the kids. Some time they are going to want to move away from home and move to college/into their own flat.

At that point, they are going to have to buy their own data storage then and transfer everything across.

Re:Why? (0)

Anonymous Coward | more than 2 years ago | (#38491774)

Do you need to be that snarky? What a useless response - complaining how not enough details are given and writing a lot yet providing absolutely nothing of value for anyone.

How about making some assumptions? If you want to spend $x then do this, but if you only have $y than do that. That way your response is useful not only for the submitter but other people as well.

Bitter Apple Troll (-1)

Anonymous Coward | more than 2 years ago | (#38491802)

What do you expect?

bonch and all his multiple accounts is angry and bitter his precious Apple is getting destroyed by Google's Android.

It's too bad the Apple crazies like him didn't kill themselves when their douchebag of a cult leader took a dirt nap a few months ago.

Re:Bitter Apple Troll (-1)

Anonymous Coward | more than 2 years ago | (#38491884)

Right, the company set to post the biggest earnings blowout in its history [bullishcross.com] is getting destroyed by a third-rate malware-ridden iOS clone that doesn't make Google or its third-party developers any money.

Re:Why? (1)

EvanED (569694) | more than 2 years ago | (#38491780)

I do think that this is one of those very vague, under-spec'ed questions. But...

You're running out of disk space, yet you're encoding audio with a technically inferior format that uses more space for less quality than other formats.

If he's running out of space on 8 TB, I'd say that audio is almost certainly the least of his problem. I don't exactly have a huge CD collection, but I do have somewhere around 300 CDs that I have ripped to FLAC, and that comes to under 100 GB. He could have 8 times the number of CDs I do and the audio portion would still only be a reasonably small 10% -- and that's ignoring the difference between Vorbis and FLAC.

Re:Why? (2, Informative)

Anonymous Coward | more than 2 years ago | (#38491976)

The submitter has very specific tastes and would rather keep conversion to a minimum because of the overhead and liabilities involved.

You see, the submitter is storing guro, bestiality, snuff, squirt/scat, and crush videos; all featuring underage animals, so the overhead must be kept to a minimum. He must make do with wired networking only, adding parts to his existing configuration as needed so his horrific pornography may be available to him immediately, whenever he wants to jack off.

I've seen his workstation. He lays a towel over his cheap office chair so he can plant his bare ass on it in comfort. He once remarked that he likes to keep his pants around his ankles so he can pull them up in a hurry when his wife or kids knock on the usually-locked door. The interior of all windows in that room are coated with aluminum foil. On the floor next to his chair lies a long athletic sock crusted stiff by what appears to be dry super glue. A marijuana pipe sits on his desk, next to the razor scratches, with loose nuggets and ampule shells of crystal meth and amyl nitrate respectively. He also keeps a noose handy so his free hand can choke his neck while his other hand chokes his chicken.

Yep, hard drives.

Re:Why? (1)

pjbgravely (751384) | more than 2 years ago | (#38492204)

It is probably because he is using ext3 as a file system. It uses a horrendous amount of space for the journal.

Re:Why? (1, Informative)

slick_rick (193080) | more than 2 years ago | (#38491804)

@Bonch: Congratulations, you are officially the Grinchy douche on Christmas.

OP: Great questions, I can not wait to see answers. Would love to hear more about what your software solution is for encoding, I've got a bunch of DVDs & Bluerays I would really like to get on the network, but a streamlined rip+encode+publish I have yet to achieve :-/ What are you using on the frontend? I've got various iDevices roaming about, and a Roku that does 720p for the projector, but haven't had much luck with mt-daapd so far.

I do not trust wireless for my projection room, I ran cat 6 everywhere when I moved in as nothing beats the reliability of copper. I've got a basement, so I put the server is right by the patch panel in the furnace room. Noise is definitely an issue. The only thing that scares me about the garage is bugs and what not gumming up the cooling enough to cause catastrophic failure. With a bunch of drives you need a bunch of airflow to keep everything cool, I can just imagine the intake fan sucking up mosquitoes, and coating the CPU cooling block with my dog's blood.

As far as rack vs tower, I started with a rack and ended up with towers. The rack systems just took up way too much floor space, and that space was poorly utilized to boot. It made a very large utility room feel much smaller.

Arduino. (5, Funny)

Anonymous Coward | more than 2 years ago | (#38491706)

Just kidding.

Re:Arduino. (3, Funny)

Anonymous Coward | more than 2 years ago | (#38492134)

I think you would need to imagine a beowulf of arduino...

sysadmin'ed by natalie portman with grits in her pants who gets hugged by cowboyneal

A little under capacity, but perfect otherwise! (2)

Zoson (300530) | more than 2 years ago | (#38491718)

With drive capacities soaring, I wonder if you'll really need 10 drives.
You might want to try something like a fractal design array. It's a small htpc case for a microatx board. It has mounts for six 3.5" drives, and these days a microatx board will have everything you need, including integrated video for all your playback needs.

File System (1)

Anonymous Coward | more than 2 years ago | (#38491720)

Well as for file system I would say ext4 since btrfs isn't quiet stable yet. Unless you wanted to go with a *BSD in which case FreeBSD is a solid choice and FreeBSD's implementation of UFS is also rock solid. For a while services like Yahoo and Hotmail where exclusively hosted on FreeBSD servers.

Re:File System (3, Interesting)

EvanED (569694) | more than 2 years ago | (#38491846)

Another option with FreeBSD is ZFS, which is pretty sexy.

Re:File System (1)

lister king of smeg (2481612) | more than 2 years ago | (#38492094)

Depending on how adventurious you are you could go with open solaris if you like zfs.

raid (2, Informative)

Anonymous Coward | more than 2 years ago | (#38491728)

start playing linux mdadm and run raid 5 array with 1 or 2 spare drives. I built my first 1TB system about 9 years ago with 14 120G drives. ran without major failure for 6 years. Play with the array (remove drives, add new drives, etc) BEFORE you have a drive failure so you understand how to repair the failure. Let me say again, learn BEFORE you have failure. Otherwise, you'll freak out at losing all your data because you screwed something up during repair.

Re:raid (1)

swalve (1980968) | more than 2 years ago | (#38492024)

That's what I do, and I'm about to split my "server" into two machines. One, a quasi-SAN with just shitloads of hard drives raided and accessed via iSCSI, and the other will do all the heavy lifting.

Larger disks (5, Informative)

Anonymous Coward | more than 2 years ago | (#38491730)

Larger disks. 4 TB should be available very soon (maybe now?).
http://www.extremetech.com/computing/108665-hitachi-ships-worlds-first-4tb-hard-drive-sticks-it-to-thor

ESXi (0)

Anonymous Coward | more than 2 years ago | (#38491748)

My house server is an ESXi box with 8 2TB drives. My fileserver is just an Ubuntu server VM running Samba.

Each TV has a Mini-ITX Windows Media Center box next to it, using that little ASUS E-350 board. Each HTPC just mounts the samba share in read only and everything works flawlessly.

MCE can handle the native DVD (and BluRay via Cyberlink software) folder formats, as well as many other file formats.

I generally just save my DVDs and BluRays on the server with no further compression, and use HandBrake to create a set of ~700MB movies for the phones.

I use mythtv (2)

JustNiz (692889) | more than 2 years ago | (#38491752)

I use mythtv. It does pretty much everything. I love it.

Re:I use mythtv (1)

jamesh (87723) | more than 2 years ago | (#38491776)

I use mythtv. It does pretty much everything. I love it.

Out of space? Just add another box. The idea that your entire media store is attached to a single server seems a bit old fashioned.

Re:I use mythtv (1)

Anonymous Coward | more than 2 years ago | (#38491914)

Myth is not suitable for a modern HTPC, because it doesn't support streaming from netflix. That's a fatal flaw for a huge number of people these days.

Re:I use mythtv (1)

jamesh (87723) | more than 2 years ago | (#38491942)

Myth is not suitable for a modern HTPC, because it doesn't support streaming from netflix. That's a fatal flaw for a huge number of people these days.

what's a netflix?

Re:I use mythtv (3, Insightful)

EdIII (1114411) | more than 2 years ago | (#38492016)

If you really want Netflix cheap you can buy a Western Digital Live TV Plus. They are under $100 and do extremely well with Netflix and a bunch of other apps. I have gone through several firmware updates and they keep adding new apps. In addition to the apps you can do streaming from just about any kind of server and USB connected flash drives.

HDMI and even optical audio out if you needed it. Netflix has a lot of HD titles, plus 5.1 audio on quite a bit too. I believe the Ethernet is gigabit, but I don't know off the top of my head.

Only thing it does not do well at all is DVD ISO files.

For under $100 though with a nifty little remote you will find yourself using it for more than just Netflix on a regular basis.

Consider it a nice little addition to a HTPC.

Re:I use mythtv (1)

symbolset (646467) | more than 2 years ago | (#38492012)

Don't most of us get our netflix some other way now? It's embedded in five phones, two tv's, five tablets, six PCs just at my house. They probably think I'm sharing the account by now. Who really needs another way to watch Netflix? BTW: am watching Netflix right now on tv while tapping this out on my tablet.

Re:I use mythtv (1)

Elrond, Duke of URL (2657) | more than 2 years ago | (#38492186)

I am curious about this. I've been a MythTV user for many years now. I don't *need* Netflix, but it certainly would be nice to have. Part of the draw of a well built HTPC is having everything in a single place controlled by a single remote.

But, you're right that MythTV does not natively support Netflix. Maybe it could be accessed via MythBrowser? Or is a real Win/Mac (on a PC) client necessary?

Is there any other Linux alternative to getting Netflix running? Or, for that matter, any other movie streaming service? I believe MythBrowser is built on top of Webkit and that it can handle the Flash plugin. Certainly, that's enough for YouTube (already handled by MythNetbrowser), but maybe not more.

Google tells me that some people have done this by having their Mythbox launch a WinXP virtual machine to run IE to run the Silverlight Netflix player. The author seems to indicate that, overall, it is not hard to implement, but the CPU requirements are fairly high. Hmmm...

Separate servers (1)

obarthelemy (160321) | more than 2 years ago | (#38491756)

Instead of tacking function after function onto the same server, I'd encourage you to use several small ones, including the $35 Raspberry Pi. That way one one piece of software starts to go haywire, it doesn't bring the whole shebang down with it.
Also, I'd go for fewer, larger disks. As long as you do backups it's not more risky, and it's a lot more practical. HDs reach 4TB these days, so you're talking 2 HD for your existing data, a 3rd one more more capacity, and double that for backups.

Re:Separate servers (0)

Anonymous Coward | more than 2 years ago | (#38491796)

Several servers, too complex for the stated mission.

If you are going to use more than one server, do it virtually, on a solid virtualization platform like ESXi. A 5 year old can figure out how to get ESXi going with vSphere.

you may want to have raid 5 or 6 (2)

Joe_Dragon (2206452) | more than 2 years ago | (#38491772)

10 disks in a raid 0 type setup is a big risk. Also lot's of pci cards eat up PCI bus io and maybe even the same io used for the network also your board like only has 100M e-net. Now if your system has a pci-e slot then a 10 port non raid card + software raid may work and is cheap then a raid card. But you may also want to get a newer MB + cpu most new amd and intel boards max out at 8 sata ports. 8 ports may work out ok or you can get a new board with a dual core cpu + a raid or non raid card also a new MB will get you GIG-E + pci-e IO.

Re:you may want to have raid 5 or 6 (5, Insightful)

sirsnork (530512) | more than 2 years ago | (#38491848)

Yup, start again.

Pick a board with plenty of SATA ports, put a modest amount of RAM and CPU in it. Make sure it's got PCI-E slots (what hasn't these days) and go from there.

Use bigger drives than you are currently, it's a bad time to buy drives so wait if you can but just build a new box from scratch and save yourself the headache of trying to migrate drives or retain data while upgrading drives one at a time in an existing array.

New machine, 3TB drives x as many as you want (6 would about double your capacity), add a 4 port PCI-E SATA card if you need it and rsync all the data across, job done

I was saying maybe a raid card but non raid card (1)

Joe_Dragon (2206452) | more than 2 years ago | (#38492002)

useing software raid is ok most boards have about 6 ports so if you want like 10 then maybe a x4 or better pci-e card may be needed.

Re:I was saying maybe a raid card but non raid car (4, Interesting)

nabsltd (1313397) | more than 2 years ago | (#38492136)

useing software raid is ok most boards have about 6 ports so if you want like 10 then maybe a x4 or better pci-e card may be needed.

Or, get an actual server board (this is gonna be a server, right?), like this one [supermicro.com] . That's six SATA ports and 8 SAS ports. If you flash the SAS ROM to the "no-RAID" version, the controller is recognized natively by Linux. In addition, you get lots of PCIe connectivity, a pair of Gigabit Ethernet ports, and IPMI (allowing remote power cycle).

Then, find a full-tower case with lots of 5-1/4" drive bays, and add hot swap bays [newegg.com] . There are smaller versions, as well...just budget what you need for drives.

I use the motherboard I referenced along with an add-on 8-port SATA card (anything supported by Linux would be fine) and two of the drive bays for ten 2TB drives in RAID-10. I boot Fedora off a pair of SSDs in RAID-1 and also have four 2-1/2" 750GB drives in RAID-10. The 10TB array serves iSCSI over 10Gbit Ethernet to ESX systems that hold all my VMs, with the 1.5TB array as local and NFS storage. There's still PCIe slots available if you need more controller cards.

With this setup, the VMs are how everything is accessed, so you can pick whatever OS you want to face client machines.

Re:you may want to have raid 5 or 6 (1, Insightful)

93 Escort Wagon (326346) | more than 2 years ago | (#38491944)

Raid seem silly in a home setup. People see it as a backup solution when it's not, and I doubt a few extra tenths of a percent of uptime is really significant at home. Just have one or two big disks for all your media and stuff, then buy the equivalent to use for regular rsync backups. You can get a third to take "off site" occasionally (e.g. your sister's house, a friends house, a drawer at work) if you want to be really careful - just rotate the backup disks every so often.

with 8TB data you are looking at 3 3 TB HDD's (2)

Joe_Dragon (2206452) | more than 2 years ago | (#38492042)

now just to fit the data on one set is like 3 HDD's and then raid 5 or 6 starts to look like a good idea.

Re:you may want to have raid 5 or 6 (2)

swalve (1980968) | more than 2 years ago | (#38492046)

I like it for expediency (no need to restore when a drive fails) and for simplicity. I found myself with single drives in a bunch of machines, and my free space management got to be a huge waste of time. It became easier to just concatenate all my storage into one volume with one "xxxGB free space" indicator and when that starts to fill up, I start replacing hard drives.

Re:you may want to have raid 5 or 6 (1)

nabsltd (1313397) | more than 2 years ago | (#38492150)

Raid seem silly in a home setup.

Once you start to use more than one hard drive worth of space, RAID is pretty much required unless you really like to spend days restoring data when a hard drive fails.

FreeNAS (1)

Anonymous Coward | more than 2 years ago | (#38491810)

For what it's worth, I've had a FreeNAS box running for about a year now and I've loved it. I recently switched from FreeNAS .7 to 8.0.2 which does remove a lot of features, so I added a Windows based server to supplement that until FreeNAS 8.1 or 8.2 comes out, I'm sure you could have a similar setup with Debian instead of Windows. My main reason for FreeNAS has always been ZFS with a decent RAID-Z setup, which I was very glad for when one drive failed a couple of months back. If all you really need is more storage then perhaps a FreeNAS based supplement would work well, link aggregation and iSCSI are pretty neat things.

Re:FreeNAS (0)

Anonymous Coward | more than 2 years ago | (#38492294)

Agreed - I'm very fond of my freenas box.
If you want many of the features that are missing, you can roll your own build.
Or you can go with freebsd - zfs functionality, and a library of programs.

Xbox Media Server (0)

Anonymous Coward | more than 2 years ago | (#38491818)

I saw somewhere that someone wrote software that replaces the old original xbox software with a pretty decent media server.

HP Microserver (5, Informative)

quenda (644621) | more than 2 years ago | (#38491834)

For a slightly more sane solution than rackmounting at home, consider the HP microserver.
Very low power (12W CPU), small, quiet, cheap, server grade, no Windows tax, holds four pluggable 3.5" drives plus optical (which some people swap for a 5th HDD for RAID5.)

http://blog.thestateofme.com/2011/05/14/review-hp-microserver/ [thestateofme.com]
http://www.silentpcreview.com/HP_Proliant_MicroServer [silentpcreview.com]
http://h10010.www1.hp.com/wwpc/us/en/sm/WF05a/15351-15351-4237916-4237918-4237917-4248009.html [hp.com]
http://forums.overclockers.com.au/showthread.php?t=905262 [overclockers.com.au]

If 8TB is full, you need to stop the obsessive collection of warez/pr0n/torrentz you are never likely to watch again.

Re:HP Microserver (1)

Anonymous Coward | more than 2 years ago | (#38492022)

"8 TB should be plenty for anyone."

Re:HP Microserver (0)

Anonymous Coward | more than 2 years ago | (#38492102)

Guessing that most is taken up by Watch Once (If at all), Store forever movies.
Cull your useless movies.
Collect only Movie Sets and Special Movies (For example Movies that tie in with TV Series or are Personal Favourites).

Re:HP Microserver (1)

Anonymous Coward | more than 2 years ago | (#38492110)

I've got one of those. Keep in mind that they consume about 55W of electricity while running, and do not support suspend-to-ram (ACPI S3). That means it will cost you about $5 per month to run it, unless you actually shut it down or hibernate it (ACPI S4 is supported). Fortunately, 55W is much lower than most other computers suitable for your purpose.

Re:HP Microserver (4, Interesting)

EdIII (1114411) | more than 2 years ago | (#38492118)

If 8TB is full, you need to stop the obsessive collection of warez/pr0n/torrentz you are never likely to watch again.

As opposed to the obsessive collecting of physical media that can be scratched and takes up 10x+ the space?

My family since the 80s has amassed literally 10k CD/DVDs as well as almost 100 laserdisc titles. Not to mention a buttload of VHS tapes that we offloaded years ago.

It has all been converted to digital storage. Since it is on multiple RAID 5 devices and I run a cron job that checks the MD5 sigs against a database I know that it is in good condition.

Of course this requires constant rollover of the data from hard drive to hard drive. Half the drives have failed over the years and it has moved between multiple NAS systems. We still have all the data.

In addition to that, we have over 100k family photos collected from all of our relatives scanned and tagged as well.

Our collection is nearing 20 TB. With the low cost of drives we have backups in lead lined containers in safety deposit boxes at two banks. We swap them out every year or so adding to it. I am really looking forward to long term archival storage that is write once and designed to last 100 years plus. I'll pay for that.

Now I know you may be thinking obsession, but we *paid* for it. Paying twice for music or movies is just plain insane and we never fell for the HD/Bluesuck shit they were shoving down our throats. Well my parents did, but Spiderman solved that problem the first time it could not be played because the encryption changed. Since then they are back on DVD only and we are waiting for a HD storage method that does not involve constant Big Brother monitoring and DRM in our houses.

Then there is the most obvious benefit of all. You only have to rip the music or movie one time. Been years since we bought an actual CD, but you get my point.

The convenience of having all of your media at your fingertips without touching physical media is pretty damn nice.

Guess how much storage space you need for thousands of DVD/CDs when they are packed into spindles and put into storage? A heck of lot less than you would expect. Fits in a closet.

Re:HP Microserver (1)

Anonymous Coward | more than 2 years ago | (#38492176)

Maybe you should work at a museum, clearly your archival talents are being wasted at home.

Re:HP Microserver (1)

EdIII (1114411) | more than 2 years ago | (#38492268)

Perhaps so. There really is no choice unless you want to eventually become a victim of Big Content. The GP strongly implied piracy as the only cause of such massive storage requirements. It's not at all.

It has to be stored *somewhere* and since managing physical media is a pain in the ass and *every* single use causes damage, another solution must be found. Be it ever so slight, there is damage with physical media. It makes no sense if we own the legal entitlements to a copyrighted work to keep it stored in a manner that constantly degrades. We have the right, both legally and ethically to take measures to protect our legal entitlement those copyrighted works. Now it would be nice if copyright law was more sane and works actually made it into the public domain within a reasonable time period. My family would pay $50 a month for access to a massive online public library of books, music, and movies. That would be more cost effective than creating it ourselves.

What about our family heritage? It's not just photos. Artwork, birth certificates, personal documents, etc. I can see within the next 20 years that diaries would be easier to keep, and more private than FaceStupid, with software designed to do it.

The article brings up a good question. What is the best way to keep a massive digital library in good condition and the most convenient way to access the data?

I gave my advice for how my family handles it and I really wish people would just drop Blueray support all together. Never buy into a copyright model where your legal entitlements and ability to protect said legal entitlements is limited by greed and unethical conduct.

migrate a bit at a time. (0)

Anonymous Coward | more than 2 years ago | (#38491860)

I had a similar situation a couple of years ago. The first thing I did was to invest in a high end Lian Li case that allowed 12 drive bays (with 3.5 to 5 adapters) and a very beefy power supply. Then I moved the motherboard and the drives to the new case.
  Expansion then consisted of adding a PCI SATA card, more drives, converting from raid5 to raid6, and then slowly upgrading all the drives to 2TB drives. Once all the drives were 2TB, I expanded the raid6 to include the 2TB on all the drives. I still have 4 bays left over for more expansion in the future, and the option to move to 3TB or even 4TB drives over time.

  This was all done over a period of time, and when 2TB drives were plentiful (and relatively cheap). In these times, it's probably smart to start with the new case and power supply, and watch for drive prices to come back down.

Check out UnRAID (1)

Anonymous Coward | more than 2 years ago | (#38491878)

Check out UnRAID here [lime-technology.com] . It is open source and lets you add disks more easily while providing redundancy. You should also check out the HTPC forum on AVS Forums [avsforum.com] .

My setup (1)

inkrypted (1579407) | more than 2 years ago | (#38491912)

I have been using Ubuntu server and Playstation Media Server (Don't let the name fool you. As for storage use XFS because it's tailor made to server large files. For storage these racks have colling fans and are allummium http://www.newegg.com/Product/Product.aspx?Item=N82E16816111045 [newegg.com] and I just add hard rives as needed. The laundry room worked best for me.

home media server? (0)

Anonymous Coward | more than 2 years ago | (#38491928)

but you can stream everything from the internet in 2011... in HD and without commercials...

get more internet, dude

No Garage (3, Insightful)

WoodburyMan (1288090) | more than 2 years ago | (#38491934)

I would think about putting the box in the garage. Yes it seems like a great location, it's out of the way and such. However it might not be the cleanest place in the house. I for one know my garage to be one of the dirtiest places. In the winter the car drags in massive amounts of sand from the the winter roads, and leaves in the spring. Spiders and other insects, not to mention baby snakes and rodents, also make their way in from time to time and would just live a nice warm dark place inside the case to live...in city area's it could also attract roaches in from outside. (Despite sonic repellents and traps they still get in). Combined all that with being near moisture (wet car or rainy days). I don't see the case lasting long there. It would need to be cleaned out fairly often to keep fans and heat sinks from gunking up. Of course I understand some people's garages are nice and clean, and not subject to some of these things, but just saying I know for me it would not work out well.

You need a NAS. (1)

pcx (72024) | more than 2 years ago | (#38491938)

Since you're not complaining about processing power or ram, you're in the market for a NAS. There are several good brands. I personally use Synology. It's a bit pricy but you get what you pay for. Personally I'd just add a few external USB drives until the prices fall (they're pretty outrageous now). When prices fall, get a nice 5 bay and stock it with 3TB drives that will give you ~12 TB in raid 5 and ~9 TB in raid 6 (recommended unless you like living on the edge).

You'll probably find the Synology can replace your existing media server (it has a pretty good support community) but since its all linux you can mount the nfs like another hard drive wherever you want the space in your existing drive structure. As your media server grows you can just get another NAS and keep expanding that way as long as you need.

NAS (1)

hakova (930861) | more than 2 years ago | (#38491972)

May I suggest setting up a NAS? Wherever you like (garage, or wherever). You can set it up with RAID, and use it as a backup server too if you would like. This way, you don't need to get rid of your current system but just add up. There are a few of them now with plenty of HDD space and flexible RAID configuration, so that you don't have to get all HDDs with same capacity, specifications, etc. Therefore, you can even upgrade HDDs as you go. I personally have Synology DS1511 [newegg.com] in my wishlist, although it is not a cheap toy. I don't feel I know enough to recommend any other file system. ext3,ext4, NTFS are what I use currently and I am naive if I am missing anything. Similarly, I don't know much about the differences between various linux flavors and BSD, therefore I think whichever you are most comfortable with should serve you well. x264/mkv is the format I use too, even for my blueray backups. It serves me well, and I personally don't plan to change it any time soon.

And why change? (0)

Anonymous Coward | more than 2 years ago | (#38491986)

It works ... don't fix it!

The end ...

I got a Mac Mini Server, and I have way too many external drives (2x4TB FW800, 2x3TB FW800/USB, 3x1TB FW800/USB, and 1x6TB USB in RAID 5, the latter being for primary backups of everything sensitive)... The server is used for everything, from song repository (All lossless) to Plex server, Backup server, Mail server, PS3MediaServer, iTunes server, Sonos server, VPN base, I even rip my vinyls on it right now ... Seriously, it works for me. As I'm a pro photographer, @ 20megs per picture, and 1500 pictures per event avg, I literally eat out HDDs. Adding more and more as I need them.

And this will probably not work for anyone else. I don't care about anyone else, it works for me. I don't want my fiancée to be able to add up a new drive or switch a failed HDD, I don't care anyone else managing this beast. Anyone from family needs to connect, I give them a password for their service, and they have their own little world they can do whatever in it.

Next purchase is a Ethernet-based NAS I will put in the basement inside a breathing fire-proof box for a tertiary backup (no, transferring 40-60 gigs worth of data every week is not supported by my provider, and I got other things to do than annoy my friends every day).

Let Me Google That For You (0)

Anonymous Coward | more than 2 years ago | (#38492000)

This question is asked every what... 2 months? 3 maybe? Go to Newegg. Pick out a case. Good vendors are SilverStone, Antec, ThermalTake, Lian-Li, and FractalDesign. Don't find what you want? Check around the net until you do.

Then buy Intel hardware (AMD will work if you're diehard but why buy inferior hardware for the same price, these days) with Nvidia video (for VDPAU, assuming you want vid output for a "media server," maybe not. AMD chips support an extremely limited subset of VDPAU.) if you want maximum compatibility and best drivers (yes even on Linux).

Install FreeBSD or ZFS For Linux (not the FUSE version, the kernel module. installs with DKMS & runs natively. available in a PPA if you use Ubuntu.) and put it in ZRAID. Incredible speeds for software RAID and save a ton of money on RAID cards. Your HDD platters will bottleneck before ZFS does. Infinitely configurable. Also since SATA is a Serial protocol, it supports chaining of devices at full speed like USB. Many people don't realize this but all you need are some port multipliers, as long as you're not trying to do anything mission critical. And if you're lazy FreeNAS works like a charm with practically zero setup.

This is not a difficult question anymore and I don't know why it's being posted. Energy efficient, silent home servers can be bought for peanuts these days, even if you get a low TDP Xeon. Have enough money for a proper rack set up? Then why are you asking here. Go ask your Dell rep or vendor of choice. And by the way, Cat6 is not necessary for any home installation; it is designed to be used in industrial environments with high EMI - it can also be used to run 100Gbps Ethernet if you have a multi-million dollar hardware laboratory at your disposal. Cat5e is 2-3x less expensive than Cat6 - and be sure to get stranded core cable, not solid. You might as well run fiber if you're going to do Cat6, it's just as expensive these days.

Myself? I use a FractalDesign Array R2 MiniITX case with a MiniDTX Zotac Intel motherboard with a Xeon 1260L (Sandy Bridge, 40 WTDP). Six 2TB WD Green drives in ZRAID for 10TB redundant, useable space - Addonics 5 port SATA multiplier and Corsair 40gb SSD for the root drive. The motherboard has a Zotac Nvidia ION2 card plugged in which runs any video I throw at it without issue. 4gb of RAM and everything else standard. Cost around $850 total but that's when hard drives were cheap, probably a bit more now.

Hard drive math and hobbyist heuristics (5, Informative)

Wrath0fb0b (302444) | more than 2 years ago | (#38492004)

Ballpark figures, this isn't exact, redo it with your preferred constants, I'm just trying to explain my reasoning against huge enclosures with > 10 drives,

Standard drive idle usage (W) ~ 10W [1]
Low-power (green) drive idle usage (W) ~ 5W [1]
Cost of power ~0.20 $/KWH
Cost of an older drive per year = $17
Cost of a green drive per year = $8.50
Replacing 6x500GB older drives with one 3TB green power savings = $95/yr

So think about that for a sec. At $150[2] for a 3TB drive, you cover the price in power savings in 18 months. That's assuming that there is zero fixed-cost per drive. At the point where you are talking about adding SATA controllers or fancy multi-bay enclosures or, worse, external enclosures with their own PSUs (and fans!), the turnaround-point for older drives is far sooner.

I'm a hobbyist, I understand that it's really cool to make do with older hardware and feel like you aren't letting anything to go waste but sometimes using old hardware instead of buying new is penny-wise and pound-foolish. Spending money on increasing how many hard drives you can accommodate instead of just buying newer high-capacity lower-wattage drives is absolutely batty; especially when you get into the price for anything remotely good in the RAID dept.

My advice, move everything to the largest capacity drives that are reasonably priced (after the flood damage is sorted). Replace the drives when you can do between 4:1 and 6:1 replacement -- should be every 3-4 years. Live happily, quietly and simpler. Save money on power transparently.

[1] http://hothardware.com/Reviews/Western-Digital-2TB-Caviar-Green-Power-Hard-Drive/ [hothardware.com]
[2] I bought some Hitachi 3TBs before the Thailand floods at $130 on Newegg. Of course you would be silly as heck to buy hard drives now for your hobby storage project before they at least fall back to pre-flood level.
[3] http://www.newegg.com/Product/Product.aspx?Item=N82E16817182221 [newegg.com]
[4] Older drives need not go to waste, they can become offline storage with a simple USB dock[3] -- make a backup, throw it in an anti-static bag, leave it at your relative's house when you visit!
[5] http://en.wikipedia.org/wiki/File:Hard_drive_capacity_over_time.svg [wikipedia.org]

Re:Hard drive math and hobbyist heuristics (1)

Anonymous Coward | more than 2 years ago | (#38492184)

Yes because nothing says "living simple" like needing every movie ever made on a bunch of hard drives in your living room. The things you freaks obssess over... (rolls eyes)

my samba server (1)

Anonymous Coward | more than 2 years ago | (#38492010)

I built a samba server a few months ago. it has 6 sata connections on the motherboard. I'm running it with 6 3 tb hard drives running raid 5 and i'm loving it. it has a lot of space so its very future resistant, and since I chose to go with an amd fusion cpu it is very low power. dual core 1.6 ghz. and the cpu heatsink is fanless, so there's no sound at all. ram is dirt cheap so I threw 8 gigs in there just because it was like 40 dollars.

the amd fusion motherboard cpu combo, 6 3 terabyte drives and 8 gigs of ram cost me like 1 thousand dollars. look around on newegg there's lots of options. hard drives are pretty expensive right now, though, luckily i got mine before they shot up. the one i got also has a pci-express expansion slot so you can easily turn that 6 sata ports into 10 with a sata card. there's no need to buy a server rack.

I'm running ubuntu server just because it was easy, but you could just as easily do it on debian, and if you're a free software/open source advocate debian is probably the way to go since it's running no binary blobs and the open source software is seperate from the core repos. though ubuntu 12.04 is going to be supported for 5 years, so long term that may be the way to go, if you want to wait that long.

I would recommend sticking with h264 though, because even with open source software its very well supported, and it's the best thing out there right now.

at the moment ext4 is the standard for most linux based operating systems, but it's going to be replaced by btrfs in the coming months(years?) once all the bugs are sorted out.

as for media software, I assume you mean on the fly transcoding media streaming software? I would look into ps3 media server or mediatomb. if you don't need to transcode on the fly why not just use a samba server to access your files?

also if you want to shy away from linux take a look at freenas which is based on freebsd, and uses the ZFS filesystem which has built-in raid5-like support, along with about a million other features. just so you know though it very strongly recommends a 64 bit compatible cpu and needs a lot of ram, though an amd fusion cpu with 8 gigs of ram should cover your needs just fine, since you have no use for deduplication.

Restrictions (1)

xushi (740195) | more than 2 years ago | (#38492032)

Hmm,

Correct me if I'm wrong, but doesn't Debian have limitations or restrictions when it comes to Media ? (codecs, etc..). I know one can bypass them with other repositories, but still that would be one thing to consider..

I for one, would stick everything in a NAS (like a readynas) for added reliability etc, and use a Mac laptop or Mac Mini connected to the TV with XBMC over SMB. That's what I have set up.

I would also probably go for the same with Linux, but with XBMC as the front end too.

Good luck!

Re:Restrictions (1)

xushi (740195) | more than 2 years ago | (#38492048)

Edit:

Just to add more info, here's an example of the ReadyNAS (NV+ RND 4000) I was referring to, with my review of it.

http://www.amazon.co.uk/review/R3MK92814ZSDKM/ref=cm_cr_dp_perm?ie=UTF8&ASIN=B000VKJEBK&nodeID=340831031&tag=&linkCode= [amazon.co.uk]

Stick 4 x 1TB or 4 x 2TB in there with X-Raid (Raid 5) and enable NFS, AFP, and SMB. Recipient or multimedia device should be powerful enough to view 1080p media, through any of the protocols above via XBMC :)

Get a NAS (1)

Anonymous Coward | more than 2 years ago | (#38492034)

I went from running my own servers (debian based) to just a simple NAS (ReadyNAS Pioneer is what I currently use). I got so tired of doing maintenance on the server al the time, and the family got tired of it sometimes being in various states of flux as I was upgrading or adding something. The NAS supports the regular streaming and file sharing protocols and updates when you want to apply them are just a button click via a browser interface. When I run out of space I just swap out the old disks one at a time, the server rebuilds the raid on the new disks, and once I have swapped them all out the NAS automatically re-sizes the entire volume. I use XBMC (xbmc.org) for a media client on a variety of endpoints (desktops, laptops, jailbroken ATV2, etc) and it all 'just works'.

I am also going to move mail from my house, probably to a gmail hosted solution because maintaining a mail server and spam filtering etc is also getting old. I still have some systems I can tinker around with, but the rest of the family likes their 'production' systems to just work.

You need RAID... (0)

Anonymous Coward | more than 2 years ago | (#38492040)

and you need at least one backup and you need a UPS. If your RAID hardware fails (such as the RAID card), you could be sunk without a backup. If you're using software RAID, you could lose everything with a mainboard failure or system crash. RAID is not a backup and still has single-points of failure.

Sounds like you're a hoarder, too, which is a polite way of saying you collect a lot of crap. If you've that much time, why not try to be more discerning and be rewarded by saving big bucks on the hardware.

My optimal solution (0)

Anonymous Coward | more than 2 years ago | (#38492050)

freebsd - because ZFS has been great for me. I'd recommend freenas, but not if you want to use it for more than a file server.
A case similar to this: http://www.missingremote.com/review/fractal-designs-arc-midi-mid-tower-case
Many drive spaces in a reasonable-sized case.
A decent cheap to mid-range SATA adapter.

Re: (1)

Anonymous Coward | more than 2 years ago | (#38492060)

> "I started off building a media library a few years ago with an old PC running Ubuntu. Folders for photos, ogg vorbis music from my CD collection, and x264 encoded mkv movies. I have a high spec machine for encoding, but over the years I've moved the server to a bigger case, with 8 TB of disk capacity, and reverted back to Debian, but still running with the same AMD Sempron processor and 2GB RAM. It's working well,

OK, so it's working well. Why are you fixing what isn't broken? Regarding any CPU choice, I've ran at least a dozen types of CPUs with Linux or BSD on them, and they are all good. I'm fond of the m68040. What I'm saying is that CPU choice doesn't matter. With x86 CPU's your choice is between uber cooling and overclocking versus twitchy overclocking. AMD's tend to be more tolerant of very low temperatures, in case you have some dry ice chillin'. Regarding RAM, it's cheap, buy as much as possible. Fill all the slots, and test the BIOS' limits. RAM has always been my upgrade bottleneck (what I run out of first). I'm mad I didn't do exactly this with my current builds. Currently running AMD because it's cheaper, or so they say.

> it's also the family mail server, and the kids are starting to use it for network storage, and it runs both link and twonkyserver, but my disks are almost full, and there are no more internal slots. The obvious option to me is to add in a couple of SATA PCI cards, to give me 4 more drives, and buy an externally powered enclosure, but that doesn't feel very elegant.

Get a new server! That's what you want to hear. Just find some nice large full tower case (10+ drive bays, easily) and spend as needed on it's innards. Building is the fun part, so try to do something you haven't done. Multiple SATA cards are for sure going to be needed, as well as plenty of drives. Don't forget the spare(s). It has been my experience that running offboard and onboard SATA with it all in the same array tends to suck. I've done it on a few wildly different machines and it's never pretty for some reason. Might just be luck.

> I'm a bit of an amateur, so I'd like some advice.

Hi! You must be new here! Welcome, try the coffee cake!

> Should I start looking at a rack system? Something that can accommodate, say, 10 3.5" drives (I'm thinking long term, and some redundancy)?

Probably not. Racks are good if you can build around them, but for most amateurs it's out of the price range, however with some digging for deals this setup could probably be done for $300 or less, depending on how many servers and how fancy you want to get (12v rails? Solar backup batteries?). Don't expect your existing hardware to work with the cases, unless you don't mind using giant rack cases. Racks are best for when space is a consideration and money isn't. Normal cases are best for everything else. The exception to this is if you have a nice location already handy, like a suitable wired & cooled closet, and you *really* want to use racks.

> Also, what about location — I could run some cat6 to the garage and move it out of the house, in case noise is an issue.

A garage is a bad choice depending on your local climate because heat is your enemy and because of the large ambient temperature swings. You will need to cool this thing, at the least. Noise is never an issue, you're a nerd, loud cooling fans come with this territory. Either get over it, quiet it, or delve in to watercooling.

> Finally, what about file format, file system, and OS/software? I'm currently running with ext3 and Debian Squeeze. Happy with my audio encoding choice, but not sure about x264 and mkv. I'd also consider different media server software, too. Any comments appreciated."

Here at the house I run a Debian shop as well, and also still use EXT3, which is to say that if isn't broken, then don't fix it. I can't comment on x264 vs mkv, but I prefer mkv for no particular reason. I just don't care enough. What do you mean by "media server software"? DLNA is the current hotness, and it all sucks and is impossible to use. XBMC is well known, and I use foobar2000 as a server (the plugin hasn't been updated in forever, but still works perfectly) and UPnPlay as a client/whatever on my handy dandy android tablet. I haven't gotten around to finding a decent DLNA thingy for Linux yet. Since when I'm at home I prefer to just jam out with the audio system on my PC, I haven't spent much time in this arena. My phone and tablet both have a shedload of storage available, so I just load 'em up from time to time like I always have.

It sounds to me like you need to come up with some requirements. What must this new server do, and how must it do it? Answer those questions and finding a fancy case to shove it in to will be pretty easy. As for your old server, keep it around but have it just host the mail and whatnots. Dedicate it to the kids perhaps. If you don't want to get a new server then pretty much just do what you said, with the external enclosure and all that. Computers aren't supposed to be pretty, they are a tool meant to be used & abused. Have fun!

ZFS? (0)

Anonymous Coward | more than 2 years ago | (#38492064)

For your use case(s), I would strongly consider a ZFS-based solution. I have similar requirements and have had loads of success running the community edition of Nexenta - a Solaris-based OS with a Debian (i.e. apt-get) user land. Kindof the best of both worlds.

my setup (1)

Darkseer (63318) | more than 2 years ago | (#38492066)

Price is alway an issue. You are using the space so you should invest in decent storage capacity. I personally spin about 15 SATA drives off of a single raid card.

I recomend the following:
Build a decent NAS head with the following recipe and share it out to all your other devices.

buy a 4U case with ~ 20 drive bays that link to a SATA bus ~$300 (http://www.newegg.com/Product/Product.aspx?Item=N82E16811219041)
                -You may need to mod the case if the port multiplier is SAS instead if SATA
buy a high point RAID controller in the 2000 class or higher (http://www.newegg.com/Product/Product.aspx?Item=N82E16816115049) (32 drives max with 4 port port multipliers and mini sas - 4 sata cable)
Start with 4x 2TB drives (RAID 5, 3 + hot spare)
1 gigabit ethernet hub

then attach it to a cheap cpu with a good amount of ram and gigabit ethernet. Stick the storage kit in the garage and run cat 6. Put the hub in the house and wire it to any machine playing back HD video, for the rest I find you can get away with wireless G.

You are looking at about a 1K investment in kit for storage. For OS I'd use your favorite flavor of linux, gentoo for me but I recomend deb or ubuntu for other people, that supports lvm. This way you can add new drives and expand the storage dynamically. You can expand ext3 or ext4 easilly, you just can't shrink them. In my opinion there no need to go exotic with the FS.

Once you set up the storage share is both via nfs to anything playing high def content and samba for everyone else. Then sit back, relax and enjoy.

External Drives & RAID (1)

nion (19898) | more than 2 years ago | (#38492068)

I'm doing about the same thing, though my overall capacity is much lower due to the fact that I haven't been able to upgrade my 5 750gb drives in a few years. However I went to an external enclosure (Venus T5 from Newegg) and raid5. If the box is in the house, yeah I can see how you might not want an external enclosure next to the tv. However, once you stick it in the garage, that becomes moot. Who cares what it looks like in the garage? Run some cat6 to it, put a gigabit switch on the network (or 2 or 5) and enjoy having a nice, quiet living room. I'm using boxee boxes as the media frontends - one actual dlink box and a pc running ubuntu and boxee.

I personally hope HD prices start to drop sometime soon, I'd REALLY like to upgrade those drives but can't justify the prices right now for the size I'd like (at least 2TB).

Was going to suggest... (1)

gQuigs (913879) | more than 2 years ago | (#38492092)

The ZaReason MediaBox (http://zareason.com/shop/MediaBox-4220.html) but you have clearly outgrown that already... Both ZaReason (http://zareason.com/shop/Servers/) and System76 (http://www.system76.com/servers/) have server models that look like they would meet your needs. I usually prefer to buy from one of them (ZaReason will ship with Debian already). Another thing I would look at is using a video card for encoding. Couldn't find a link for how to do this, if anyone does please chime in. As for filesystems ext4 is the easy choice. If you build it in a year (and have update software including fsck for btrfs) I would go with that. For the software... what are using on the frontend/need to be compatible with?

Re:Was going to suggest... (1)

gQuigs (913879) | more than 2 years ago | (#38492104)

I swear I've changed that setting before...

Dammit.

A better chassis (1)

GPLHost-Thomas (1330431) | more than 2 years ago | (#38492098)

If the issue is just plain physical space for putting more HDDs, get one of these Supermicro storage solutions [supermicro.com.tw] . There's from 15 to 36 HDD chassis, so take your pick. It will cost "only" few hundred bucks. Then yes, you don't want this to sit in your main room, it would be too noisy. If there's no humidity in your garage, and it doesn't get too hot in there in the summer, then using a cat6 to it should be fine. If you don't want to change your motherboard just right away, then just use the old existing one you have currently for the moment, anyway any ATx board will fit in.

Build a new box.. (0)

Anonymous Coward | more than 2 years ago | (#38492100)

Build a new box and do it yourself. Pre-fab is never for nerds.

As far as a case with lots of room to grow storage, I recommend the Fractal Design R3. It has 10 HDD bays with noise dampening mounting, noise dampening materials on the panels, 7 120mm fan mounting points, and comes in around $100. Corsair has some good kit too, and these days it's not hard to find a mobo with 8 sata ports...

Rack system (0)

Anonymous Coward | more than 2 years ago | (#38492106)

Get a Norco RPC-4220($350 - 4U / 20Drives) and then grab a couple SUPERMICRO AOC-SASLP-MV8 ($110ea). Get whatever MB works for you and you don't need as much a PSU as you'd expect(I have a Corsair CMPSU-620HX). The case is a little on the cheep side but a ton of my friends also have one and they work just fine, especially for the cost!

While I use WHS to manage the disks, my friends who are linux geeks would want me to say ZFS is absolutely the way to go and if I had their skill set I'd agree. Sounds like you could probably get ZFS working. I'd say all of us are quite happy with these setups. Now is a poor time to be buying drives tho :\ 3TB drives and even the 2TB are pretty expensive these days.

Oh, I would recommend buying some new case fans: Cooler Master 80mm Ultra Silent ... or whatever you like. I've found many MBs don't have the fan controllers needed so the fans run at full speed which is loud.

Serviio for media server (1)

eshadowrun (917452) | more than 2 years ago | (#38492130)

For media server, I recommend Serviio [serviio.org] , and not _too_ worry about video encoding that much (on-the-fly transcoding, but a few mkv files do have issues, depending on how they were encoded). And Linux/Windows/Mac version are available for free.

Dedicated unRAID server box (2)

theyost (973129) | more than 2 years ago | (#38492164)

Once your data grows past a certain level redundancy becomes a must. For about 3 years I have been running an unRAID server & I don't know how I survived before it. It allowed me to start with 3 drives (approx 250MB each) to my current configuration of 14 hard drives (mostly 1TB but also a couple smaller drives and three 2TB drives). After you have the system setup you don't need a monitor. I have it in my basement which helps with noise issues that result from having 12 drives spinning 24/7. Because this is a SOFTWARE-based raid-like system I can mix/match different types of drives to get the most storage for the money. I HAVE had a drive fail and the rebuild was easy. Another plus is the data is NOT stripped so if the motherboard ever fails the data can still be accessed (any computer that can access data on a "reiserfs" (I think) filesystem). One recommendation... Wire for Gigabit Ethernet! Once you start transferring HD Video (terabyte-size files) you'll find it helpful. Another recommendation... Find a smaller (100GB-200GB) drive to use as a "cache drive." This will speed up the transfers to your unRaid box :) Good luck! PS1: http://lime-technology.com/ [lime-technology.com] PS2: I have been running version 5.0-beta14 without any issues

Thoughts from my home storage server experience. (4, Informative)

jafo (11982) | more than 2 years ago | (#38492168)

I wrote about the latest storage server I built back in 2008, and a lot of my thoughts at the time are written up in http://www.tummy.com/Community/Articles/ultimatestorage2008/ [tummy.com]

However, to answer a few of your questions...

External disc enclosures? Avoid them like the plague. My initial experience with the 5 bay eSATA enclosures was pretty good -- sometimes it wouldn't pick up the external drives, but usually I could get it to find them after some tweaking, rebooting, etc... I ended up getting 3 of them, the AMS DS-2350S, which at the time were well reviewed, etc... I have since pulled all 3 of them out of active use and have them just sitting around. I don't know exactly the mode of the failures, but eventually after replacing some with others, I finally put them in internal SATA enclosures, which have been very reliable (I used these Supermicro CSE-M35T-1.

Also note that eSATA connectors don't really hold on that well. If anything, they're not as robust as internal SATA connectors, despite being outside the case where they can get banged around.

If I were to do it over again, I'd probably stick with the case I started with, with 5 internal 3.5" bays, and 3 front 5.25" bays, and put the Supermicro in there. I'd also probably go with fewer big drives rather than more smaller drives like I did previously (even though at the time the drives were free, I had them from another project).

As far as running it in the garage, don't even think it, unless your garage is not where you store your cars. I have some computers that I've run in the garage for the last 9 months, and they are filthy, I've had a lot of fan failures, lots of dust, insects, and random other crap. I put mine in our furnace room, which has enough extra space.

As far as using a server case? Hard to see the payback there unless you have a cabinet. Most server cases are HUGE, heavy, and expensive. A 3U case with 12 drive bays likely costs $500, plus you usually have to deal with special form-factor power supplies, expect to spend another $200 on one of those. I wouldn't do it, and I have a 3U 12-bay Chenbro case just sitting at my office that I could re-purpose.

As far as the file-system, I selected ZFS (via zfs-fuse under Linux) and I've been VERY happy with it. The primary benefit is that it checksums *ALL* data and can recover from some types of corruption or at least alert about corruption if it can't correct it. So, if you are storing photos or home videos that you may not be accessing very often, that's good peace of mind to have, I know in 10 years I won't go to look at some photographs I've taken and find they were silently corrupted. Of course, you could get similar benefits by saving off a database of file checksums and checking and alerting if they are bad. Really the only downside of ZFS that I've seen is that if you need to do a RAID rebuild it is a seek-heavy task rather than just streaming. I have a 8x2TB drive array that I'm currently rebuilding (drive failure, at work), and it's 33% done after 31 hours. A normal RAID-5 array would have rebuilt that in what, 10? The system is idle except for the rebuild.

If you care about the data going into it, make sure you checksum and verify the files regularly.

The 8 port PCI SATA card I got is fantastic, it's a Supermicro with the Marvel chipset and is very well supported (even supported by Nexenta).

Finally, all this data is encrypted, so if someone were to burgle us I only have to worry about them getting the hardware, I don't have to worry about them now having scanned bills and other documents and other personal and private data, etc... This is why I'm running ZFS in Linux, it gave me encryption plus ZFS (not available otherwise in 2008), as well as being an OS I'm very familiar with.

As far as OS, I am personally running CentOS on my system because that means I can install and set it up and then forget about it for quite a few years, except for regularly running "yum update". Debian should be fine, but you will get/have to track upstream changes more frequently.

8TB? (0)

Anonymous Coward | more than 2 years ago | (#38492172)

Seriously? I've been ripping DVDs, several per week, for years, and I am not even near 8TB.
I have probably 300 movies at 1-2GB each (High quality AVC/AAC, with all original audio tracks and closed captioning saved), which makes about 400GB.
I have another 3TB drive that saves the original full size DVD rips, but that's hardly necessary.
With an 8TB drive, you could old over 4000 movies at a reasonable encoding - that seems a bit excessive unless you are opening your own iTunes store.
I guess if you are an illegal downloading fiend with BitTorrent running 24/7, you might fill up that amount of space, but then you would also end up with 27 different file formats, various quality encodings, etc. Buying that many movies on DVD, or on iTunes would cost a fortune. Either way, I wonder who would find time to watch 8TB of movies. Just properly tagging the metadata would take you forever.

I think the first issue you need to address is separating the use cases.
Get a separate server or NAS for storage. (And make people delete stuff they don't reasonably need - the average person has no problem fitting everything important in 80GB or so - if your family needs terabytes of storage, they are either storing raw video, VMWare images, or are junk collectors).
At any rate, the NAS could easily have 4 drives of 3TB each, making 12TB.

The TV connected thing can be separate, and probably doesn't *really* need more than 1TB, but depending what kind of machine you use, you could put 1 or 2 internal 3TB drives. Also note that drives are still increasing in capacity and decreasing in price, so it doesn't make sense to reserve space for 10 drives, especially when something like SATA3 may come out in a few years and turn all your investment into waste.
(I use a Mac Mini with 1 external drive because it's low power and can also run iTunes without Windows).

As for encoding, OGG Vorbis quality is excellent, but quality is low. There is a theoretical patent issue, but for practical purposes, AAC will run in everything, and sounds better than MP3. Likewise, AVC/AAC in an MP4 container will run on iPad, etc., so I don't see a lot of advantage to using MKV, even though it might be a fine format.

Why so complicated? (0)

Anonymous Coward | more than 2 years ago | (#38492188)

I see everyone suggesting expensive server setups, drive chassis, even racks! For a simple home media server you dont need much. I currently have a simple big tower chassis (11x5.25" and 3x3.5" bays) with a generic motherboard (1xPATA, 4xSATA) that holds 10 hard drives. How? I stuffed it with a couple non-raid controllers and 5.25" to 3.5" bay converters. Theoretically, I can expand for up to 18 drives before even considering a second chassis. Just make sure you have a beefy power supply to take the load when you turn it on. Ive also applied software raid to protect myself from failing drives, which is hardly taxing the CPU. Software raid has an added bonus that you can mix drives of different sizes by smart partitioning.

*Not* RAID 1 (1)

Theaetetus (590071) | more than 2 years ago | (#38492210)

I went away from RAID 1 for my home server - mirrored drives are great when you want exact copies of data with a backup in case of drive failure... but that's not precisely what I want - I want the right copy of the data. If the main drive is corrupted, then I don't want that corruption to be copied to the mirror drive. Writes can be verified... retained data, less so.

So - and feel free to replace this with your OS of choice - I went with OSX and Time Machine. The main drive stores all of my audio/video/picture files, and I do incremental backups to a second drive. If a file is corrupted, I can roll back a week and restore. If the whole drive dies, I can do a full restore from the backup.

As for other levels of RAID - the biggest bottleneck in your system is probably going to be your network, not your hard drive read speed... Though we'd all love to have RAID 5+1 systems, there's really no need if it's a media server... Realistically, you're only serving one stream of data at a time, maybe two, but that's it.

coraid (0)

Anonymous Coward | more than 2 years ago | (#38492240)

http://www.coraid.com/products/san_storage

Check out the SheevaPlug (0)

Anonymous Coward | more than 2 years ago | (#38492242)

It's a plug computer with 2 or more USB ports. Ethernet ... all the usual stuff.
$99 last time I looked. Then hang a USB 2.0 hub or two of it, add drives as needed.

The documentation online is a bit sparse, but I saw has a hdmi port (probably just a
text console with the limited memory).
So hook to a cheap monitor, usb keyboard and mouse, set it up, disable the F1 interrupt
on boot if no monitor present stuff, plug in a closet with an ethernet cable to the hub/switch, drives and
your off and running.

Thinking about it myself.

Load More Comments
Slashdot Account

Need an Account?

Forgot your password?

Don't worry, we never post anything without your permission.

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>
Create a Slashdot Account

Loading...