Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

What Do You Look For in a Big Iron Review?

Cliff posted more than 9 years ago | from the ask-and-he-will-try-and-provide dept.

The Media 262

ValourX writes "We're starting to write more reviews of enterprise-class hardware and software and although we've done pretty well with our reviews, the high-end products are a lot trickier when it comes to testing and evaluation. Obviously it is not possible to build an enterprise-grade 'your neck is on the line' production environment just for writing reviews, but maybe we can do something smaller, just for testing purposes. What do you as an IT professional want to read in a review for a server OS or a high-speed switch, or a big iron server or proprietary workstation? What tests should we run? What results and feature comparisons are going to be most meaningful to you?"

cancel ×
This is a preview of your comment

No Comment Title Entered

Anonymous Coward 1 minute ago

No Comment Entered


Not Speed (5, Informative)

jellomizer (103300) | more than 9 years ago | (#10976009)

Well the 2 main issues with Big Iron Equipment is How Well it handles Load and Scalability. For Load They should max out the system slightly above the recommended specs and see how well it handles it. Most people don't care for overall benchmark but more issues that affect the user. Say it was a WebServer We don't care how many pages/second it can handle but how well we get the webpages when the system is maxed out. Do we have to wait 5 minutes and the page just pops in. Or do we wait 5 Minutes for a page to load but we see the results of it coming in. When working above the required load how much does the system heat up (causing possible failures in the future). Secondly is how well can it scale, Can Extra Processors be added on, Can you add/hotswap processors on the system. What is the Max Ram it can hold can you add more is there room to add more. How compatible is it with competitors stuff (Say an IBM Server with a Sun Storage Array) how well do they follow the standards so you are able to use the server even if the company who produced it died.

Speed (which a lot of people put there Big Irons to the test) is really not that important of a detail. A PC with a 3 Ghz Processor will out perform a Sun Fire15k with multiple processors, for any single task. But when it starts handling load the Sun Fire will handle it better then the PC. When companies decide to buy the Big Iron they want it to be an investment that can last them at least 3-4 years preferably 4-10 years. And all they need to do is add stuff to it so that it scales with the time.

Re:Not Speed (-1, Flamebait)

Anonymous Coward | more than 9 years ago | (#10976119)

a WebServer We don't care how many pages/second it can handle but how well we get the webpages when the system is maxed out. Do we have to wait 5 minutes and the page just pops in. Or do we wait 5 Minutes for a page to load but we see the results of it coming in.

Yes. That definitely has to do with jardware. Not how the server software handles requests. Sure.


Re:Not Speed (1)

jellomizer (103300) | more than 9 years ago | (#10976189)

From the Article.
IT professional want to read in a review for a server OS or a high-speed switch, or a big iron server or proprietary

A server OS doesn't have much to do with the hardware but it has a lot of do about handling requests. Also If you have a 1 CPU 3 Ghz system vs. 3 CPU 1 Ghz System. In most cases the 3/1 Ghz system will handel the load better then the one with the faster chip because information on the 3 CPU can be processed at the same time. (Yes I know there are not many 3 CPU systems they are usually in 2^x (2,4,6,8,16,32,64,128,256,512,1024...) I just used 3 because right now most fast processors are in the 3ghz range.

Re:Not Speed (1, Informative)

Anonymous Coward | more than 9 years ago | (#10976263)

1 CPU 3 Ghz system vs. 3 CPU 1 Ghz System

Ashat, if you have ever done any real or psudeo simulation you wouldn't have said that. The 3 ghz system will outperform any (yes I said any) 3 by 1 ghz system. You have to factor in things like bus transfering information from one cpu to the other. Unless your bus is runing at 1+ghz even then a cpu has to divy up the work. So to complete debut your post, all simulation and real world expreinces I have witnessed shows that a single cpu of a given speed is faster than a multiples cpu that add up to the single cpu speed.

Re:Not Speed (5, Insightful)

Dr Caleb (121505) | more than 9 years ago | (#10976272)

For Load They should max out the system slightly above the recommended specs and see how well it handles it.

Not a bad idea, but all I see is the manufacturer lowering the maximum specs to any tests will show it 'overachieving'.

What I'd like rated is the support side. My AS/400s self detect hardware problems, phone IBM to report the problem, and a tech is dispatched. The IBM support centre phones me to tell me the system detected a problem, and that a tech is on the way. Usually the tech shows up with parts in hand inside an hour. Before the hardware has caused any downtime! I've never had a catastrophic failure on an AS/400.

Good support, redundant and hot swappable hardware, like RAM, makes for the best big iron. Low to no downtime are just as important as throuput and storage.

"WebServer"? (1)

Saeed al-Sahaf (665390) | more than 9 years ago | (#10976325)

Say it was a WebServer

Do they even use "Big Iron" for web servers? Very much? Aren't they all mostly SPARCs or Vanila Intel? File serving and number crunching would be the standard "Big Iron" useage, right?

Re:Not Speed (1)

sunryder (192810) | more than 9 years ago | (#10976333)

Upgrades are not a big concern for many customers (especially larger customers). Usually by the time customers are ready to upgrade, it is simply cheaper to purchase entirely new hardware.

I read that in a report several years ago and didn't beleive it myself. Then I started a small IT business (with a partner) and have completed several server redeployments. After the third time doing this, I remembered the old article. I tried to find it, but couldn't find the exact one again. At any rate, since then, we've been taking this into account when purchasing new hardware. Then I read this today, and it reminded me again about this.

The price/performance ratio of new machines increases so much, that it just doesn't make sense to upgrade older hardware in terms of the performance or cost.

Re:Not Speed (2, Insightful)

Fulcrum of Evil (560260) | more than 9 years ago | (#10976335)

For Load They should max out the system slightly above the recommended specs and see how well it handles it.

Nah, push it until it falls over and see how it degrades. Stick a line on the graph where the rated capacity is.

Stupid question (1)

apankrat (314147) | more than 9 years ago | (#10976340)

Say it was a WebServer We don't care how many pages/second it can handle but how well we get the webpages when the system is maxed out.

Wouldn't this to a much larger degree depend on the software rather than a hardware ?

Re:Not Speed (-1, Offtopic)

Anonymous Coward | more than 9 years ago | (#10976343)

Damn, your post gave me a headache. The content was decent, but your writing is horrible. First, you Seem to capitalize Random words In sentences. Secondly, "secondly" is not a word. Also, have you ever heard of a question mark. Please, try harder next time.

Most important feature (5, Funny)

RandoX (828285) | more than 9 years ago | (#10976014)

If you knew our operations guy, you would test resistance to physical attacks.

Re:Most important feature (2, Informative)

Bimo_Dude (178966) | more than 9 years ago | (#10976195)

Hey! You know as well as I know that almost all or our servers resist physical attacks fairly well. Workstations on the other hand...

Lots of little (0)

Anonymous Coward | more than 9 years ago | (#10976239)

..green men hogging peoples karma these days... [freeshell.org]

Remember karma isn't REALLY an integer ranging from -1 to +5

Mostly... (4, Funny)

mr_z_beeblebrox (591077) | more than 9 years ago | (#10976017)

Pictures. I like hot chicks standing next to big servers. Big servers in action shots are good too.

Re:Mostly... (4, Funny)

lucabrasi999 (585141) | more than 9 years ago | (#10976038)

I like hot chicks standing next to big servers

I just love that this has been moderated as "Insightful".

Re:Mostly... (0, Flamebait)

lucabrasi999 (585141) | more than 9 years ago | (#10976066)

Offtopic? You mean pointing out the mistakes made by moderators is Offtopic? Good! I got Karma to Burn!!!!

Re:Mostly... (0)

Anonymous Coward | more than 9 years ago | (#10976088)

For the love of god, everyone knows this by now... if you want someone who posted a funny comment to get karma, you don't rate it funny. Funny doesn't give you karma. So put insightful, interesting, etc instead.

Re:Mostly... (1, Insightful)

lucabrasi999 (585141) | more than 9 years ago | (#10976150)

No shit?

Not to drag this even further OT than it already is, but I already knew that. However, when moderating, you should moderate what you think the thing is. If it's funny, moderate it funny. If it's off topic, moderate it off topic. You shouldn't be moderating just to add karma to a poster.

If you are that worried about somebody else's Karma, you obviously don't have enough to do. Try to turn the computer off once in a while and maybe take a nice walk in the woods.

Re:Mostly... (0)

Anonymous Coward | more than 9 years ago | (#10976207)

Comments about moderation are offtopic, according to the rules of this site.

What kind of action? (3, Funny)

WillerZ (814133) | more than 9 years ago | (#10976090)

I'd be interested in how well it works after the following:

Coffee spilt in one of the CPU PSUs.
Coffee spilt on the keyboard (if present).
Coffee spilt in one of the disk system PSUs.
Swapping two of the disks in an pack... ...while the system is on. ...while the system is off.

More seriously, it would be handy to know the ratio of workload handled to watts consumed. Workload:cooling required would also be handy.


duh! (-1, Offtopic)

Anonymous Coward | more than 9 years ago | (#10976023)

Does it run linux?

Will it get along well w/ old people in Korea?

Will it do ironing for us in Soviet Russia?

Re:duh! (1, Funny)

Anonymous Coward | more than 9 years ago | (#10976079)

In Soviet Corea, big irons you!

No, no...you gotta mix it up like this... (2, Funny)

Anonymous Coward | more than 9 years ago | (#10976117)

...IN SOVIET KOREA, old people iron YOU!

Re:No, no...you gotta mix it up like this... (0)

Anonymous Coward | more than 9 years ago | (#10976278)

You left out the only.

In Soviet Korea, only old people big iron you.

Fo Shizzle (0)

Anonymous Coward | more than 9 years ago | (#10976024)

I would like to see a review of a high speed, stackable gigabit managed switch.

Real numbers (2, Insightful)

WillerZ (814133) | more than 9 years ago | (#10976025)

Real-world numbers from some inductry-standard benchmarks would be good. You can get TPC-C and SPECint from most vendors, but those are run after weeks of tuning by their internal experts.

I would like to see what they get in a regular user's hands.


Re:Real numbers (1)

mr_z_beeblebrox (591077) | more than 9 years ago | (#10976062)

I would like to see what they get in a regular user's hands.

When you say "regular users" I hope you aren't refering to my regular users, because let me tell you what YOU would get:
user "Phil, I can't open pdfs with this"
you "That's not what it's for, and yes I will help you open your e-mail on your PC....sigh"

Re:Real numbers (1)

WillerZ (814133) | more than 9 years ago | (#10976125)

You are correct. What they get in a regular user's hands is broken.

I meant to ask how they performed in a competent admin's hands. It could also be interesting to derive the marketing fudge-factor -- if you know what you measured and what they claimed, you should be able to say how truthful they are.


Here are a few useful tests: (5, Funny)

Skyshadow (508) | more than 9 years ago | (#10976027)

Useful Tests:

Bossman Compatibility: Verifies that the hardware vendor has taken my boss's boss out to dinner and purchased suitably expensive drinks. Rating based on the number of stars the restaurant recieved, although points may be docked if the filet mignon was a little overdone. This one is related to the...
CYA Verification: Vendor must have a name recognizable to people who read periodicals such as "CTO Magazine" so, when it breaks down, I can say "who ever hear of XVY Company's gear being bad?" If the vendor is a company like Dell which also sells home PCs, this metric should also include going to my boss's boss's house and verifying that his Dell is running okay so I don't have to hear shit like "I don't know why we got Dell, my desktop at home has problems all the time, too, and it's only six years old!"
Sweetness Factor: Not as much of a factor as it once was, depending on how big of iron we're talking about. But it the thing has, say, requires a cooling tower that happens to have a waterfall built into it, that's point right there. May conflict with....
The Under-Desk Operation Profile: Since it'd take at least a month and a dozen SRs and books of useless paperwork just to get the beastie screwed into a rack at our NOC, the server must both fit nicely under the desk in my cube with all the other machines and not be too loud. Generation of excess heat is a plus since the facilities people have set 61 degrees as a reasonable temperature for my office in the winter.
Extra-App Capacity Testing: For when some moron in another department comes in and convinces my boss's boss that "all that server is doing is running the backend for our entire operation, so can we put our incredibly messy half-working app on it too and treat it like QA?" If this server can alert a Terminator unit to go to the aforementioned coworker's home in the middle of the night and slay him and his family, this requirement can be waived (oh, I wait for the day this will be waived....)

I'm sure there are a few other benchmarks you could run, but honestly these are the Big Five that I decide on.

Re:Here are a few useful tests: (1)

WillerZ (814133) | more than 9 years ago | (#10976146)

If it fits under any desk ever created, it ain't big iron.


Re:Here are a few useful tests: (0)

Anonymous Coward | more than 9 years ago | (#10976191)

You've never wielded an axe or a soldiering iron have you?

my 2 cents (1)

grub (11606) | more than 9 years ago | (#10976043)

What do you as an IT professional want to read in a review for a server OS
Security, scalability, robustness. Use your common sense when thinking of a definition of those terms.

or a high-speed switch
Managed how? (ssh, telnet, www?) Any layer 3 routing or filtering capabilities? Packets per second? Backplane speed?

or a big iron server or proprietary workstation?
For our research we're phasing out our SGI machines and going to free OSs (Linux cluster, Free OSs on the desktop) so that's not applicable here.

What tests should we run? What results and feature comparisons are going to be most meaningful to you?
Doom 3 framerate? :)

big iron? (4, Funny)

Frogg (27033) | more than 9 years ago | (#10976046)

when i'm choosing a big iron, i try to find one which can get the big creases out of my big pants

Cash (2, Funny)

CrazyJim1 (809850) | more than 9 years ago | (#10976134)

How many shells does it hold, and is it a revolver or clip. Does it come with a silencer to avoid pesky street light cameras?

Re:big iron? (0)

Anonymous Coward | more than 9 years ago | (#10976181)

> when i'm choosing a big iron, i try to find one which can get the big creases out of my big pants

-1 Sad

IBM (3, Funny)

mdf356 (774923) | more than 9 years ago | (#10976050)

As an IBM employee, I want to see brain-washingly favorable reviews of IBM hardware. Especially the ones that will make me money. :-)


Re:IBM: Model 204 (0)

Anonymous Coward | more than 9 years ago | (#10976232)

What about kick-ass database management systems like Model 204, that run on IBM mainframes or plug-compatibles?

Fool-proof uptime (3, Informative)

Anonymous Coward | more than 9 years ago | (#10976051)

Can the system be expanded without rebooting, can you manage it using computer operators that wouldn't trust to determine which end of a mop should be applied to the floor.

Vendor-Specifics (4, Interesting)

Vengie (533896) | more than 9 years ago | (#10976053)

In many a large setting, a big concern is "does it play nice with XYZ." [Insert cliche about certain-hardware manufacturer that set the "random" retry ethernet window to minimum, rather than minimum+random, to achieve better performance for its cards, intentionally mucking with interframe spacing....] XYZ is going to be: Specific app or other (hardware) product. If the apps are internal (as some of ours are) then you can't help us -- but there are some fairly customizeable out-of-the-box apps that you could test against....

Basically, none of these purchases happen in a vacuum. The merits of the technology matter, but "playing nice" is a dealbreaker. If this causes ANYTHING to break, forget it for now. et cetera.

Load Test (3, Insightful)

Chucklz (695313) | more than 9 years ago | (#10976075)

Can it survive a good /. ing ?

Re:Load Test (0)

Anonymous Coward | more than 9 years ago | (#10976122)

1) Set it up as a webserver.
2) Post a link to a site it is hosting.
3) Record valid data, such as load average, heat, whatever, in the remaining 5-10 minutes the server has to live.

Copy Protection in software? (1)

Jucius Maximus (229128) | more than 9 years ago | (#10976081)

Please tell us if there are any stupidities in installing, running or backing up the software (or software components) related to copy protection. If the company does not respect the paying user, then I have no respect for the company and won't buy their product.

Re:Copy Protection in software? (1)

swordgeek (112599) | more than 9 years ago | (#10976252)

In enterprise software?

Doesn't happen. Aside from asinine license keys tied to the MAC address of a system, I've never seen any stupid copy protection schemes anywhere except at the consumer/small business level.

Who paid for it (4, Insightful)

bstadil (7110) | more than 9 years ago | (#10976095)

First on the list needs to be a clear no nonsense statement how this "review" came about.

Who asked for it and more importantly did anyone pay for it either directly or indirectly.

Big Iron? One word... (0)

Anonymous Coward | more than 9 years ago | (#10976100)


Perhaps I'm jaded... (1)

Telastyn (206146) | more than 9 years ago | (#10976103)

But I already know that "enterprise" software is going to require me to do far too much work to get something not exactly what I need for far far far more than it's worth.

True costs (5, Interesting)

ChiaBen (160517) | more than 9 years ago | (#10976104)

I have had problems in the past looking at various hardware and comparing the true costs of it, especially support. With the third party support companies out there (we use Terix, amongst others), there are so many options, and with yearly support contracts in excess of $100,000, for our relatively small company, mis-calculating these in a recommendation can be a very big deal.

Just my $.02... oh, also just plain reviews of support companies on different hardware would be good also.

Cost Analysis (2, Insightful)

azcoffeehabit (533327) | more than 9 years ago | (#10976106)

A good cost analysis is worth a lot. Say you look at a new and shiney server system, it has the latest OS, servers, and features. But what is that worth?

If the cost of this "new" server is 5X more expensive (as a package) than another system that gives you the same functionality and comparable performance then knowing that this alternative exists and what the performance / price difference is would be valuable.

Satisfaction (-1, Offtopic)

Anonymous Coward | more than 9 years ago | (#10976110)

The only reason to use Big Iron is because you can't get satisfaction any other way (e.g. lack of size, stamina) or maybe because it's not convenient (i.e. man-whores are a nuisance). What I look for in Big Iron is something substantial enough to fill me, and strong enough to keep going. I don't want to down at the store looking for new steel when I could be cleaning the crust of my trusty friend--

wait... this story appears to be about servers... never mind.

Scaling claims & Installation complexity (5, Interesting)

akad0nric0 (398141) | more than 9 years ago | (#10976111)

I've worked with too many companies whose products *do not* scale the way they claim, or whose products will techincally scale, but are at that point virtually useless. Use bogus data, who cares, but test the data volume, throughput, storage, archival, etc. to the limits and make sure the product is still useful. This is the single biggest problem I've had with enterprise installations, and the problem as an architect is that it's difficult to test on a very tight timeline for product evaluation. I've had egg on my face more than once because I had to take the vendor's word for it.

Second, install the application yourself. Don't let the vendor do it for you. And when you install it, install it as an enterprise would. That is, if it's an n-tier application, or has multiple components, don't take the "default" installation and put all of the components on one system. Of course this will work. Try distributing the components over multiple systems like an enterprise would. Often this is where the complexity comes in and products falter.

One company I worked for purchased some software from Tivoli. After 6 months, and a team of engineers onsite from the vendor, they still couldn't get the components to talk for more than a day without problems (after weeks of installation), and still couldn't get useful data out of the database due to its size, so we took our $500mil back and bought something else. Having an evaluation that would've tested this would've saved us a bundle.

Re:Scaling claims & Installation complexity (1)

chinakow (83588) | more than 9 years ago | (#10976320)

This reminds me of another thing that should be checked, I remember from my RAID card days that if you dealt with small files like text documents all the time, and had cache on the controller you could see significant speed increases and at the same time, using cache for something like CAD drawings was just sill because each file would fill the cache and make everything feel slower, so maybe an extended test of large and small file would be inorder also, because what works for a documentation company will probably not be as good for a CAD company.

Just my thought

Comparison against clusters of commodity HW (0)

Anonymous Coward | more than 9 years ago | (#10976115)

The Google article before seems to suggest that racks of commodity hardware scale bettr than big iron.

I'd expect such a review to compare the two.

Environmental Factors (2, Interesting)

SrJsignal (753163) | more than 9 years ago | (#10976118)

As someone who has to build, integrate, then deliver systems to other peoples' server floors I have some things that would be nice to know. How much power does the thing ACTUALLY use, not what the manual says, but real world usage (all you need is a clamp annmeter and a split extension cord) This test helps us determine power requirements if we deliver 100 of these, and cooling requirements.

Make it fail (1)

thegameiam (671961) | more than 9 years ago | (#10976132)

My opinion is that you can only truly understand gear when it's failing: show me what happens when you cram 100Mbps down a T1 interface - do packets drop, or does the router punk out? Show me what happens when, while you're doing that, you hot-swap two of the other cards. What happens when you do a processor-fail-over while under load?

Also, attack the box - look for what's listening, and pound it with every known security vulnerability. Tell me whether passwords are stored as one-way-hashes ONLY, and what are the pwd recovery procedures? Does SSH cost extra?

That's the type of testing I'd like to see.

Thanks for asking,

Frame rate (0)

Anonymous Coward | more than 9 years ago | (#10976133)

Frame rate when playing Doom3?

From a Network Admin perspective... (4, Interesting)

Obiwan Kenobi (32807) | more than 9 years ago | (#10976138)

** For a server OS

How easy is it to install? How easy is it upgrade? How easy is it, if its a different architecture (ie, Windows, Linux, Mac), to migrate big programs (Exchange, databases) from one to another? How well does it gel with existing servers? Do they recognize one another? Do they acknowledge? Can they fit into existing Active Directory-type listings effectively?

Most to all shops are not created overnight. They are built on mistakes or tried-and-true methods that are (usually) quickly outdated. The problems arise when you try to "fix" the existing problems by bringing in more robust OS's and capabilities. It is the meshing of these that is more important to Network Admins that tales of how well this server did on a single machine in a non-network environment.

** High-speed switch

Does it scale (how easy is it add one to five or more on a single chain?)? How is the admin interface? Is it web-based? Console (ie, serial port) based? Does it have both in case console is all that's available? Can you break it or overrun it with traffic?

** Big iron server or proprietary workstation?

Someone else has mentioned scale so let me throw in something different: How easy is it to recover? Does it have Raid? (Well, it should obviously) Break it, remove a disk and see if you can recover from it easily. "Lose" a driver and see how quickly you can recover.

Something I'd love to see is a review that includes a call to the tech support of that server. Don't tell them you're a reviewer, just tell them you got a problem. See how quick they respond, how informative they may be, how far does it have to go before they call in reinforcements? (ie, higher level support)? Will they call on-site repair? If so, how long did you have to troubleshoot before they determined it? Sometimes a card or piece will break and front line support will make you bleed through their ignorant manuals step-by-step when its clear that Piece A is broken and need a on-site tech with experience with that hardware to come and replace it.

** What tests should we run?

Stress, along with installing/upgrading hardware.

** What results and feature comparisons are going to be most meaningful to you?

I believe that over the course of this comment writing and thinking back over my dealings on big iron hardware, that comparisons in regards to tech support, informativeness, and responsiveness are something that can immediatley be added to the review process.

Something more long-term would be how long did the server run before downtime, problems, burnouts, or hardware failures.

please include: (4, Funny)

spectrokid (660550) | more than 9 years ago | (#10976143)

+ doom fps
+ Gentoo compile time
+ Overclocking possibilities
+ Case mods, preferably with blue neon lights

Large SMP systems (2, Informative)

jamesdood (468240) | more than 9 years ago | (#10976144)

Reviews for this sort of equipment are pretty much meaningless. I might buy a 16-way server to run Oracle, you might buy the same system to run large scale data analysis. PCs are easy to review and evaluate, they are commodity; can be used for any of multiple purposes. When I buy a large SMP system, I am buying it for a specific purpose, and the chances are it will never be re-purposed. So before spending uberbucks on a system I want to talk to the vendors other customers who are running similar workloads on the same tin. If the vendor gives me a long list of folks who use their systems for similar applications that is usually a good sign, if they can't then I move on.

Large scale SMP systems require a slightly different mentallity than PC systems, as anyone who has managed a P690 or E10k will attest. You expect performance, you expect reliablity, you expect service, and for what you pay you better get it!

What concerns I would have (0)

Anonymous Coward | more than 9 years ago | (#10976155)

First, how reliable is the hardware? Both in terms of MTTF (Mean Time To Fail) and MTTR (Mean Time To Repair). To what extent can the hardware "self repair"? For example, we have a system which has redundant "hot spares" for the CPUs, memory, and power. If a CPU fails, the hardware, independant of the software, can usually recover the work in progress on that CPU to the "hot spare". The memory subsystem is constantly checking itself. If a memory bank is "weak" or reporting temp errors, the data in that bank is copied to a spare bank and the spare bank is switched as active. Again, without any help from the software. Is the I/O redundant? Again, on this system, there are multiple "paths" to the disk subsystem. If one "path" fails, the OS (not hardware in this case), will redrive the I/O on another "path". The application is not impacted, other than a slightly enlonged I/O time.

How reliable is the OS and other major software components such as the database software? How long can it run between reboots? What causes the reboots that are done?

How secure is the OS and other major software components such as the database? Not only from virus, worms, and "hackers", but from errant applications? If an application "goes wild", can it cause another application or the OS to terminate?

How auditable is the system? Things such as who accessed the system, when, and what did they do? This is likely a combination of things in the OS, the database software and other "OS level" software. It should not depend in any way on the application itself.

How easy is it to debug problems? As an example, you have something that runs at 2 a.m. to produce an overnight report. This something has a problem. How easy is it for a programmer to determine why the application failed? How fast can it be fixed?

Those are a few of my major concerns.

Trust in the vendor is our #1 concern. (2, Insightful)

loony (37622) | more than 9 years ago | (#10976156)

I work at a fortune 5 and my team supports things like E10K, F15K, Regatta, Superdome... The most important part is if you can trust the vendor. If you buy a $5mil frame, you want to make sure that not only will it be supported and the company will be around - but you also have to believe that the vendor will in the future be able to provide adequate patches and updates.

Nothing is more annoying than if you buy a big frame and then you find out that a silly little piece of software is no longer maintained. Or like HP announced today, that they are once again changing their HP-UX roadmap and once again proved that they can't be taken seriously if they predict anything further out than 3 months.

It all comes down to the simple fact that in the end, almost all of the big boxes are the same to the application. Sure, some have hard and some softpartitioning. Sure, you have different cpus, memory latencies and whatever - to the app it is just a bunch of system calls. But in the end, if you can't run your app on it, its useless, no matter how fast, redundant or whatever it is. We have completely moved away from selecting the box by its hardware properties. They are all sufficiently redundant and whatever. We go purely by how well the software we need to run is supported on the OS and if they have a roadmap that can be trusted.


procedure (1)

shrapnull (780217) | more than 9 years ago | (#10976165)

1. Unpackaging : What all do you get with it?

2. Assembly : Standard 19" rack? Power needs? Built-in interfaces? Recommended operating environment?

3. Configuration : Specs, options, adheres to open standards? Why is it more/less expensive?

4. Security/Reliability : What to use for management (ssh, telnet, serial console, proprietary menu system). How secure can you make it? What does it do when maxed out?

5. Maintenance : What parts are user replaceable or which parts can the user order without a certification on the product or shipping it off?

6. Tech support : What plans are available and how much do they cost?

7. Can it do what they say it can and how gracefully does it bow out when overloaded?

8. Whas it built with common sense in mind?

Feel free to add more...

Service. (1)

timster (32400) | more than 9 years ago | (#10976167)

Break it. Call support. See if you can understand what they are saying. See if they can understand what you are saying. See if they can understand what is wrong, or if they lead you through meaningless troubleshooting steps. See how long it takes for someone to show up to fix it. See whether they can actually get it fixed.

The rest of it is not all that important, really.

One possible test... (1)

bigmaddog (184845) | more than 9 years ago | (#10976170)

Do this one last. Pour a gallon of steamy coffee on the appliance while it's operating. If it survives the procedure, give them props in the review. Otherwise, attempt to get technical support for the symptoms. This is not so much to evaluate the potential synergy of electronics and coffe but rather to gauge the support. Mind you, I don't get to deal with a lot of "big iron" things, but the question of support seems to be a very important as well as a very intangible one. All you can go on is testimonials from the web (which, for all you know, are bogus) and recommendations, really, and some preceived reputation of the vendor, but those really only cover the extremes, because the people on the extreme end of (dis)pleasure are the ones who are the most likely to say something. I'm not aware of any systematic way of trying to rate the average user experience. Here's a story: one client has Dells and is thrilled with them, and gets good service all the time, no matter what's goingon. Someone else with Dell allegedly had their service contract terminated (they were refunded the money, but still) beacuse the replacement part they needed was more expensive than the support plan. This is just anecdotal evidence - how am I to reconcile things like this into an overall opinion of their service? Maybe they should also indicate where the support centre is located/outsourced to, so you can be arbitrarily patriotic (or try to impose sanctions) with your purchases.

sorry to be blunt, but... (4, Insightful)

SuperBanana (662181) | more than 9 years ago | (#10976171)

This is going to be harsh, but you need to hear it.

Obviously it is not possible to build an enterprise-grade 'your neck is on the line' production environment just for writing reviews

In order for the review to be accurate, that's how it has to be tested. Evaluating enterprise equipment in a non-enterprise environment with people who have no enterprise experience is pretty much worthless...and you're not going to fool anyone.

There's also no market for this sort of thing. Equipment on that level is bought because of high level executive briefings, price negotiations, migration options, and politics. Why? Because the market is so cutthroat and all the features that matter are there. The decisions are not made on whether or not a power cord was included, it was easy to unpack, the manuals were clear, how well built it looks, and how it did on SysMark SuperServerSimulator 2005...which is about the only thing all you 2-guys-with-a-webserver "hardware review" sites know how to do.

Further- often when a hardware vendor wants to get a contract, they provide a unit for evaluation.

On top of that, the major analyst firms already fill what little niche there is, and they have really big names 90% of the important people with Nice Shoes will recognize, which means even if that analyst is wrong, the decision to go with their recommendation is justifiable and won't get the Nice Shoes person fired. You'd be lucky if .01% recognized your name, much less trusted it. "Jones! Why does our website keep crashing?" "Well, we're having a lot of hardware problems." "Why did we go with ABC for our servers?" "Oh, XYZhardware.com said they were the best." "Jones, clean out your desk."

So...sorry, there's no market for what you're trying to do, and you don't have the means to do it.

Change (1)

COMON$ (806135) | more than 9 years ago | (#10976177)

I want to know that if all goes wrong, I can switch out for new equipment simply. Inevitably I will have to upgrade my hardware and software, I want to know that the next product to come out will be a seamless transition.
Much like other postings, I want to know how well can it handle change?

Commitment from the manufacturer. (1)

metlak (543393) | more than 9 years ago | (#10976180)

If the system is from a lesser-known manufacturer then I would like to know some background information to help evaluate the company and help tell if they will be around for the lifespan of the product. If it is from a well-known company, I would like to know the level of commitment they have for the product line and if there has been talk about phasing the product line out. I know it is impossible to predict this completely but there may be some signals if there is a lack of longevity/commitment.

Does the company support linux on it (1, Insightful)

codepunk (167897) | more than 9 years ago | (#10976184)

I have very specific requirements when it comes
to big dog servers and I just bought two more today.

1. Does the hardware vendor support linux or just pay lip service to it.

2. Can I get it without a os loaded or can I get it
preloaded. If it so much as comes with a oem windows
cd in the box I will ship it back.

3. Have they pissed off the community lately.

I just bought over 40k in servers today and guess who did not see a penny of that? DELL

Reliability (1)

YetAnotherName (168064) | more than 9 years ago | (#10976186)

With the advent of TCP-level load balancing and what-not, the speed is less of an issue so much as keeping the damn thing up in the first place. So, as a result, I like to see, hot-swappable everything. Not just power supplies, not just hard drives, but VME cards, bus cards, and even CPUs in symmetric systems would be a big plus.

Big Iron - Devaluing the Brand (4, Interesting)

iBod (534920) | more than 9 years ago | (#10976197)

When I hear "Big Iron" I think mainframes.

In particular, big IBM mainframes (s/3x0) running something like MVS (maybe VM at a push).

Anyone else think the term "Big Iron" is used innapropriately to describe a bunch of piddling little boxes that don't even need an air-conditioned datacenter equipped with an automatic Halon fire extinguishing system?

Re:Big Iron - Devaluing the Brand (5, Funny)

cakefool (801210) | more than 9 years ago | (#10976285)

If it wouldn't make a good sci-fi set, or look like a CRAY, it ain't big iron. I recently relaxed the requirement that it has tape reels and men with clipboards wandering through it. They can now be women...

RAS (1)

HogGeek (456673) | more than 9 years ago | (#10976217)

All of the vendors that sell big iron use the RAS acronymn: Reliability, Availability, and Servicablility.

So they claim it, but does it work?

Reliabilty: The quality or state of being reliable
Is the system built using good design methodologies, and practices?
Quality components?

Availability: The quality or state of being available
Does the system have many single points of failure?
Are those points truly supseptible?

Servicability: The quality or state of being serviceable
Can I change broken parts with incuring an outage?
Can I add/remove/change without incuring a outage

When we spend hundreds of thousands, to millions of dollars on equipment, it better run, and be fixable without the system having to be stopped, and incuring a outage!

Test those things, prove they etiher work or don't, and the best is what will be bought.

Reviews are insignificant in this setting. (2, Insightful)

l33t-gu3lph1t3 (567059) | more than 9 years ago | (#10976218)

Simple one-day, weekend, or even weeklong reviews are meaningless in the corporate IT environment. Hell, the merits of any particular vendor's gear isn't truly relevant either. I've worked in an institutional IT environment and a corporate one, and this is how purchasing works:

1. Requirements solicitation - figure out what needs we need to fill, be it wifi net access, a file server, etc
2. Vendor research - contact the usual suspects in the field (networking, big iron servers, etc) and arrange for consultation and formal bids to be made. NOTE: this step is skipped ENTIRELY if the company/institution already has a corporate account with a vendor that provides the appropriate services that you require.
3. Formal bidding process - pit the vendors against eachother, it's fun when you get them onsite to demo their gear. Generally vendors will lower prices to sweeten their bid.
4. Award the contract to one of the vendors, or (more likely) have funding denied to you by the beancounters and end up doing a half-assed implementation of what any of the vendors was going to do.

Individual machine or software reviews are a *tiny* part of the process for securing enterprise level hardware/software services.

JMeter, Tomcat and a Standard App (1)

Genady (27988) | more than 9 years ago | (#10976219)

Develop a standard app on Tomcat that you can use to test. Develop a JMeter test set for that app. (I'm thinking shopping card or some other transaction based application) Ramp JMeter from relatively few concurrent users to the infrastructure's breaking point. Figure out why it breaks. Tell us.

The beauty is that JMeter can run a pretty good stress test with a small farm of clients, and Tomcat is the reference application for J2EE. You're not just testing Tomcat though, you're testing the backend database as well and how well the Box/OS deals with Java issues. This sort of test should be fairly portable thus it will work for most hardware/OS combinations.

Big Iron is about scalability and reliability. You can't really test reliability, though you could call support and claim a part is broken and see what support is like. In which case I'd say call three different times with three different theoretical issues and average the response. Scalability though is where it's at as far at the machine/OS is concerned.

Now All that said. I don't doubt that SPEC already has a test that does just that. (Scalability of J2EE Apps) but you might get something from developing your own test.

Reliability and system availability (1)

raider_red (156642) | more than 9 years ago | (#10976223)

Okay, I'm mainly saying this because we've had so many server failures at work in the last month. What features does the machine have to make it fault tolerant? Can they actually be demonstrated as part of the review. Do the automatic failovers actually operate. Will they operate properly if the server crashes under a full load.

Well, (-1, Flamebait)

Minwee (522556) | more than 9 years ago | (#10976225)

It would be nice if it was reviewed by somebody who actually had to work with that kind of hardware and actually had an idea of what was important, but I guess if you really just don't know you could always Ask Slashdot.

Getting reviews that make sense (1)

sander (7831) | more than 9 years ago | (#10976226)

As an IT professional, what I'm really looking for in Big Iron reviews are reviewers who actually have not just a small glimmering of what they are talking about but that they have at least moderate amount of clue. Like say knowing the difference between LPARsand domains. Having actually used large systems would of course be nice but I suspect thats far too high expectations...

yes please tell me what I wanna hear (0)

Anonymous Coward | more than 9 years ago | (#10976228)

Wouldn't the audience you seek just judge for themselves what their requirements are?
Is there consumer brand now to address 'mission critical' so now we have to find a new way to emphasise the 'mission critical' phrase to avoid the wave of 99.999999% claims from everybody selling 10k hardware.

Lots of details matter (3, Funny)

robocord (15497) | more than 9 years ago | (#10976235)

1. How much redundancy is available
a. Are there multiple fans or fan trays?
b. Are there multiple power supplies?
i. How many are needed to power the system?
ii. Can they be powered on and off individually?
b. Are there multiple CPUs?
i. Can they fail independantly, without outage?
ii. Can they be partitioned or dedicated?
c. How about multiple storage controllers?
2. How maintainable is it?
a. Hot-swapability
i. CPUs?
ii. Fans?
iii. Power supplies?
b. Manufacturer longevity
c. Product line stability
d. Off-the-shelf parts?
3. Physical specs
a. It's gotta be rack-mountable, right?
b. How many U high?
c. How deep?
d. Are there pluggy bits on the front, back, both?
e. How much does it weigh?
f. How bloody annoying are the rack rails?
g. Can you open and close it with things mounted directly above and below?
h. Can you swap out any and all parts without unracking?
i. How much heat does it generate?
j. How much power does it require?
k. Is there a maximum rack density specified?
l. Is it loud enough for OSHA to require ear plugs?
4. Expandability
a. How many net ports minimum/maximum?
b. What kind of net ports can it have?
c. How many storage thingies (hard drives, etc)?
d. Is there an upgrade path for the CPU(s)?
5. Servicability?
a. Is there a "lights out" managment board available?
b. Does it require dedicated management software?
c. Does it support SNMP?
i. Standard MIBs?
ii. Custom MIB(s)?
iii. Can it send traps?
d. Are you forced to connect a monitor/keyboard?
e. Is it supported by the obnoxious management/monitoring software of my choice?
F. Miscellaneous
a. Can it run Linux?
b. Does it force me to run Microsoft software?
c. Ok then, what the hell O/S does it run?
d. Can I have the source?
e. Please?
f. There's no SCO crap in there, right?
g. If I fill a whole rack with them, will it impress the chicks?
h. Ok, then how do I impress chicks?
i. What the hell's a chick, anyway?

I'm sure I've left out a ton of stuff, but those are some quick thoughts.

The nr. 1 is support. (0)

Anonymous Coward | more than 9 years ago | (#10976240)

Test support. Tweak an obscure setting beyond reason, load the machine beyond it's capacity, then request the vendor to send one of their support engineers. Insist it's not a top-tier person but the casual support engineer a customer would run into.

Evaluate how many engineers were sent, how long it took them to find the problem, how quickly did they manage to resolve it, and weither the recommendations they presented to prevent the problem from occuring again made sense.


Big Iron reviews (0)

Anonymous Coward | more than 9 years ago | (#10976242)

I prefer the small, transportable kind that fit in a suitcase. The big ones do a much better job on my shirts, though.

The important thing is the Big Iron Boards. The folding kind are the best, but the ones that drop out of the wall are a close second.

It's a good thing I reread the story, because at first I thought it was about golf clubs. How silly that would have been!

Three things (1)

Himring (646324) | more than 9 years ago | (#10976260)

1. Rollout.
2. Administration.
3. Upgradeability.

(All of this, and 1000 more things, are summed up by vendors with this magic word, "scalable"):

I help admin a national network and you're right, it's often hard to know how well a product is going to perform until after the fact. Testing only goes so far, but of all the concerns I've dealt with these are the three I try to stick to.

DDOS it... see how it holds up... (1)

JRHelgeson (576325) | more than 9 years ago | (#10976267)

I learned something about hardware from a simple worm.. I wrote an article on it and I'll re-post it here. Additional comments are at the bottom:

Blaster was a worm, and of worms in general I would say that there is little new to be learned from them. They simply exploit holes that haven't been patched in vulnerable software from Microsoft. The security community continues to lambaste Microsoft regarding their alleged push toward making security their #1 priority, which actually comes in second place - after profits of course.

I did learn something new with blaster though. I have a very good friend that works for a large ISP. They have a number of Cisco 12000 series GSR routers as well as Foundry Big Iron Switches. For those who are not familiar with the Cisco 12000 series routers, let it be sufficed to say that it is Cisco's biggest, baddest router that stands up to 6 feet tall and comes from the factory with a 4 barrel carburetor, dual testosterone modules and a custom paint job with flames painted on the side (pin stripes are optional). These switches are designed to handle hundreds of gigs of traffic across their backplane and through their interfaces. If the ISP were forewarned that they would be seeing 300 mbps of traffic coming from the MS Blaster worm, they would have said "Bring it on!"

For those of us that aren't CCIE's, Cisco routers and Layer 3 switches have a function called CEF, or Cisco Express Forwarding. CEF is a technology that by its simplest definition caches routes.

If a packet from my computer is destined for yahoo.com, it will first hit the DNS server to resolve the host name to its IP address. My computer will then send packets to my ISP with the destination IP of yahoo.com ( My ISP's router, presuming it's a Cisco router with CEF enabled, will look at its internet BGP tables and determine the optimal route my packet should take on the internet to arrive at that destination. Once the router has processed the route, it caches it so that all future packets coming from my home IP address, destined for yahoo.com will automatically be routed using the cached route. This takes a tremendous load off the router CPU as each packet no longer needs to be processed by the CPU, hence the term "Express Forwarding".

What the blaster worm did was send out hundreds of thousands of ICMP pings per second. This usually wouldn't be a problem for the router, except for each packet was destined for a unique IP address. What started happening is that each route was looked up, routed, and stored in its cache for future packets - only there weren't any future packets. What happened next was the memory space allocated for caching CEF routes filled up, and once full, the router simply purged its cache so that every packet had to then go to the CPU to be routed. Once this happened, all hell broke loose.

CPU utilization on the routers jumped to 100%, which should never happen under normal conditions, but this was clearly not a normal condition, and the internet came to a crawl.

As you can see, Worms have an ability to push hardware beyond its designed purpose and now ISP's are wondering what level of testing goes into these devices to see if they can withstand the severe abuse that falls outside of the design parameters.

Hope this helps,

frames per second (0)

Anonymous Coward | more than 9 years ago | (#10976270)

haff life two, duh!

What Do You Look For in a Big Iron Review? (1)

Cougem (734635) | more than 9 years ago | (#10976277)

'What Do You Look For in a Big Iron Review?'

Whether it has a de-scaler and how long it takes to boil.

FMEA (2, Informative)

WeirdKid (260577) | more than 9 years ago | (#10976302)

I always ask for an FMEA - Failure Mode and Effects Analysis - for typical and HA deployments. Big, expensive equipment tends to fail in big, expensive ways, and I want to know all the ways it can fail, all the potential effects of those failures, and what impact they have on my enterprise. Then, I want to know the recommended mechanisms and patterns that can be employed to minimize failure impact.

Support (read: people) (1)

Ruzty (46204) | more than 9 years ago | (#10976313)

When features are the same or similar from vendor to vendor their support organiztion can be a deciding factor. When reviewing big iron break something on purpose and make a call to the support line. The review should definitely include the response from that process.

Testing the support system simulates the "your neck is on the line" environment without much infrastructure cost expenditure. It is definitely very valuable information for those trying to narrow down the field. I know I wouldn't consider buying even the best whirly-gig in the world if I can't get it fixed quickly when it's b0rken.


Reliability, reliability and reliability. (1)

kfg (145172) | more than 9 years ago | (#10976315)

Oh, and if you can add a bit of reliability too, that would be nice.


Proprietary infrastructure == bad (1)

gorodish (788476) | more than 9 years ago | (#10976316)

Some big-iron-type products out there require some kind of proprietary infrastructure to be built around them in order to take advantage of all of their special features. Examples would be requirements of particular web browsers or browser plug-ins, particular network services, etc.

In the case of software, do they require certain files to be in odd, non-configurable locations in a file system? Does the software make good use of the services of the OS (syslog, inetd, /etc/init.d, DNS, LDAP)?

I always look for "lowest common denominator" access, i.e. will I be able to manage this hardware or software without too much pain using a CLI over a low-bandwidth connection.

Also, how fussy will the vendor be with regard to self-maintenance? Do they have intelligent people on their support lines who will listen to my opinions, or are they script-reading drones who just want to point a finger at another vendor and get you off the phone? Will they even let you change the hardware or software configuration without a tech being there?

how easy is it to buy (1)

mo (2873) | more than 9 years ago | (#10976319)

Here's something I'd like to see in a big iron review:
* Are the prices openly availiable
* If not, can I get them via email, phone, fax?
* How many phone calls to a sales guy does it take to get a price list.
* You mean he wants to fly out to discuss pricing?
* How much cheaper is my buddy at SavvyCorp able to buy it for since he knows the right guy to haggle with.

I'm not sure you can.... (1)

nine-times (778537) | more than 9 years ago | (#10976322)

I don't know. It's always kind of good to get whatever information I can, but if I'm looking for any kind of computer equipment, whether desktop or server or anything, my concern is something I can't really imagine being tested in a simulated environment.

What I mean is, when I have home stuff or testing equipment or generally anything that's "for play", my requirements are erratic, and usually any single requirement can be overcome if the product has a certain "coolness" factor. But that's not what you're talking about anyway.

When you're talking about being on the job, my concerns are often not an issue of benchmarking or anything of the sort. I'm not interested in, "I've tried it in some simulated ideal environment, so it should work great in the real world!" The question are always, "but what about a non-simulated environment?" and "What about a non-ideal environment?"

Speed and performance are important, but ultimately I want to know, when you stick this machine in the rest of my network, which is held together with duct-tape and the cat-in-the-jar, and give limited access to users who are going to be using it for real-world purposes, and give limited users who are going to try to do things they shouldn't be doing and whatnot-- in that environment, how does it work? If it remains secure and stable and working, that's what I want to hear. I want it to keep working with as little headache as possible, and I want the administration to be easy. If it's fast, that's gravy.

For that, I always rely on my own real-world experience, the real-world experience of geeks that I trust, and "word around the street" in places like /.

Realise what they are designed for. (1, Interesting)

Anonymous Coward | more than 9 years ago | (#10976329)

For example when you test something like a mainframe enviroment you have to realise that it isn't designed to perform the same tasks that a PC or a workstation or a server is ment to perform.

A Mainframe looks like a dinosaur when you grade it by PC standards, but when you actually see what they do and what they are designed to do you quickly realise that no PC or PC cluster could be made to do the same things at anything close to a reasonable cost.

For example take I/O operations for instance.

You have your standard PC PCI slots that run a 66mhz and are 32bit. That means that the a PC has about 120-130MB/s worth of bandwidth to move information from one device to another. Give or take.

Now when you look at a Mainframe enviroment you notice that it's very distributed by comparision, a modern top of the line Z Series has a theoretical 26TeraBytes worth of I/O operations at it's disposal.

Completely blows anything in the PC or workstation or server world away. There is no way you could create with a PC cluster a cost effective and reliable and backward compatable way of doing what a Mainframe can do and still be in the same price range.

So when testing computers test them for what they were designed to do and the enviroment they were designed to operate in and avoid making meaningless connections in between things like a cluster of PC servers aggragate SPEC CPU score vs a Mainframe's.

Or compare the $ per cpu power of a Itanium proccessor vs a Power970 (mac g5) proccessor. It's mostly pointless and meaningless except for curiosity sake.

Render power and floating point math. (1)

Anubis333 (103791) | more than 9 years ago | (#10976332)

A lot of studios and production houses shop around for render servers. Distributed rendering would be a great benchmark.. There are free renderman compliant renderes that people could benchmark with, not to mention many open source renderers out there.
Load More Comments
Slashdot Account

Need an Account?

Forgot your password?

Don't worry, we never post anything without your permission.

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>
Sign up for Slashdot Newsletters
Create a Slashdot Account