Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

NT faster than Linux in tests

Hemos posted more than 15 years ago | from the ouch dept.

Linux 723

Mike_Miller writes "The lastest Mindcraft Study claims that Microsoft Windows NT Server is 2.5 times faster than Linux as a file server and 3.7 times faster as a web server. Their white paper shows that NT beats Linux on every test. " Anyone have a critique?

cancel ×

723 comments

Sorry! There are no comments related to the filter you selected.

Sponsors (0)

Anonymous Coward | more than 15 years ago | (#1934934)

I notice it takes 10000 requests per child.. In the past, a lower figure was recommended in case of any leaks. I wonder if that could be a factor.

Rebuttal on LWN (0)

Anonymous Coward | more than 15 years ago | (#1934935)

Check out the beginnings of a rebuttal on Linux Weekly News.

insert foot... (0)

Anonymous Coward | more than 15 years ago | (#1934936)

Well, i was under the impression that Linux still needs work in the SMP department... especially with more than 2 processors. NOT THAT I LIKE NT IN ANY WAY!! While I am suspicious of MS sponsoring the test, maby it is proof that Linux needs more work in the high end SMP department.
Coward

Look at the OS configurations (0)

Anonymous Coward | more than 15 years ago | (#1934937)

Linux 2.2.x has the same default window size. The memory limit is hard coded in Linux, unless you apply some patches.

um these arnt fair conditions (0)

Anonymous Coward | more than 15 years ago | (#1934938)

i jsut looked at the graph the fact that they hit the linux box less prove the test wernt fair. a car going 2 miles an hour will always get there after a kid ridign a tricycle at 3 mph

Where was the bottleneck? (0)

Anonymous Coward | more than 15 years ago | (#1934939)

Why no mention of where the bottleneck was on the Linux system? Was all processor time being consumed? Was the kernel snatching excessive cycles? Was the IO channel overloaded? The network saturated?

What exactly was the bottleneck, Mindcraft?

non-linux friendly network cards? (0)

Anonymous Coward | more than 15 years ago | (#1934940)

Maybe I'm completely off-base here, but weren't the network cards (Intel EtherExpress) they used the same ones which had poor Linux support because one company that made one of the chips on the card refused to release the specs?

Possible explanations? (0)

Anonymous Coward | more than 15 years ago | (#1934941)

Looking at what they listed as processes running before the benchmarks, it looks like they have apache being started by inetd. With the added overhead of having to start a new copy of apache for each incoming request, you would expect to see a huge performance drop. I also find it hard to believe that they could have accidently missed this while configuring the Linux box.

Job Security (0)

Anonymous Coward | more than 15 years ago | (#1934942)

Hi.
I'm mcse certified to optimise iis. I couldn't dream of administering a linux system, even if I had the ability to turn a p-pp-pp-pp-age of a book by dr. seuss. You see- my arms were inadvertantly removed by dr. kervorkian when I was a witness to a mercy killing gone awry. I am also blind. My hearing is declining as well.
Would somebody help me please? I want to stomp out any need to ever again have to use telnet. It hurts my nose!

Lower the threshhold on this post (0)

Anonymous Coward | more than 15 years ago | (#1934943)

This post was written just as a flame bait/web site advert. He gives no solid examples to his biased openion. Anyone can make speculations like this. Please send this off to -1.

Hmmm... this makes sense - sort of (0)

Anonymous Coward | more than 15 years ago | (#1934944)

Remember that post a while ago that announced how 2600 posted some "proposed" hacks of what they would do to certain company's sites? Well, on the "hacked" Microsoft site [2600.com] they have a little headline that says:


Windows NT Server 4.0 Outperforms Solaris

Mindcraft, a Microsoft-certified testing lab, recently released a report that shows Windows NT Server 4.0 on a dual Pentium II/450 MHz system with 2 GB of RAM is more than 25 percent faster and offers 2.7 times better price-performance than Solaris x86 on a 486DX2.



Sounds familiar. Idiots...

oh yeah, im sure that wouldnt be biased either. (0)

Anonymous Coward | more than 15 years ago | (#1934945)

"hi, i sell linux computers, and i think linux is good"
"hi, im payed by microsoft and i think NT is good."


"hi, i have a brain and i think you are both full of shit"

No surprise (0)

Anonymous Coward | more than 15 years ago | (#1934946)

> and their latest work (e.g., SQL Server 7.0,
> Office 2000, NT 5.0/Win2000, IIS 5.0) has been
> of exceptional quality

What latest products? Where can I buy Office 2000 or NT5? or Win2000?

Sure, their vaporware has been of exceptional quality

(I see, I'm falling for flame bait)...

Yeah this is flame bait! (0)

Anonymous Coward | more than 15 years ago | (#1934947)

I agree. :)

Yeah this guy is a script kiddie... (0)

Anonymous Coward | more than 15 years ago | (#1934948)

http://slashdot.org/c omments.pl?sid=99/03/23/1318246&cid=1495 [slashdot.org]

proof?

Send this to -1 .. I agree this post doestn deserve +3 .. it's obvious flame bait..

Put this to good work (0)

Anonymous Coward | more than 15 years ago | (#1934949)

Either they're lying or they're telling the truth. If they're lying, replicate the tests and let Red Hat sue the testing company. If they're telling the truth, replicate the tests and figure out which part of the Linux code is too slow. Optimize it, release it. Either way, problem solved.

only 10 start servers? (0)

Anonymous Coward | more than 15 years ago | (#1934950)

No wonder it was so slow. If I was going to run a high traffic site I think I would always keep 100 or so spare servers around.

Yeah this guy is a script kiddie... (0)

Anonymous Coward | more than 15 years ago | (#1934951)

Hey by calling my fellown AC blind, you have proved his point man.

Seriously, How does it feel like to suck bill gates? Does it taste good? Is this for kysh?

net posts mentioned in white paper (0)

Anonymous Coward | more than 15 years ago | (#1934952)

The net posts asking for help that are mentioned in the white paper appear to have been most likely made under the pseudonym:

will@whistlingfish.net

Use DejaNews.

EtherExpress works great! (0)

Anonymous Coward | more than 15 years ago | (#1934953)

I've been using one for several years, as have other people I know. The driver seems to work great...

- RF (dfelker@cnu.edu)

Yeah this guy is a script kiddie... (0)

Anonymous Coward | more than 15 years ago | (#1934954)

You mean kysh from varesearch on irc.debain.org #debian?

He's a cool dude.. um.. person :)

Critique? Sure. (1)

Anonymous Coward | more than 15 years ago | (#1935153)

"A statistician uses statistics in the same way a drunk uses a lamp post - more for support and less for illumination."

Note the hardware (1)

Anonymous Coward | more than 15 years ago | (#1935154)

Note the hardware used in the test: A couple of quad-processor machines with RAID. This sort of configuration is more on the fringe of Linux's abilities, while NT is supposed to work at its peak on a 4-processor system, and I'm sure has much more mature RAID support.

I'll bet if the test were repeated on a couple of single-processor boxes with standard IDE disks, the results would be very different.

I'd walk away from this test with the following conclusion: Linux needs more tuning for higher-end hardware.

Of course, note the spin of the article: If you don't read closely, it looks like NT is 2.5 times faster than Linux in some sort of overall sense.

Targeted benchmarks (1)

Anonymous Coward | more than 15 years ago | (#1935155)

Mindcraft's 'credibility' is blown (if it had any to start with) as any company that seeks to do nothing twist tests to meet the clients satisfaction can not be considered as any better than straight MS marketing/FUD.

I note that the tests & machine vary widely from previous Netware vs NT tests - why?
No mention of relative cost is made which is strange as cost/performance is a rather important factor (how much is NT with 140+ client licenses anyway?)

Why take a machine with 1GB of RAM - is this typical of the average PC server?

MS are simply hoping that media will simple report that 'NT is 3.5 times faster than Linux' as they assume (rightly?) that is all corporates will remember. The only answer is to ignore the Mindcraft study and keep publishing (carefully selected of course) benchmarks showing Linux speed. Thats all that counts in the end.



Erm.. (3)

deicide (195) | more than 15 years ago | (#1935173)

Apache collapsed after 250 threads on a Quad 400Mhz Xeon? Something is definately screwy there..

I have average close to 60-70 Apache threads running as a regular load on Pentium-120 with 64megs of ram without any problems. Most of those are database-generated, rather than plain file GETs. Someone has been either drinking or got paid some dough..

Memory allocation: boot.ini vs lilo.conf (1)

Cardinal (311) | more than 15 years ago | (#1935175)

> Used 1024 MB of RAM (set maxmem=1024 in boot.ini)

Maybe it's just me, but the fact that they went to the trouble of editing the boot.ini but not the lilo.conf is suspicious. Is mem=1024M really that hard? I'm quite certain the feature is documented.

only 970 MB RAM = misconfiguration (2)

William Aoki (392) | more than 15 years ago | (#1935176)

If a Linux kernel can only see 970 MB RAM, it's been misconfigured. To make it see more, there's a source file that has to be edited - but why isn't it an option selectable from 'make config'? SMP is... maybe this'll motivate the change.

Not a flame... (1)

Skyshadow (508) | more than 15 years ago | (#1935181)

...but I find it amazing that you can defend MS with a straight face, especially considering that one of the posts on your /.-lookalike homepage is about how you had to fight with MS tech support to fix your web server...

----

Say what? (3)

Skyshadow (508) | more than 15 years ago | (#1935182)

"The Linux kernal limited itself to 970 megs of RAM"... Say WHAT?

Really; the Winbox had most of its services shut off, while the Linbox was running SMB, NFS, etc. My guess is that they were probably hitting those other services while they were taking the numbers.

Besides, this runs contrary to every other (non-MS paid-for) study I've seen. Mayhaps someone should do some independent verification. Be sure check if the Windows numbers were a "demo".

Hey, they lied to Justice; why wouldn't they lie to us?

----

Hard to believe. (2)

Eric Green (627) | more than 15 years ago | (#1935188)

I agree, it is hard to believe.

And: These people are *LIARS*. They say the posted messages asking for help on the Linux newsgroups. There are *NO* messages from the mindcraft domain anywhere on the Linux newsgroups. So I did DejaNews searches of "performance tuning", "performance tune", "kernel tuning", "kernel 2.2 tuning", between January 1 1999 and today, and examined the results to see if there were any messages that may have been by MindCraft researchers (i.e., that were referring to performance problems with a large-memory machine). There were *NONE*. Zero. Zilch. Which means that if they did ask any performance tuning questions, they did not use those words in the message.

Anyhow: VA Research already loaned a quad-processor Xeon machine to PC-Week and it blew away NT 4.0 in their SAMBA benchmarks. VA Research's quad-processor Xeon machine is the same machine that we sell, and the same machine that Penguin Computing sells (we all get them from Intel, and then dress them slightly differently once we get them, e.g. VA Research uses a Mylex RAID card while we and Penguin use ICP-Vortex RAID cards). So we already have the benchmark that shows that their SAMBA benchmark is full of ****. But that's not going to matter to pointy-haired bosses because they recognize only those reports and studies that say what they want to hear.

Am I steamed? You bet! I *HATE* liars!

-- Eric

Biased? (2)

gavinhall (33) | more than 15 years ago | (#1935205)

Posted by The Chicken of Darkness:

Let's see here... they were using a ZD program to test SMB performance? To a certain degree NT would have an edge, given MS made the SMB protocol and since it is a Ziff-Davis program. The headline doesn't even specify "fileserver". I'm sure Apache would shread NT as a netserver, performance wise and in up-time.

tests by linux newbies (2)

gavinhall (33) | more than 15 years ago | (#1935206)

Posted by LOTHAR, of the Hill People:

I supect that the people conducting the test were not proficient linux users/administrators. The Linux installation followed defalut settings except for Kernal automounting. The fact that

"NFS file system support = yes "

makes me wonder how the drives were partitioned (RAID configuration as well)

The test also mentioned

"The Linux kernel limited itself to use only 960 MB of RAM"

Which is a subject dicussed here last week

The NT installation was not default, the Registry was directly changed.

"Server set to maximize throughput for file sharing"
"Set registry entries: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Servic es:
\NDIS\Parameters\ProcessorAffinityMask=0
Tcpip\Parameters\Tcpwindowsize = 65535"

"Used the affinity tool ..."

I do not consider the test valid.
The NT installation was tuned (if even slightly)
The Linux installation was not.

We need o third party with a though understanding of both OS's to administer an accurate test.

Apache and SMP (1)

EricRCH (728) | more than 15 years ago | (#1935208)

Does anyone know if Apache needs to be recompiled to support SMP or if there are SMP optimizations that can be added to it readily? If MS server had such optimizations and Apache didn't that might cause a significant performance to the Apache/Linux server. Same questions about SAMBA.

Later,
e.

P.S. There is a lot of MS thrashing in this thread. Let's please try to stick to exchanging information and ideas. After that we can say they suck.

They could have been faking things... (1)

adamsc (985) | more than 15 years ago | (#1935214)

It took me all of five minutes to read the cited paper which carefully listed the exact configuration of all hardware and software used, with justification.
While I quite agree with your comments about posters who don't read the article first, there is the possibility that the hardware specs listed were lies. Remember - they tried the same thing on video during the trial. It would not be surprising to learn that an "accident" was made somewhere along the line. ("Gee, we never realized that our rogue packet generator might have messed up the webserver performance for linux. Sorry!")

Well, they can apparently configure NT... (1)

pb (1020) | more than 15 years ago | (#1935217)

I'd like to see them contact someone who knows how to configure a Linux box before saying they ran benchmarks showing something. They claim to have contacted Red Hat and a couple of newsgroups for help, but they don't apparently have that many UNIX gurus on staff...

Maybe if they'd, say, bought a box from VA Research for this purpose, or asked them for help, or contacted the Samba team or the apache team...

Heck, if I had too much money to toss around, I'd be happy to run benchmarks like this, and refute their claims. Linux ran wonderfully on my P133, and NT4 (any revision, later ones are worse) runs horribly on it. I've never seen NT do anything besides eat memory and watch applications crash.

bullsh*t (1)

_damnit_ (1143) | more than 15 years ago | (#1935224)

enough said

Single CPU kernel? (2)

ninjaz (1202) | more than 15 years ago | (#1935226)

Compare and contrast:

NT 4.0 is 2.5 times faster than Linux as a File Server and 3.7 times faster as a Web Server

with ZDNet's findings on the subject of the same benchmarks of Linux vs NT benches.

The same thing could have happened if smp were "Accidentally" left out of the linux machine's kernel.

I doubt that any amount of "tuning" would generate this type of difference. With apache, perhaps realtime ip resolution would do it, but I don't see how that would figure in with Samba.

Quote (1)

Iffy Bonzoolie (1621) | more than 15 years ago | (#1935241)

I always thought that was Mark Twain...

Mark Twain? (1)

Daniel (1678) | more than 15 years ago | (#1935245)

I thought Twain said that. Or maybe they were both quoting a third party?

Daniel

"If you pick up a starving dog and give him food, he will not bite you. This is the principal difference between a dog and a man." -- this is Twain, I'm sure. Fortune sayeth so. :-)

MS's marketing assault on Linux (1)

Andy Tai (1884) | more than 15 years ago | (#1935252)

Linux is getting MS so worried that they sponsor tests to discredit Linux. This is the beginning of a marketing campaign against Linux--Microsoft is going to War [atai.org] .

We work with you to define the goals (2)

EDA Wizard (2225) | more than 15 years ago | (#1935254)

"We work with you to define the goals you want to achieve via testing."

As their main web page states, they define the goals before they test. The only goal was to say NT runs faster than Linux. I've never heard of this company before. I now know why.

Big mistake from Mindcraft (1)

Chainsaw (2302) | more than 15 years ago | (#1935255)

Somebody at Mindcraft must have had a really bad hangover. It seems like they have mistaken Linux for NT and vice versa. Just swap all the values in the page and it should be correct.

Content is BIASED AS HELL. (0)

Svartalf (2997) | more than 15 years ago | (#1935263)

Mindcraft, Inc. conducted the performance tests described in this report between March 10 and March 13, 1999. Microsoft Corporation sponsored the testing reported herein.

The test figures do NOT reflect real-world performance behavior of NT vs. Linux. NT can't cope with the loads they're claiming- we've seen NT boxen just like the Dell supposedly used collapse under the load whereas a Linux box keeps on chugging.

HTTP Error 403 (1)

Paulo (3416) | more than 15 years ago | (#1935270)

HTTP Error 403

403.9 Access Forbidden: Too many users are connected

This is what I got when I tried to connect to your site, a NT box apparently running ASP.
So, is that what you mean when you talk about the quality of M$'s latest products? :-)
Oh, what the hell, let me congratulate you for one of the best trolls I've ever read on Slashdot. Well done. Really.

Erm.. (3)

maugt (3520) | more than 15 years ago | (#1935275)

More than something screwy. You can't use more than 100 threads on IIS 4. It uses Microsoft's Transaction Server thread resource pooling to do thread management, and MTS is internally limited to never handing out more than 100 threads. So basically at least some of it is incorrect.

Hard to believe. (1)

Shane (3950) | more than 15 years ago | (#1935281)

This report is rather hard to believe. Especially since all the tests done by zdnet and pc world shows linux to royally stomp NT in these two catagories.

I for one think varesearch or penguin computing should refute this report on their comparable hardware as SOON as possible as to avoid too many mis-informed people spreading this nonsense around.

I don't think microsoft honestly expects this report to hold up... I think its more of a attempt to cause reasonable doubt in the minds of would be linux users.

just my two cents.

MS Sponsered this (2)

emad (4377) | more than 15 years ago | (#1935288)

If you look at the certification section it is "mindcraft" admits that MS paid them to do this benchmark. Frankly, I can't see how that could possibly make an objective benchmark.

If you look at the other whitepapers that this company has done it is very evident that they are highly biased towards NT.
Just look at the SMP Ultra Sparc machine getting
beat 4x over by some NT PC.

Over at linuxtoday.com one of the Samba team members gives other information

emad

Sponsors (4)

edgy (5399) | more than 15 years ago | (#1935302)

Yeah, this study is sponsored by Microsoft, if you read the fine print:


Mindcraft Certification

Mindcraft, Inc. conducted the performance tests described in this report between March 10 and March 13, 1999. Microsoft Corporation sponsored the testing reported herein.


Looks like you can buy anything you want with enough money. It doesn't make it a true indication of a real-world situation.

I think that there's enough evidence to the contrary already out there, and this will only serve to discredit Mindcraft.

Also, it seems they crippled the Samba Server (4)

edgy (5399) | more than 15 years ago | (#1935303)

According to a posting [linuxtoday.com] on Linux Today by Jeremy Alison of the Samba Team, it seems that the Mindcraft study crippled the Samba server in the tests:


From Andrew Trigell (original Author of Samba):

They set "widelinks = no" now I wonder why they did that :)

In case you haven't guessed, that will lower the performance enormously. It adds 3 chdir() calls and 3 getwd() calls to every filename lookup. That will especially hurt on a SMP system.

Faster --ThAn-- Linux (0)

riddley (6000) | more than 15 years ago | (#1935309)

Time for 6th grade English again?

Results switched? (1)

Wari Wahab (6244) | more than 15 years ago | (#1935314)

Looks like it.. Similar to other reports, but favouring NT instead

Lies, Damn Lies, and Statistics (1)

itp (6424) | more than 15 years ago | (#1935316)

Just to refute one point, I'm running Linux SMP right now, and I've worked in labs where Linux SMP has been used extensively, and as of 2.1.x/2.2.x, SMP support is definitely not weak.

--
Ian Peters

This is all I have to say... (0)

CMiYC (6473) | more than 15 years ago | (#1935317)

dont[root]:~# queso www.mindcraft.com
209.218.193.11:80 * Windoze 95/98/NT


Go figure.



---

There is no conspiracy. (0)

xinit (6477) | more than 15 years ago | (#1935318)

There is no conspiracy. Grow up.

  • This xinit comment sponsered by Microsoft

edgy: you got it. (1)

Signal 11 (7608) | more than 15 years ago | (#1935327)

I researched and found the same thing. I'll be using a higher-grade toilet paper this week after reading the report. I'm certain this is simply the ms press machine at work.

TO ANYONE POSTING IN THIS THREAD:

Please, we know microsoft does these things, let's PLEASE try to keep on topic here. Criques of the article only, not microsoft's *cough* media strategy.




--

Look at the OS configurations (2)

Signal 11 (7608) | more than 15 years ago | (#1935328)

>Linux 2.2.x has the same default window size. The memory limit is hard coded in Linux, unless you apply some patches.

I'd beg to differ here - passing mem=000M via lilo will cause linux to address that memory - provided the system has that much addressible space.



--

Not a flame... (1)

Syberghost (10557) | more than 15 years ago | (#1935352)

Besides, the problems I was having were in NT5 using IIS5 on a P166 w/64 megs of RAM...not the most optimal system configuration for such things.

That's a kick-ass web server configuration for Linux, FreeBSD, OpenBSD, BSD/OS, etc.

It only sucks with NT.

Using Samba, Win95 clients (1)

Syberghost (10557) | more than 15 years ago | (#1935353)

Hell, tests even show that Samba on NT beats regular NT networking.

Using Samba, Win95 clients (1)

Lx (12170) | more than 15 years ago | (#1935374)

Even if this were true, they were using a microsoft Client to talk to a Microsoft server, and using samba instead of something like NFS. It stands to reason that a Microsoft file server doing Microsoft file serving might perform better than a Linux box doing Microsoft file serving. I wonder how the tests would have turned out if all of the clients were Linux boxes, and a non-MS oriented file server setup were used.


-lx

Lies, Damn Lies, and Statistics (1)

redwraith (12532) | more than 15 years ago | (#1935378)

Apparently they were smart enough to de-tune Samba, but not smart enough to tweak the rest of the Linux box. I would suspect that Microsoft said "Hey, Mindcraft, see what it takes to make NT whoop Linux! If it works out, we'll publish it."

But, in all fairness, SMP and RAID are probably pretty weak points for Linux right now, so besides the untweak they did with Samba, they did pick the right part of linux to tear apart.

Of course, what really matters, is when NT 2000 ships, will it be able to compete with Linux then?
It should be interesting to see who can actually tweak their systems more in the year (or two, or three...) to come.

SLASHDOT IT! (0)

Mr.P (12985) | more than 15 years ago | (#1935382)

nt

First impression (0)

jabber (13196) | more than 15 years ago | (#1935389)

I have yet to read the paper, but if it comes from M$, I wouldn't be surprised to see that the computers used for the 'comparison' were different.

Does anyone have specs comparing FS performance between the filesystems supported by Linux and by NT? On the same system?

First impression (1)

jabber (13196) | more than 15 years ago | (#1935390)

Well, mostly to ask the /. bunch if anyone here had personal experience of comparing Linux filesystems and NT. I thought my post said as much.

When I typed it, there were no other comments posted, so it seemed like a fair question. Now, after reading the article in question, I come back to such charming company. Nice.

Having now read it thoughtfully (2)

jabber (13196) | more than 15 years ago | (#1935391)

There are not many things to shoot at.

Linux does not appear to have done well. How does this test translate into a real world situation? Isn't Slashdot running on a lesser machine than the test server? And cranking nicely with perl and Apache doing the dirtywork?

Someone has already mentioned the 960 MB self imposed Linux RAM use limit... Looks like a typo more than anything else.

Pretty graphs that an MBA would appreciate looking at.

The testbed was purely Win95 and Win98 machines running Microsoft TCP/IP - how this translates into 'extend and embrace' is interesting.

The one major anti-Linux thing said was that documentation and support were not forthcoming for the kernel and Apache, but the Samba docs were decent. Is this because Samba is a 'clone' of a Microsoft product?

Just how intimidating is the lack of formal documentation, for an enterprise level web server? After all, the people responsible for handling such an animal would surely have readily available access to the 'routine' expertise, and quirks and oddities are not something even Microsoft documents eagerly.

Ah well.. Back to time off. :)

The cost of such a system (2)

Rayban (13436) | more than 15 years ago | (#1935394)

Well, it says that the NT servers peaked at *112* clients during the SMB test. Looking at the street value of the system, a 20 user pack is about $2000 CDN. 112 users would cost a company *over $10000 CDN* for just the software alone!

This is just plain stupid. :)

Possible explanations? (1)

Saihttam Yrrebnarg (13595) | more than 15 years ago | (#1935399)

linux's SMP support at over two processors is immature in the least, so there is a considerable performance hit. I think that the mindcraft guys put the wrong line in for the amount of memory on the linux machine (append mem = 1000M or something similar instead of 1024M), some of the background programs running under linux were performing tasks that NT doesn't really ever bother with, and the configuration files were set wrong for samba.

On the other hand, the NT machine was very well-tweaked by a team of technicians who are used to dealing with NT-based (biased?) studies. The tcp/ip stack, as well as some features on the network card were also adjusted differently on the two setups. The NT raid support is also considerably more mature. All in all, it wasn't really too fair.

Linux would kill NT on a 16mb p100 or so, because all of the hardware would be well-supported, the SMP-issues wouldn't exist, the raid issue would probably not be there on a p100, and NT does not run very well at all on an older machine. You really need at LEAST 32M to make nt tolerable. More is better. Linux is fine for a number of things on 16.

Apache and SMP (1)

jerodd (13818) | more than 15 years ago | (#1935404)

Apache needs no special configurations options to use SMP, as it uses a seperate process (or pthread, nowadays) for each client. I assume this was static content, which isn't very CPU-intensive, so that shouldn't really be relevant.

I should also add that with $50,000 worth of hardware, no one in their right mind would be buying a Dell for a non-NT machine. Much better go with a real computer like an AS/400. I should also be noted that with that kind of hardware, you would run GNU/Linux on a Sun machine, and a $50,000 Sun machine has a much better I/O interface that would speed up intensive file serving (which is all this is--glorified file serving).

It would have been far more informative to see how well it could handle 100 simultaneous clients using a perl script of the complexity of (for example) the Slashdot engine. *That* would be an interesting comparison of Apache/Linux/GNU and Windows/IIS.

Wonko is /.'ed! (1)

jerodd (13818) | more than 15 years ago | (#1935405)

I keep getting `403 Too many users' errors when trying to go to wonko.com [wonko.com] . Sometimes it also says `403 Can't find server root' or some other message about a messing file. Odd.

I've never seen the /. effect happen from a page with just a casual mention as a tagline before. Interesting.

Hahah (0)

NaTaS777 (14285) | more than 15 years ago | (#1935417)

Saw this on Linuxtoday.com. These guys were paid by MS to do this :) They suck!
FUD FUD FUD!
BUY my cd at http://www.mp3.com/pedophagia

give them a real world test (0)

quarter (14910) | more than 15 years ago | (#1935422)

slashdot the hell out of mindcraft

see how long their server stands

Possible explanations? (1)

Anonymous Shepherd (17338) | more than 15 years ago | (#1935447)

I've seen enough charts and statistics to know that one can almost get anything to look good if the right/wrong questions are asked.

I am ignorant of Linux, running WinNT. Having admitted that, I wonder, barring outright lying, what are some possible explanations for these results?

They use a Quad Xeon 400MHz system. Is there any problems with this? Does Linux support 4way SMP well? Does it support the Xeon well?

Would it, in otherwords, be more fair to test this on a single Xeon 400MHz system? Unless I get responses to the fact that Linux does indeed have robust and reliable 4way SMP... I was under the impression that it was a new thing..

It just seems, unless Linux is 4way SMP capable, that they just ran a test of a 4way system(NT) vs a 1way system(Linux)... Hopefully someone will give me a correct answer.

Another thing that occurs to me is the use of 8HD raid for data storage...

What is Linux's benchmark there?

What if mindcraft had run the tests on a P100 with only 16mb of memory? Who would have succeeded then? My point being(unless Linux really does suck), the choice of the hardware is as important in the high end as the low end.

They could have compared the two OSes across a range of hardware, rather than this specific setup, to see which one is the best solution for a setup, excepting that they were paid by M$ to run this test in the first place.

AS

Not worthy. (1)

cHiphead (17854) | more than 15 years ago | (#1935451)

The benches are not worth of slashdotters eyes. First, they describe PEAK PERFORMANCE in EACH chart. What about sustained/average rates??? Second, why don't we all get together and sue (woohoo, another Class Action Suit) M$/Mindcraft for false advertising/slander against Linux (AFTER we find somebody willing to buy a copy of NT and Linux and test it openly and independently). Third, who the hell is that Wonko42 guy? his site, http://wonko.com, shows just how current he is. An attemp at /.ing that fails miserably (and turns my stomach, too).

I would agree... (1)

Mr. Piccolo (18045) | more than 15 years ago | (#1935454)

Linux is just getting into SMP and probably needs more tuning. Though I'm pretty sure they didn't do their own tuning other than replacing the kernel.

Doesn't somebody make a special Server Edition for this sort of thing though? I though it was RedHat but I can't find it... If there is, try running the test with that and see what happens.

Biased? (1)

MikeTurk (18201) | more than 15 years ago | (#1935457)

they were using a ZD program to test SMB performance?

So? ZDBOp's own tests on Linux (using lowly 2.0.36) showed that it was almost twice fast as NT. In fact, the headline of the article was Linux Up Close: Time To Switch [zdnet.com] . Unfortunately, this version has pulled the damning charts.


Mike
--

Wow, Linux sucks in their universe. (1)

MikeTurk (18201) | more than 15 years ago | (#1935458)

They act as though frequent updates are a Bad Thing. As if hoping and praying that the next NT SP will fix your problem, be released within the next year, and be affordable were better...

grrr...


Mike
--

I didn't see where they counted.... (2)

warpeightbot (19472) | more than 15 years ago | (#1935487)

The time lost every time they had to reboot NT.


'nuff said.

Using Samba, Win95 clients (1)

Hamshrew (20248) | more than 15 years ago | (#1935496)

Actually, the last test I saw(performed by ZD, no less), indicated that Samba serving Win9x clients is about 3 times faster under heavy load than NT.

Raid using NTFS. (1)

helli (24455) | more than 15 years ago | (#1935519)

In what way would it be sane to use NTFS instead of ext2 as the filesystem on the Linux box? It does seem odd that one of the system are allowed to use there native filesystem and not the other, in a I/O intensive test?

No surprise (1)

Zoltar (24850) | more than 15 years ago | (#1935522)

>>>>

I will have to say that I think ie4 is the best browser I have used and I have never had a problem with the MS Office products. When people bash MS it usually isn't for the entire product line, it's for the OS and their business practices.

I don't see how you can state that office2000 and win2000 and ie5 are "of exceptional quality" when they are far too new and untested by the masses. Win2000 doesn't even exist in commercial form, just beta.

only 970 MB RAM = misconfiguration (2)

Zoltar (24850) | more than 15 years ago | (#1935523)

They claim that they contacted RedHat for help with configuring the kernel and RedHat wouldn't help. Makes me wonder... They also said they posted in various news groups and didn't receive any help. This goes against all of my experiences with the the newsgroups. This really smells of FUD... You have to give the boys from Redmond credit, they are very good at promoting their products and FUD.

Using Samba, Win95 clients (1)

BobMarley (25330) | more than 15 years ago | (#1935524)

Actually, linux running SAMBA has repeatedly been tested as being a "better NT server than NT". I recall one study done by someone at DEC about 4 years ago. He tested a SAMBA machine (I don't remember the OS, but I'm fairly certain it wasn't linux) against NT (3.51 was then current, I think), DEC Pathworks, and the AT&T SMB server. The SAMBA machine beat them all hands-down. (Sorry I couldn't come up with a reference, but I read it on the SAMBA-users mailing list a few years ago.)

.c.

Look at the OS configurations (3)

blach (25515) | more than 15 years ago | (#1935527)

Look at the OS Configurations:

For one, NT used 1Gb of ram will Linux used only 960MB. Surely they could have passed the parameter MEM=1024M to the kernel ...

Additionally they tuned tcpwindowsize under NT to 65536, and adjusted buffers on the network card to 200 (from 32).

They made no TCP/IP stack adjustments OR adjustments to the netcards under linux.

Just look at the sections explaining the myriad of things they did to "tune" NT. Then look at linux. Enable NFS. The following daemons were run. blah blah. Didn't bother to work on anything.

James

First impression (1)

davidbak (27436) | more than 15 years ago | (#1935535)

Your comment is essentially: "I haven't bothered to click on the link to read the paper but I wouldn't be surprised to see that Microsoft's cheating." It took me all of five minutes to read the cited paper which carefully listed the exact configuration of all hardware and software used, with justification. You couldn't spend the five minutes to do that but went ahead and spewed your useless comment. Why? -- Dave

Wow, Linux sucks in their universe. (0)

CrosseyedPainless (27978) | more than 15 years ago | (#1935537)

I'm glad to be in this universe, instead! For one thing, I've never put my "system in a state where you must reload all software from scratch since you need to recompile and reinstall the kernel." Funny, I can recompile and reinstall kernels until my fingers bleed, yet never need to reload all software from scratch. And did you see, that over a two month period, there were six kernel updates?!!? There you have it! I guess that's bad. But, even idiocies like these are just fine, because "MINDCRAFT, INC. SHALL NOT BE LIABLE FOR ERRORS OR OMISSIONS CONTAINED HEREIN, NOR FOR INCIDENTAL OR CONSEQUENTIAL DAMAGES RESULTING FROM THE FURNISHING, PERFORMANCE, OR USE OF THIS MATERIAL." Hey, that's cool. I wish I could not be liable for errors or omissions in my work. But then, I don't work for Microsoft, like they do....

No surprise (1)

CrosseyedPainless (27978) | more than 15 years ago | (#1935538)

I would gladly flame you, but you seem to base your glowing reports of quality on unreleased software. Is any of that software you mentioned out yet? SQL Server 7.0, maybe? I know the rest isn't. Anyhow, the released versions of those packages blow, and had you claimed exceptional quality for them, you would have earned every flame you got. Although, you were astute enough to say "(at least, from my point of view)." Have you ever used Office 97, NT 4.0/Win95, or IIS 4.0? Doesn't sound like it. Thanks for playing. Have a nice day.

No surprise (0)

Wonko42 (29194) | more than 15 years ago | (#1935552)

I think people tend to make far too many biased statements when arguing either for or against Microsoft products. Linux supporters invariably claim that Linux is faster and more stable, while Microsoft devotees tend to rely on the easier setup and configuration of Microsoft's operating systems and software.

This test just goes to show that every operating system has good parts and bad parts. While Microsoft may be the current lead in speed and usability, Linux is still a little bit ahead in stability (notice I said "a little bit"). I think the main reasons for this are resources and one of the fundamental rules of capitalism.

Microsoft has far more resources than Linux hackers could ever hope to have. While they may not have nearly as many developers working on the product, they do have a relatively closely-organized team all communicating within the group and heading for specific goals. They have entire company departments devoted to testing, user interface design, code beautification, etc. Linux, on the other hand, doesn't have quite as organized a structure. This isn't to say that the development of Linux is worse or less efficient, it's just disadvantaged in some areas, by no fault of its own.

As for that major capitalistic principle, it gives Microsoft both an advantage and a disadvantage. Microsoft's goal is to make money. In order to attain this goal, they must have a superior product in order to get people to buy it (let's leave the monopoly crud out of this for a moment). Therefore, Microsoft strives to make the best, fastest, and most usable product out there. And when they fail, they generally try to fix the problem as soon as possible via service packs, hot fixes, etc. On the other hand, this is also a disadvantage because rather than implementing features that are needed by a few customers, they implement only features that are needed by a majority of customers. This is where Linux has an advantage, obviously, due to its open-source model.

I won't get into it much further than this, but I think we should all at least give Microsoft the credit they deserve. For the most part, they do try to have a good quality product, and their latest work (e.g., SQL Server 7.0, Office 2000, NT 5.0/Win2000, IIS 5.0) has been of exceptional quality (at least, from my point of view). Why not view this as a challenge? I'd be willing to bet that Linux could catch up with and even surpass NT's benchmarks within a year or less.

And yes, I expect flames galore for saying good stuff about Microsoft...

--
Wonko the Sane

Not a flame... (1)

Wonko42 (29194) | more than 15 years ago | (#1935553)

Er, no...I never had to fight MS tech support. I've never even called MS tech support. Never even looked up their number. And my webserver is a different matter entirely...it sucks because I don't know what I'm doing, not because of any fault of Microsoft. Besides, the problems I was having were in NT5 using IIS5 on a P166 w/64 megs of RAM...not the most optimal system configuration for such things. Thanks for visiting the page, though. :-)

And it should be pointed out that my intent was not to defend Microsoft, but rather to try and make people give them credit where they deserve it. I can honestly say that I would prefer to use Microsoft products under many circumstances, mainly because they're easy to use and stable enough for my purposes. Linux on the other hand, is almost impossible for me to use (I'm only a beginner when it comes to Linux, while I've had years of MS experience). I just think Microsoft has a pretty bad and not entirely fair stereotype going against them...

--
Wonko the Sane

No surprise (1)

Wonko42 (29194) | more than 15 years ago | (#1935554)

I have used all those products, actually, and for the most part, they were okay. Not the best by any means, but okay. I don't particularly like Office 97, although there's no better alternative for Linux. IIS 4.0 with NT 4.0 isn't all that bad...it performs much faster than Apache in some areas (CGI/ASP is one), while in other areas it's severely lacking, but it's good enough for me. NT 4 is a nice operating system if configured properly, although quite certainly not the best. Win95 just plain sucks, as does 98...

--
Wonko the Sane

HTTP Error 403 (1)

Wonko42 (29194) | more than 15 years ago | (#1935555)

You're getting the error because my site is hosted on a Win98 box running Personal Web Server...which has a maximum connection limit of 10 users at a time. My site doesn't generally get Slashdotted.

--
Wonko the Sane

Yeah this guy is a script kiddie... (1)

Wonko42 (29194) | more than 15 years ago | (#1935556)

That post was extremely tongue-in-cheek, my blind friend. It was meant to be a jab at the possible crackability of the new printer protocol, not as an actual statement that I'd be looking forward to cracking peoples' printers.

--
Wonko the Sane

No surprise you're a brainwashed sheep (1)

Madhatter (33678) | more than 15 years ago | (#1935602)

Microsoft the credit they deserve? You go to www.netcraft.com and see how many servers out there actually run Microsoft IIS. You will find far many more running Netscape/Enterprise and Apache. Microsoft's own famed Hotmail runs on unix! When they tried to move it from Unix to NT it CRASHED HORRIBLY.

NT can't handle the big loads. On smaller loads it is known to show more performance, but you start pounding that sucker and it goes to hell. Unix and Linux keep on chuggin.

Did you not see the emphasis that this report put on the lack of support for Linux? Did you see how they fine tuned the nics and the CPUs under NT and not on Linux? Microsoft is trying to cause FUD in the minds of everyone.

The only respect they deserve is that they have one hell of a marketing scheme. Oh yeah, Bill Gates is good at legally pirating software and calling it his own.

But you bring up a good point on OSs (1)

Madhatter (33678) | more than 15 years ago | (#1935603)

Sorry for the flaming earlier. I'm ready to go home and a garbage NT machine is keepin me at work. But hey, the PC is only as good as the operator.

Linux has the real world proof anyway... (1)

Madhatter (33678) | more than 15 years ago | (#1935604)

Just look at where Linux and Apache have made it to in the real world. The tests they conducted were so far fetched it is really hard to believe. There are alot more big companies out there that use apache and Netscape/Enterprise. IIS just can't handle the traffic that they can. Microsoft's Hotmail dosen't even use IIS because it handle the load of nearly 1 million users.

The only thing Microsoft has for them is that they are easy to install. That is what they are trying to capitalize on here, and it looks like its going to work very well even with what the linux news web sites will have to say. FUD all the way.

ZD Bench tests (1)

eof (33820) | more than 15 years ago | (#1935607)

Unfortunately, tests like WebBench and NetBench hava a _lot_ of subjectivity built in. For example, the WebBench homepage WebBench [zdnet.com] states "WebBench also has dynamic test suites where we've re-written the CGI application as:"..."a Windows NT Microsoft Internet Server API dynamic link library". With that in mind, MindCraft could have easily run the native API dynamic tests on NT, versus the "regular" tests on Linux, greatly biasing the results. Yet their "white paper" never becomes specific on exactly which tests were performed, only the overall results. Hardly conclusive.

--
"Modern cryptography is nothing more than a mathematical framework
for debating the implications of various paranoid delusions".

NT beats up Linux (1)

fredm8 (33973) | more than 15 years ago | (#1935612)

Read the report. There seems to be an issue that the people whom did the benchmark didnt/dont know how to tune Linux, Samba & Apache. However they did know enough to disable NT services and edit the Registry (which MS states is at your own risk), and dedicate a CPU per NIC. Multi-threaded ? Multi-tasking ? O/S.
There are enough comments from knowledgeable people posted to this article to indicate that the knowledge to tune Linux to outperform NT does exist.
Here is how to fix the problem. Offer to support Mindcraft to tune Samba correctly, recompile the kernel to use 1GB of RAM and fix the Apache configuration. Then challenge Mindcraft to post the result of the re-run benchmark. If Mindcraft wish to be seen as accurate and un-biased they will jump at the chance to re-run the benchmark with a correctly tuned Linux server.
Otherwise simply sounding off in a forum like this only serves MS's purposes further, by allowing them to get your focus on irrelevant activities, not saving the world.
P.S. I too expect Linux, Samba, Apache properly tuned will outperform NT.

They shouldn't use that for enterprise anyways... (1)

InferiorFloater (34347) | more than 15 years ago | (#1935618)

Why not get a real server, rather than some intel garbage? Like a Sparc UE or something?

Even if it is a quad xeon 400, saying that NT posesses 'superior scalability' is like saying is has 'mature engineering' rather than code bloat...

qoute (1)

rainman81 (34872) | more than 15 years ago | (#1935619)

I'll qoute good ol' Winston Churchill on this one.
"There are three kinds of lies. Lies, damned lies, and statistics."

Sponsors (1)

diehard (75848) | more than 15 years ago | (#1935645)

Insane.

Response at lwn.net (4)

Timbo (75953) | more than 15 years ago | (#1935646)

Please submit any inconsistancies you see in this document (and if you don't see any, please shoot yourself in the head) to lwn@lwn.net. They are readying a response as we speak.

No surprise (0)

Fnkmaster (89084) | more than 15 years ago | (#1935648)

Don't know why this is moderated so high. Flame bait or no, the fact that you blindly accept the false claims of this oh-so-unbiased "benchmark" should preclude this from a 3 rating.
Load More Comments
Slashdot Login

Need an Account?

Forgot your password?

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>