Beta

Slashdot: News for Nerds

×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Performance Benchmarks of Nine Languages

michael posted more than 10 years ago | from the tmtowtdi dept.

Programming 954

ikewillis writes "OSnews compares the relative performance of nine languages and variants on Windows: Java 1.3.1, Java 1.4.2, C compiled with gcc 3.3.1, Python 2.3.2, Python compiled with Psyco 1.1.1, Visual Basic, Visual C#, Visual C++, and Visual J#. His conclusion was that Visual C++ was the winner, but in most of the benchmarks Java 1.4 performed on par with native code, even surpassing gcc 3.3.1's performance. I conducted my own tests pitting Java 1.4 against gcc 3.3 and icc 8.0 using his benchmark code, and found Java to perform significantly worse than C on Linux/Athlon."

cancel ×

954 comments

testing (-1, Offtopic)

Anonymous Coward | more than 10 years ago | (#7928157)

this is a test?

Trig functions... (3, Interesting)

eples (239989) | more than 10 years ago | (#7928169)

I am not a compiler nerd (IANACN?), so maybe someone else can answer the following simple question:

Why are the Microsoft languages so fast with the Trig functions?

Re:Trig functions... (4, Funny)

Kingpin (40003) | more than 10 years ago | (#7928216)


They probably cheat and use undocumented native OS calls.

Re:Trig functions... (-1, Troll)

Anonymous Coward | more than 10 years ago | (#7928297)

You probably cheat when you call youre own private or internal visible methods.

I bet you cheat too when you code youre own test apps too. its theyre platform they can do what they want. its as propriety as the mac, you code what they let you code. If you dont like it, bugger aff.

Troll.

Re:Trig functions... (3, Insightful)

bigjocker (113512) | more than 10 years ago | (#7928221)

What is interesting in these functions is that, as pointed in the article, there seems to be something wrong with Sun's implementation for Java. Removing this test JDK 1.4.2 executes almost on par with Visual C++ (the winner).

This is (once again) proof that Java is not slow, in fact it's really fast. It's slow starting, and yes, consumes more memory than native code, but the gained security, stability and ease of programming (reduced development times) are worth the memory use increase.

Also, the memory use should be addressed by project Barcelona (I believe these will be available in the forthcoming J2SDK 1.5, along with generics, enums, etc).

Re:Trig functions... (4, Insightful)

Dr. Evil (3501) | more than 10 years ago | (#7928284)

Don't forget that it is also percieved as slow since just about any application anyone has seen for a desktop environment written in Java has a sluggish GUI.

Yeah, I know Java's strengths aren't in the Desktop arena, they're in development and the back-end.

Re:Trig functions... (5, Informative)

csnydermvpsoft (596111) | more than 10 years ago | (#7928454)

If more people would use the SWT libraries (part of the Eclipse project) instead of the crappy AWT/Swing libraries, then this misconception would go away. SWT works by mapping everything to native OS widgets if possible, giving it the look, feel, and speed of a native app. I used Eclipse for quite a while before finding out that it is almost 100% pure Java (other than the JNI code necessary for the native calls).

Re:Trig functions... (4, Informative)

tealwarrior (534667) | more than 10 years ago | (#7928309)

What is interesting in these functions is that, as pointed in the article, there seems to be something wrong with Sun's implementation for Java.

For many math functions java uses a software implementation rather than using the built in hardware functions on the processer. This is to ensure that these function perform exactly the same on different architectures. This probably accounts for the difference in performance.

Re:Trig functions... (2, Informative)

be-fan (61476) | more than 10 years ago | (#7928407)

Its not proof of anything. Its just proof that the JIT manages to handle JIT-ing properly. All of these were simple loops where the JIT could do its work and get out of the way for the rest of the program. They are not indicative of real-world performance, unless all you are doing are such loops.

Re:Trig functions... (2, Informative)

techiemac (118313) | more than 10 years ago | (#7928410)

One issue with Java is it all depends on the Virtual Machine.
I, for one, would _never_ trust Java in a mission critical embedded environment. In fact you still see assembly in those envrionments from time to time. Imagine using Java for a fly by wire system. Would you fly on a plane that was using Java for fly by wire? I, for one, would not.
Java is great for some things. But you get too many cases where companies use a new technology without adequate due diligence simply because its the NTOW (New Technology Of the Week). I still say that a server written in C (written properly of course) will outperform a server written in Java.

Re:Trig functions... (0)

Anonymous Coward | more than 10 years ago | (#7928261)

You mean, why are the others so slow with trig?

MS faster? They must be cheating. It must be pre-loaded like IE is. They must be radioing home and telling Bill Gates about all the porno you download and every math calculation you do.

Re:Trig functions... (2, Interesting)

hackstraw (262471) | more than 10 years ago | (#7928322)

M$ is pretty tight with Intel (hence the term Wintel). They might have licenced or somehow gotten some code optimisers from Intel. On Linux, the Intel compiler is often 100% faster than gcc on double precision code (like trig functions).

Re:Trig functions... (1)

Ianoo (711633) | more than 10 years ago | (#7928337)

That's a good question, and I'd like to find out the reason. Is it possible that they're using different methods to compute the functions? There are several ways you can compute trig values, such as using taylor series approximations (e.g. sin(x) = x + x^3/3! + x^5/5! + x^7/7!...) or complex exponentials (sin(x) = (e^(jx) - e^(-jx))/(2j) ). I don't know which method is generally accepted as best for computer calculations. Maybe Java computes the approximations in Java code while in the CLR the function that actually computes sin(x) is unmanaged assembly code. I don't know.

Re:Trig functions... (2, Interesting)

coats (1068) | more than 10 years ago | (#7928471)

The Intel x87 uses Pade representations in hardware, accurate to IEEE REAL*10 EXTENDED, i.e., about 24 significant digits.

Otoh, the P4 SSE2 uses a vectorized software model that doesn't have these; I don't know whether the MS compiler generates x87 hardware code or SSE2 vectorized software.

Java specifies using a 32-bit model for these functions, and is probably doing them in software. But what software? And does it use the vectorized SSE2?

Re:Trig functions... (5, Interesting)

mengel (13619) | more than 10 years ago | (#7928433)

Probably the Microsoft languages use the Intel trig instructions.

In the case of Java, you find that the Intel floating point trig instructions don't meet [naturalbridge.com] the Java machine spec. So they had to implement them as a function.

It all depends if you want accuracy or speed.

2nd post!! (-1, Offtopic)

Anonymous Coward | more than 10 years ago | (#7928174)

Allow me to be the first to say... BFD [urbandictionary.com] !

Accurate? (5, Interesting)

Nadsat (652200) | more than 10 years ago | (#7928178)

Not sure of the accuracy. Benchmark is on a loop:

32-bit integer math: using a 32-bit integer loop counter and 32-bit integer operands, alternate among the four arithmetic functions while working through a loop from one to one billion. That is, calculate the following (while discarding any remainders)....

It also relies on the strength of the compiler, not just the strength of the language.

Re:Accurate? (1)

Catskul (323619) | more than 10 years ago | (#7928231)

I think its pretty hard to separate the difference between the strength of the compiler and the strenght of the language. You could try with many compilers on the same language, but that still doesnt factor out maturity since the newer languages' compliers havent had the time to mature.

Re:Accurate? (0)

Anonymous Coward | more than 10 years ago | (#7928358)

It also relies on the strength of the compiler, not just the strength of the language.

Of course, when you move out from your perl-world you'll realize that most languages are just specs. Different implementations yield different results.

Why did VB do so bad on IO. (4, Interesting)

nberardi (199555) | more than 10 years ago | (#7928180)

Why did VB do so bad on IO compared to the other .Net benchmarks? They were pretty much equal up until the IO benchmarks? Any chance of getting the code published that was used to test this?

Re:Why did VB do so bad on IO. (-1, Troll)

Anonymous Coward | more than 10 years ago | (#7928301)

I am what most people would consider a highly trained technical professional. Unlike most people who spout off at this site, I have the certificates to prove this, and furthermore they're issued by the biggest software company in existence.

I know how to tell facts from marketing fluff. Now, here are the facts as they're found by SEVERAL INDEPENDENT RESEARCH INSTITUTES:

Expenses for file-server workloads under Windows, compared to LinuxOS:
  • Staffing expenses were 33.5% better.
  • Training costs were 32.3% better.


They compared Microsofts IIS to the Linux 7.0 webserver. For Windows, the cost was only:
  • $40.25 per megabit of throughput per second.
  • $1.79 per peak request per second.


Application development and support costs for Windows compared to an opensores solution like J2EE:
  • 28.2% less for large enterprises.
  • 25.0% less for medium organizations.


A full Windows installation, compared to installing Linux, on an Enterprise Server boxen:
  • Is nearly three hours faster.
  • Requires 77% fewer steps.


Compared to the best known opensores webserver "Red Hat", Microsoft IIS:
  • Has 276% better peak performance for static transactions.
  • Has 63% better peak performance for dynamic content.


These are hard numbers and 100% FACTS! There are several more where these came from.

Who do you think we professionals trust more?
Reliable companies with tried and tested products, or that bedroom coder Thorwalds who publicly admits that he is in fact A HACKER???

--
Copyright (c) 2004 Mike Bouma, MCSE, MCDST, MS Office Specialist

Permission is granted to copy, distribute and/or modify this document
under the terms of the GNU Free Documentation License, Version 1.2
or any later version published by the Free Software Foundation;
with no Invariant Sections, no Front-Cover Texts, and no Back-Cover
Texts. A copy of the license is included in the section entitled "GNU
Free Documentation License".

heres the link to the code (2, Informative)

manifest37 (632701) | more than 10 years ago | (#7928352)

code [cowell-shah.com]

Re:Why did VB do so bad on IO. (5, Informative)

enkafan (604078) | more than 10 years ago | (#7928411)

Because the guy who wrote the code decided to use the VB6 compatability features instead of the .NET runtime for VB. Why one would do this, I have no idea.

Re:Why did VB do so bad on IO. (1)

subjectstorm (708637) | more than 10 years ago | (#7928443)

"Christopher W. Cowell-Shah's benchmarking code, available at:"

http://www.cowell-shah.com/research/benchmark/code [cowell-shah.com]

That's from Bacule's site (fails.org) referred to in the article synopsis. The vb benches are linked at the bottom if you want to check em out. Hope this helps.

Wow (1)

Catskul (323619) | more than 10 years ago | (#7928186)

Wow.... according to this gcc C sucks : (

Re:Wow (3, Interesting)

finkployd (12902) | more than 10 years ago | (#7928207)

Well, for performance it does. For cross platform compilation it rocks the house. If you really want performance you need to be using something like Intel's C compiler (which oddly was not tested)

Finkployd

Re:Wow (-1, Flamebait)

Anonymous Coward | more than 10 years ago | (#7928296)

Thank you, Captain Obvious. Your work here is done.

optimizations ? (0)

Anonymous Coward | more than 10 years ago | (#7928396)

gcc -march=pentium4 -msse2 -mfpmath=sse -O3 -s -mno-cygwin

no loop unrolling ???
no inline functions ???

In fairness, gcc is not the world's greatest
optimizing compiler. It is the world's
most portable compiler. That code he
compiled for cygwin can uploaded to a
15 year old VAX and compiled and run ...
or to a beowulf cluster.

Try that with VB.

Re:optimizations ? (0)

Anonymous Coward | more than 10 years ago | (#7928464)

He missed a lot of optimizations.

I was able to get pretty close to his numbers on a 1GHz system.

Re:Wow (3, Insightful)

thoolihan (611712) | more than 10 years ago | (#7928462)

Keep in mind too that these benchmarks were all run on windows. I think gcc plays a lot nicer with glibc compared to the windows native libraries. Also, as pointed out, it's about being portable, not the most optimized compiler.

-t

Re:Wow (2, Informative)

Daniel Boisvert (143499) | more than 10 years ago | (#7928468)

I found it interesting to note in the benchmark design that the Visual C++ compile used the "omit frame pointer" option, while gcc did not. It seems to be the consensus over at the Gentoo Forums that this flag makes a fairly noticable difference (if negatively impacting debug options), and I'd like to see the C piece re-run using this option. It's tough enough to compare apples to apples in tests such as these, but at least try to use the same compile flags where available..

Dan

Michael you worthless tub of shit (-1, Offtopic)

Anonymous Coward | more than 10 years ago | (#7928188)

You are a waste of my time


I LOVE U

Thanks

java vs C (-1, Troll)

Tirel (692085) | more than 10 years ago | (#7928189)

Java isn't and never will be as fast as C/C++, people who say "but oh it's compiled it runs just as fast" haven't a clue what they're talking about.

I suggest reading some kernel documentation or tesitng real-world applications and then you'll notice what I'm saying is true.

Re:java vs C (1)

j-b0y (449975) | more than 10 years ago | (#7928478)

Which is true, but missing the point somewhat. Given an half decent optimizing compiler, C/C++ should almost always be faster than Java, but not so much faster that the speed becomes a deciding factor over the advantages that Java has compared to C/C++.

Now, there are always a certain class of programmer who must always have the absolute fastest performance possible. For them, C/C++ is the weapon of choice, if they don't have the time to hand-craft assembler.

For everyone else, it becomes a decision based on personal perference, technical imperative, management decision or dogma.

Java Performing worse then C (4, Insightful)

ViolentGreen (704134) | more than 10 years ago | (#7928210)

I conducted my own tests pitting Java 1.4 against gcc 3.3 and icc 8.0 using his benchmark code, and found Java to perform significantly worse than C on Linux/Athlon.

Why is this a suprise? C has been most commonly used for so long because of it's speed and efficiency. I think anyone who has done much work with either developing or running large scale java programs knows that speed can definitely be an issue.

Re:Java Performing worse then C (4, Insightful)

Kingpin (40003) | more than 10 years ago | (#7928299)


All that matters to anti-Java zealots is speed. The list of benefits coming from using Java is too long to take the speed-only view seriously.

Re:Java Performing worse then C (3, Insightful)

finkployd (12902) | more than 10 years ago | (#7928387)

Not always though, I think the thing people neglect to consider is that there are times when performance and scale are important enough that the benefits of Java do NOT outweigh C, and vice versa.

I feel sad for someone who only has enough room in their world for one computer language.

Finkployd

Was the JVM tuned? (1)

FerretFrottage (714136) | more than 10 years ago | (#7928306)

No question that C is faster than Java, but with a properly tuned JVM, I can get performance pretty close. The biggest penalty I saw was in startup. Besides the standard command line args, if you don't know what jvm.cfg check it out and learn how to use it.

Re:Java Performing worse then C (2, Interesting)

csnydermvpsoft (596111) | more than 10 years ago | (#7928321)

I think anyone who has done much work with either developing or running large scale java programs knows that speed can definitely be an issue.

I would consider myself part of that "anyone," and I disagree with you. Other than load times (which aren't as bad as they used to be), Java can perform as fast or faster than C code. The main thing is to use a good VM - IBM's J9 VM significantly outperforms Sun's.

Re:Java Performing worse then C (1)

mindriot (96208) | more than 10 years ago | (#7928351)

Actually, it's more of a surprise that Java performed /equally well/ in all benchmarks -- except the trig benchmark. The dismal performance in the trig benchmark is the second, much bigger surprise. It would be good to see the source code used to find out whether the code is BS or Java actually totally fails to produce efficient trig code.

Under Windows... (3, Insightful)

Ianoo (711633) | more than 10 years ago | (#7928211)

I see once again that Eugenia (a supposed pro-Linux pro-BeOS person who doesn't use Windows) has done all her benchmarks [i]under[/i] Windows. I have a feeling that Python would perform a lot better if it was running in a proper POSIX environment (linked against Linux's libraries instead of the Cygwin libs). Probably the C code compiled with GCC would perform a fair bit better too.

Re:Under Windows... (0)

Anonymous Coward | more than 10 years ago | (#7928271)

Eugenia didn't write this article. Of course you would have known had you RTFA or spent more than five seconds glancing at it before firing off a knee jerk troll.

Re:Under Windows... (0)

Anonymous Coward | more than 10 years ago | (#7928285)

I see once again that Eugenia
I see once again that Eugenia is using her usual pseudonym of "Christopher W. Cowell-Shah".

Re:Under Windows... (0)

Anonymous Coward | more than 10 years ago | (#7928305)

I once again see a person who comments without reading the article. The benchmark (however crappy it may be) was not performed by Eugenia.

Re:Under Windows... (1)

Ianoo (711633) | more than 10 years ago | (#7928375)

Now that I have RTFA, I see that Eugenia herself didn't write it (I don't bother to look any more since OSNews seems to be a one-woman show most of the time), but my point about POSIX and the results under Windows stand. As an aside, I wonder how Mono would perform?

Re:Under Windows... (1)

be-fan (61476) | more than 10 years ago | (#7928379)

Eugenia didn't do the benchmarks, and she uses XP Pro mainly. But these where low-level numeric benchmarks. Except for the I/O one, they wouldn't have changed due to linking against different libraries.

Re:Under Windows... (0)

Anonymous Coward | more than 10 years ago | (#7928444)

Do not insult Eugenia, for she is a gorgeous geek babe!

Re:Under Windows... (2, Informative)

mindriot (96208) | more than 10 years ago | (#7928477)

The article was actually written by Christopher W. Cowell-Shah. Also, as others noted, except for the I/O benchmark (where gcc was actually faster) the benchmarks couldn't profit much from the OS around it since they were just computationally intensive, but didn't make much use of OS-specific library functions.

Quick! What's the difference between an apple and (-1, Troll)

Anonymous Coward | more than 10 years ago | (#7928220)

the goatse man. One is full of crispy sweet flavor, and one has a giant asshole.

Seriously - does anyone makes a language decision based on performance reasons alone? I would have to imagine this sort of information does nothing but feed trolls. It certainly provides nothing in the way of a valid comparison.

Re:Quick! What's the difference between an apple a (0)

Anonymous Coward | more than 10 years ago | (#7928243)

Serisously? Where is the joke? I didn't find anything funny in your post. I don't get the joke. Nice attempt though.

Visual C++ runs faster on Windows. Duh (1)

aardwolf204 (630780) | more than 10 years ago | (#7928234)

Visual C++ runs faster on Windows. Duh

Re:Visual C++ runs faster on Windows. Duh (1)

bsharitt (580506) | more than 10 years ago | (#7928325)

Yeah, I've used GCC on Windows, and I'm not surprised that it didn't fare as well. It's defiantly not one of the best Windows compilers.

Re:Visual C++ runs faster on Windows. Duh (1)

mindriot (96208) | more than 10 years ago | (#7928417)

Hm, I don't really think that it has much to do with the fact that it runs on Windows. Except for maybe the I/O bound benchmarks, the others, couldn't be expected to gain much from being 'optimized for Windows'. And gcc was actually better on I/O. VC++ wins on the math benchmarks -- so MS may just have the better set of Intel optimizations.

Besides, it would've been kinda difficult comparing the same set of compilers under native Linux, don't you think?

Not that it would have helped (1)

holzp (87423) | more than 10 years ago | (#7928238)

Not that it would have helped them survive the slashdotting, they run [netcraft.com] linux. Well, when they were up and running that is....

Eugeina Song (Socre:5, funny) (-1, Troll)

Anonymous Coward | more than 10 years ago | (#7928242)

Imagine a place
Where all the trolls play
To rape penguins,
Throw rotten fruit,
To smash the windows,
and to get stung by bees.

Imagine a place
Where you can't get moderated down for obvious flamebait
Where the report abuse function dosent work
But when you attack the Queen Be, you get flamed to a crisp

Imagine a place, where Xandros is always reviewed
Where Gnome trolls bash KDE
Where Macs whack Gnomes
Where Slackware and Debian trolls murder their own mothers

That place is real, and its called Eugenia's Trolling supersite, OSnews.com!

.NET Languages and IL (3, Interesting)

ClubStew (113954) | more than 10 years ago | (#7928247)

Why benchmark the various ".NET languages" (those languages whose compilers target the CLR)? Every compiler targeting the CLR produces Intermediate Languages, or more specifically MSIL. The only differences you'd find is in optimizations performed for each compiler, which usually aren't too much (like VB.NET allocates a local variable for the old "Function = ReturnValue" syntax whether you use it or not).

Look at the results for C# and J#. They are almost exactly the same, except for the IO which I highly doubt. Compiler optimizations could squeeze a few more ns or ms out of each procedure, but nothing like that. After all, it's the IL from the mscorlib.dll assembly that's doing most the work for both languages in exactly the same way (it's already compiled and won't differ in execution).

When are people going to get this? I know a lot of people that claim to be ".NET developers" but only know C# and don't realize that the clas libraries can be used by any languages targeting the CLR (and each has their shortcuts).

Re:.NET Languages and IL (1)

narcolepticjim (310789) | more than 10 years ago | (#7928412)

It's possible to write C# code that will perform better than VB.NET code, as there are some differences in what the languages can do.

VB.NET doesn't support unsigned integers, for instance. VB.NET supports late binding, which would perform much worse than C# in similar apps. Also, C# can use unmanaged code blocks, which could also provide better performance.

Okay, I'm stretching, but there are ways in which C# would outperform VB.NET

Which Java VMs were used? (1, Redundant)

csnydermvpsoft (596111) | more than 10 years ago | (#7928249)

I can't get to the article because of the slashdotting, so can anybody tell me which Java VMs were used in this test? A friend of mine did a study for a supercomputing course in which he found that IBM's J9 Java VM significantly outperformed Sun's JVM.

Java is dying (-1, Offtopic)

zBoD (86938) | more than 10 years ago | (#7928283)

Fact: Windows 98 is dying

It is common knowledge that Java is dying. Everyone knows that ever hapless Java is mired in an irrecoverable and mortifying tangle of fatal trouble. It is perhaps anybody's guess as to which Java is the worst off of an admittedly suffering Java community. The numbers continue to decline for Java but Java 1.4 may be hurting the most. Look at the numbers. The erosion of user base for FreeBSD continues in a head spinning downward spiral.

All major marketing surveys show that Java has steadily declined in market share. Java is very sick and its long term survival prospects are very dim. If Java is to survive at all it will be among hobbyist dilettante dabblers. In truth, for all practical purposes Java is already dead. It is a dead man walking.

Fact: Java is dying

Re:Java is dying (1)

csnydermvpsoft (596111) | more than 10 years ago | (#7928369)

<i>Fact: Windows 98 is dying</i>

Hmm... someone needs to learn how to use search-and-replace better.<g>

String manipulation? (1)

defunc (238921) | more than 10 years ago | (#7928286)


What about some string manipulation numbers?

Would like to see... (3, Interesting)

CaptainAlbert (162776) | more than 10 years ago | (#7928288)

...some analysis of the code generated by Visual C++ and gcc side by side, particularly for those trig calls. If there's that great a discrepancy between the runtimes, that's a good clue that either one of the compilers is under-optimising (i.e. missing a trick), or the other is over-optimising (i.e. applying some transformation that only approximates what the answer should be). I didn't see any mention of the numerical results obtained being checked against what they ought to be (or even against each other).

As any games/DSP programmer will tell you, there are a million ways to speed up trig providing that you don't *really* care after 6dps or so.

OK, maybe I'm just bitter because I was expecting gcc 3.1 to wipe the floor. :)

Re:Would like to see... (1)

iggymanz (596061) | more than 10 years ago | (#7928461)

naw, gcc usually loses to compilers totally optimised for one type of processor family. Gcc has to support 30+ processors, so first code made into intermediate form which then is used to generate code for specific processor. There's just no way gcc is going to be hotter than Intel's C compiler for x86, or MIPS*PRO C on MIPS, or Sun's C on Sparc, etc.

Meaningless (0)

Anonymous Coward | more than 10 years ago | (#7928290)

This is a meaningless benchmark.

In *realworld* applications, different aspects starts to matter, non of which these benchmark covers:

1. IO speed
2. Allocation/deallocation of large (potentially paged) memory segments. How does the language organize memory (wrt memory access pattern) so that cache-locality is maximized?
3. Global optimization starts to matter bigtime
4. Ability to use native features, such as IO completion ports, AEM on linux, etc, etc.
5. bla, bla, bla.

I have a lot experience with large system-level software and Java is sometimes too slow to be useful. For most business applications, however, Java is good enough.

What about the version of Windows (1)

Sonic McTails (700139) | more than 10 years ago | (#7928303)

OSNews is showing the slashdot effect so I can't read the article, but I'm assuming that this was all tested on one version of Windows, like XP. In my personal experience, I found that some programs run faster in other versions, and I found the Java programs ran faster in Windows 98/ME then in XP, so I wonder if this was taken into account and if it wasn't, how much of a difference it would make.

Sitting on a Benchmark (2, Interesting)

Foofoobar (318279) | more than 10 years ago | (#7928307)

Well unfortunately, comparing Java to C# on a Windows machine is like comparing a bird and a dolphins ability to swim in water; Several components of C# are integrated right into the operating system so naturally it's going to run faster on a windows machine. Compare C#, C++ and Java on machines where the components aren't integrated and then we will have a FAIR benchmark.

Oh wait! C# only runs on one operating system. Can you name any other development languages that only run on ONE OS, boys and girls? Neither can I.

Re:Sitting on a Benchmark (3, Informative)

Mathetes (132911) | more than 10 years ago | (#7928399)


Oh wait! C# only runs on one operating system. Can you name any other development languages that only run on ONE OS, boys and girls? Neither can I.


Ximian's Mono has a C# compiler for open OS's:

http://www.go-mono.com/c-sharp.html

Re:Sitting on a Benchmark (0)

Anonymous Coward | more than 10 years ago | (#7928404)

I suppose you have heard of penguins!

Re:Sitting on a Benchmark (1)

Haeleth (414428) | more than 10 years ago | (#7928408)

Can you name any other development languages that only run on ONE OS, boys and girls? Neither can I.

AppleScript?

Re:Sitting on a Benchmark (0)

Anonymous Coward | more than 10 years ago | (#7928470)

Idiot. Think before posting, please.

cant compare Java and C (1)

stonebeat.org (562495) | more than 10 years ago | (#7928311)

Java is intended to run on a Virtual Machine environment, and C is not.
That is Java code can be run on any platform as long as it has a Java virtual machine. You can not do that with C++.
you are comparing apples and oranges.

Re:cant compare Java and C (1)

nberardi (199555) | more than 10 years ago | (#7928360)

Actually the way that they compared them is totally valid. They took simular operations and ran time checks on them. And if what you are saying is true C would always be better because it is closer to the processor than the extra layer managed code adds on. But this shows some discrepency in that old logic.

This is a totally valid test because they aren't testing anything that is deeper than runtime. A more biased test would have been to test memory usuage because C uses pointers and doesn't rely on a GC so it would always have lower memory usuage.

According to this... (1)

nberardi (199555) | more than 10 years ago | (#7928315)

According to this Visual C++ is faster than GCC? But what really surpised me was that a managed language was actually faster than a compiled language. Because look at Java 1.4 it kicked arss in all but a couple categories. Also what I don't understand is why VB.Net got such a high time for IO operations when all the .Net languages are compiling to the same IL. Also why didn't they try compiling VC++ into managed code too.

I would really like to see the code they used for some of these tests because I am sure some of the operations can use some further optimization.

But what really surprised me is that some of the managed code .net kicked butt over GCC.

How can that be, everybody has been complaining about .Net for a while now about how slow it is, and how slow managed code is. Are there any other sources that do simular comparisons?

I don't know how accurate this article is but I am glad to see .Net and Microsoft getting some high scores because it will be a good article to point some of my compiler geeks too. Because there modo so far has been GCC kicks arss even against VC++.

JRE compiled with? (1)

Ruzty (46204) | more than 10 years ago | (#7928329)

One of the performance issues may be which C compiler the JRE was compiled with. I mean think about it, Java is only as good the the application that runs the compiled byte code you hand it.

So, if the JRE was compiled without optimization flags for your CPU type then all java byte code you execute with that JRE is going to perform poorly.

-Rusty

this is just so bogus (4, Insightful)

rpeppe (198035) | more than 10 years ago | (#7928334)

Benchmark code like this does not represent how these languages are used in practice. Idiomatic Java code tends to be full of dynamic classes and indirection galore. Just testing "arithmetic and trigonometric functions [...] and [...] simple file I/O" is not going to tell you anything about how fast these languages are in the real world.

Am I the only one? (1)

gregarican (694358) | more than 10 years ago | (#7928336)

I don't see why it matters in terms of compilation speed. You could use lower level assembly or machine language and even get faster results. But I personally judge a language on its practicality, logic, structure, rules, portability, elegance, security, etc.

All that being said :-| I think that Java in terms of building bytecode, client execution, and server execution is far behind the pack. Unless you have some multiple processor system with tons of bandwidth and memory it seems as if Java code is dog slow. Not trying to troll, just being honest. I can always pick out a website that delivering Java content by the number of times I keep on moving my mouse to ensure my system's still responding.

They should benchmark development time (2, Interesting)

smartin (942) | more than 10 years ago | (#7928339)

Someone should do a study on the time taken to design, implement and debug a resonably complex chunk of code under C++ and Java. I'm pretty sure that the result would show the huge advanatage of Java over C++.

Re:They should benchmark development time (5, Interesting)

ultrabot (200914) | more than 10 years ago | (#7928450)

Someone should do a study on the time taken to design, implement and debug a resonably complex chunk of code under C++ and Java. I'm pretty sure that the result would show the huge advanatage of Java over C++.

The difference b/w Java and C++ would be dwarfed by the difference b/w Java and Python. Java may be 30-40% more productive than C++, but Python is 1000% more productive than Java. And yes, this applies to larger projects. J2EE may come to its own w/ projects that have hundreds of mediocre programmers, but if you have a mid-size team of highly skilled developers creating something new & unique (something like Zope or Chandler), Python will trounce the competition.

Missed a test (1)

eples (239989) | more than 10 years ago | (#7928340)

I think it would be nice to see string functions tested as well - some languages that are particularly poor with the Math are also particularly good with the Strings. IMHO

Bring out the Skeptical Hats... (1)

ThosLives (686517) | more than 10 years ago | (#7928344)

Hrm. I read this article, and it's clear to me that this is not a comparison, per se, of language. There doesn't seem to be any mechanism for accounting for the compiler used. The article also does not seem to indicate such things as file size, time to compile, or that information, which is also part of the "performance" of a language. I'm not sure how you could come up with a benchmark that would do this in the first place. A good study would look at the various languages across different platforms and compilers, determine the variability of "time to perform function X" due to compiler / platform, and then compare against the differences in different languages to determine if there is a statistically significant difference. Right now we only have some partial data - we don't know how much is due to the language, the compiler, or some other unknown noise factors.

These guys need to learn how to run designed experiments and provide useful data! This seems more like something to placate management...

Re:Bring out the Skeptical Hats... (1)

nberardi (199555) | more than 10 years ago | (#7928405)

I have to agree with most things that you brought up because they are valid. But really with a test like this compile time doesn't matter, because programs only need to be compiled once. And if there is a comple seconds between compile time or a couple mintues that doens't really matter because if both programs get used for a year straight with out a recompile the minute or two extra that it takes is really null and void.

Plus a compile time would just be a representive of the compiler used to compile the compiler and how much optimization the compiler might be doing.

Sweet Lord (0, Flamebait)

FortKnox (169099) | more than 10 years ago | (#7928346)

Visual C++ (an MS compiler/IDE) comes out on top.

Java1.4.2 works as good as native code.

Benchmarked on the Microsoft operating system!?!?!

Lemmie go ahead and tell you the responses:
They cheated on the benchmark!
I ran my own benchmark and gcc rocked the casbah!
Java is just slow starting and takes up too much memory.

This article, itself, is flamebait.

Not really a proper comparison (0)

3lb4rt0 (736495) | more than 10 years ago | (#7928350)

either in the original XP based article or the Linux based comparison.

Comparing fully compiled against JIT/interpreted code seems like a fools errand especially when some of the components test aren't native to the platform the test is run on.

What about coder's performance? (3, Interesting)

G4from128k (686170) | more than 10 years ago | (#7928353)

Given the ever accelerating clockspeed of processors, is the raw performance of langauges that big an issue? Except for CPU-intensive programs (3-D games, high-end video/audio editing), current CPUs offer more than enough horsepower to handle any application. (Even 5-year old CPUs handle almost every task with adequate speed). Thus, code performance is not a big issue for most people.

On the other hand, the time and cost required by the coder is a bigger issue (unless you outsource to India). I would assume that some languages are just easier to design for, easier to write in, and easier to debug. Which of these langauges offers the fastest time to "bug-free" completion for applications of various sizes?

Re:What about coder's performance? (2, Insightful)

nuggz (69912) | more than 10 years ago | (#7928428)

I agree, time to market is key.

I like python, it is easy to write, and keep it somewhat clean.

Speed or accuracy? (5, Interesting)

derek_farn (689539) | more than 10 years ago | (#7928354)

The Java performance is best explained by an article by Prof Kahan: "How JAVA's Floating-Point Hurts Everyone Everywhere" http://www.cs.berkeley.edu/~wkahan/JAVAhurt.pdf also see "Marketing vs. Mathematics" http://www.cs.berkeley.edu/~wkahan/MktgMath.pdf I suspect the relatively poor floating-point performance of gcc is also caused by the desire to acheive accurate results.

Where is Fortran? (2, Insightful)

Aardpig (622459) | more than 10 years ago | (#7928361)

It's a pity that the present-day language of choice for high-performance computing, Fortran 90/95/HPF, was not covered in this study. There has been anecdotal evidence that C++ has approached Fortran, performance-wise, in recent years, but I've yet to see a proper comparison of the two languages.

Alternative comparison, compiler shootout (5, Informative)

SiW (10570) | more than 10 years ago | (#7928362)

Don't forget about the Win32 Compiler Shootout [dada.perl.it]

Java results no surprise (1)

Shadowhawk (30195) | more than 10 years ago | (#7928366)

That's not surprising in either case, in terms of Java's results. It's well known that Sun has spent more time optimizing the Windows version of Java than any other, including the Solaris version.

proposed manned moon/mars shot deemed safer than (-1, Offtopic)

Anonymous Coward | more than 10 years ago | (#7928371)

trans-atlantic flight? that's gooed news? at leased we'll still be able to go somewhere?

no wonder the fuddites have promoted the 'illegal' aliens? trying to steal their pateNTdead magnetron minimizer no DOWt?

About the Python performance (5, Insightful)

ultrabot (200914) | more than 10 years ago | (#7928376)

Note that Python is pretty easy to extend in C/C++, so that speed critical parts can be rewritten in C if the performance becomes an issue. Writing the whole program in C or C++ is a premature optimization.

Subset (1)

tedgyz (515156) | more than 10 years ago | (#7928384)

I think this is a pretty good analysis. The very fact that it is non-partisan is a blessing. As a Java web app developer, I notice that if I take a subset of the graph [osnews.com] for int and I/O operations, Java is quite competitive (#1 for the sum of those two metrics) with the other platforms. The trig stuff seems to really skew the results.

What about development ease... (2, Insightful)

Slick_Snake (693760) | more than 10 years ago | (#7928393)

There is more to programming languages than pure speed. I have written programs in a number of languages and find it irritating when people start dissing languages because they are slower at run time. I have on numbers occations written applications for people in "slow" languages because it was easy to write, easy to maintain, and was not a time critical application. In most cases the speed limiting factor is the human sitting in front of the screen.

IMO a program should use whatever tools are available and appropreate for the job, and not just worry about what is faster.

My benchmark of six languages (0)

Anonymous Coward | more than 10 years ago | (#7928414)

1. Bulgarian
2. Russian
3. English
4. Korean
5. Japanese
6. Chinese

Java on Windows and Linux (1)

little_id (513727) | more than 10 years ago | (#7928427)

I conducted my own tests pitting Java 1.4 against gcc 3.3 and icc 8.0 using his benchmark code, and found Java to perform significantly worse than C on Linux/Athlon.

From my experience Windows has always had the edge on the Java VM. But this could all be coming to a change:
Red Hat Linux 9 and Java 2 Platform, Standard Edition 1.4.2: A Winning Combination [sun.com]

A quote from the article states
"Internal tests reveal that performance of NPTL with a client/server application has been impressive. On a 2 X 1.6 Ghz P4 Xeon system, time to completion was 191% faster with J2SE 1.4.2 running on NPTL, compared with J2SE 1.41 running on the original Linux threads."

Their server is getting a good test right now (1, Funny)

Anonymous Coward | more than 10 years ago | (#7928431)

They should have picked the fastest language for their server.

Read the OSNews thread (5, Insightful)

be-fan (61476) | more than 10 years ago | (#7928453)

There were a number of problems with this benchmark, which are addressed in the OSNews thread about the article.

Namely:

- They only test a highly specific case of small numeric loops that is pretty much the best-case scenario for a JIT compiler.

- They don't test anything higher level, like method calls, object allocation, etc.

Concluding "oh, Java is as fast as C++" from these benchmarks would be unwise. You could conclude that Java is as fast as C++ for short numeric loops, of course, but that would be a different bag of cats entirely.

Quoting the results section here... (5, Informative)

Jugalator (259273) | more than 10 years ago | (#7928456)

Site was showing signs of Slashdotting, so I'll quote one of the more important sections...

Results

Here are the benchmark results presented in both table and graph form. The Python and Python/Psyco results are excluded from the graph since the large numbers throw off the graph's scale and render the other results illegible. All scores are given in seconds; lower is better.

int long double trig I/O TOTAL

Visual C++ 9.6 18.8 6.4 3.5 10.5 48.8
Visual C# 9.7 23.9 17.7 4.1 9.9 65.3
gcc C 9.8 28.8 9.5 14.9 10.0 73.0
Visual Basic 9.8 23.7 17.7 4.1 30.7 85.9
Visual J# 9.6 23.9 17.5 4.2 35.1 90.4
Java 1.3.1 14.5 29.6 19.0 22.1 12.3 97.6
Java 1.4.2 9.3 20.2 6.5 57.1 10.1 103.1
Python/Psyco 29.7 615.4 100.4 13.1 10.5 769.1
Python 322.4 891.9 405.7 47.1 11.9 1679.0

this is comparing libraries (0)

Anonymous Coward | more than 10 years ago | (#7928465)

The guy's trig. VS. trig. stuff is just
comparing libraries, not code gen.

IBM Java (3, Interesting)

PourYourselfSomeTea (611000) | more than 10 years ago | (#7928475)

Using the IBM Java VM, I've been able to achieve consistently cutting my runtimes in half over the Sun VM. Anyone currently using the Sun VM for production work should test the IBM one and consider the switch.

My application that I benchmarked is data and network and memory intensive, although not math intensive, so that's what I can speak for. We consistently use 2 GB of main memory and pump a total of 2.5 TB (yes, TB) of data (doing a whole buch of AI style work inside the app itself) through the application over it's life cycle, and we cut our total runtime from 6 days to 2.8 days by switching to the IBM VM.
Load More Comments
Slashdot Account

Need an Account?

Forgot your password?

Don't worry, we never post anything without your permission.

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>
Create a Slashdot Account

Loading...