Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Speed Test 2: Comparing C++ Compilers On WIndows

timothy posted about a year ago | from the for-windows-use-per-second-per-second dept.

Windows 132

Nerval's Lobster writes "In a previous posting, developer and programmer Jeff Cogswell compared a few C++ compilers on Linux. Now he's going to perform a similar set of tests for Windows. "Like all things Windows, it can get costly doing C++ development in this environment," he writes. "However, there are a couple notable exceptions" such as free and open-source cygwin, mingW, Express Versions of Visual Studio, and Embacadero. He also matched up the Intel C++ Compiler, Microsoft C++ Compiler, and the Embarcadero C++ 6.70 Compiler. He found some interesting things — for example, Intel's compiler is pretty fast, but its annoying habit of occasionally "calling home" to check licensing information kept throwing off the rests. Read on to see how the compilers matched up in his testing."

Sorry! There are no comments related to the filter you selected.

Calling home (1, Insightful)

six025 (714064) | about a year ago | (#45528707)

>> its annoying habit of occasionally "calling home" to check licensing information

Calling home for the latest NSA exploits to inject in to your application? /tinfoil-hat-no-so-paranoid-these-days-dept

Re:Calling home (2)

NoNonAlphaCharsHere (2201864) | about a year ago | (#45528763)

..."calling home" to check licensing information kept throwing off the rests.

Oh, they meant the tests.

Re:Calling home (0)

Anonymous Coward | about a year ago | (#45530119)

what? the israeli-excstacy-drug tests, or the compiling of the lists of illegal israeli insider trading?
"/., the tech buck stops here!"

Re:Calling home (0)

Anonymous Coward | about a year ago | (#45528773)

It's pretty trivial to monitor your traffic and determine the difference between a license check and a binary.

So no, probably not. But let's keep an eye on it.

Re:Calling home (4, Interesting)

fuzzyfuzzyfungus (1223518) | about a year ago | (#45528991)

Depends on the paranoia of the DRM implementation.

A simple "We sell to corporations, so we don't expect cracks but we do expect optimists trying to install a few extra seats" licensing system really just needs to (ideally over SSL, hear that, LG?) phone home the serial number at intervals and get a 'yes/no' in response.

The more paranoid systems, designed on the assumption that the client will be under active attack, may be considerably murkier. Something like 'BD+', for instance, for Blu-ray DRM, mandates a proprietary, blackbox virtual machine (with access to the client hardware's memory and the ability to run native code, including applying persistent patches to the client system, in addition to what it does internally) that executes whatever blob of BD+ code burned into the disk you try to play.

A system like that, or an AV-like 'signature' system that is designed to have one or more parts of the client DRM continually interrogating one another in novel ways to detect compromises in the authentication mechanism would be more or less indistinguishable from a binary, because it would be one.

For something like a compiler, rather than a game, or some precious 'premium content', I'd expect the former; but I've underestimated human stupidity before.

Re:Calling home (3, Informative)

Kardos (1348077) | about a year ago | (#45529007)

You might think so, but no. If the licensing check is done in the clear, one could spoof the reply and thus bypass the licensing effortlessly. More likely it's encoded/encrypted in some way that you can't (easily) fake a green light. So you may be able to identify the transmission of "very large binary" or equivalent, but differentiating between a "license check" and "tiny binary" would be tricky since you won't be reading the stream.

Re:Calling home (0)

mikael (484) | about a year ago | (#45531035)

There was a company that sold (or still sells) a blob of software that calculated a registration key based on the configuration of the hardware; serial numbers of the CPU(s), motherboards, GPU boards, installation date of the software, time zone. If any of those changed, you would have to apply for a new registry key.

Re:Calling home (1)

Kardos (1348077) | about a year ago | (#45531221)

That seems like a terrible idea. You'd have many legit claims for new reg keys, how would you know when to deny the requests from people pirating the software? Ugh.

Re:Calling home (1, Troll)

Ravaldy (2621787) | about a year ago | (#45529345)

Who mods this garbage up?

Re:Calling home (0, Troll)

Anonymous Coward | about a year ago | (#45529567)

your expectations of moderators --------


bar for getting mod points _____

Re:Calling home (1)

Anonymous Coward | about a year ago | (#45529587)

Sure, it's nonsense. Then Stuxnet happened. When "someone" is investing that much effort into injecting malicious binary code into specific industrial equipment, I think anything is possible if there was sufficient motivation behind it. Ordinary people as targets? Obviously not worth the trouble, so for most of us it's something we can joke about and forget. But let's say someone is compiling programs to run some important industrial equipment along similar lines as Stuxnet's target. Maybe they're doing it precisely because they don't trust outside code that they can't audit themselves. In that circumstance I wouldn't put it past nefarious people trying to introduce something via the compiler itself. It is a possibility that people have pointed out for years [bell-labs.com] as a theoretical possibility, even though it's probably a pretty challenging vector.

Checking licensing information is pretty far removed from actually injecting code, but if the process also involves automatically downloading updates, there are possibilities for making it happen (e.g., spoofing the update servers).

Re:Calling home (0)

Anonymous Coward | about a year ago | (#45533577)

Remember Stuxnet required a great deal of human interaction to work. Physical break-ins at 2 different companies to secure security certificates. Developers needed in-depth knowledge of control systems and centrifuge internals and access to similar PLC's Omnis, and other SCADA systems for building and testing the code. And finally someone had to walk into the lab and insert a USB into a networked PC to kick things off. Your average hacker does not have the capability to create and distribute this type of attack. I work with oil and gas refineries and pipelines which do not allow any remote access to any of their networks. Any attacks on these systems would require in-house help to succeed. If the controls or systems were compromised there are dead man switches in place to prevent catastrophic failures. If you want to attack infrastructure it is easier to just blow-up the pipelines or power lines.

Re:Calling home (1, Offtopic)

jones_supa (887896) | about a year ago | (#45529949)

Who mods this garbage up?

You can bet it's not me. I almost get sex more often than Slashdot mod points.

Re:Calling home (0)

AliasMarlowe (1042386) | about a year ago | (#45530027)

Who mods this garbage up?

You can bet it's not me. I almost get sex more often than Slashdot mod points.

Damn. I get 5 mod points almost every day. It can be quite exhausting.
A couple of years ago, I was getting 15 mod points daily for a few weeks. Couldn't have taken it much longer...

Re:Calling home (1)

jones_supa (887896) | about a year ago | (#45529913)

>> its annoying habit of occasionally "calling home" to check licensing information

Calling home for the latest NSA exploits to inject in to your application? /tinfoil-hat-no-so-paranoid-these-days-dept

They are also trying to ram down the users' throats a sign-in feature with VS2013 too [wordpress.com] .

Crickets... (1)

Anonymous Coward | about a year ago | (#45528715)

I do believe that no one on slashdot cares about this.

Re:Crickets... (-1)

Anonymous Coward | about a year ago | (#45529225)

perhaps snot.... It has come to the attention of Dice Human Resources that certain individuals on /. staff have tested positive in speed(meth) tests, and certain other have been avoiding the israeli-excstacy tests.

Can someone please compile the list of disproportionate amount of contracts awarded to israeli-affiliated entities? Perhaps we can hold-off on the list of those refusing to discuss israeli-excstacy testing.

Also, a great idea is to compile the list of those who actively deflect from dicussion of *illegal israeli neclear weapons*, *illegal israeli settlements*, *illegal israeli insider trading*.

Perhaps it would be prudent to include a compiled list of illicit israeli-employed wikipedia-editors, and others who apparently COLLUDE and CO-CONSPIRE with the various elements of the AIPAC-lobby, a foreign-intelligence-agency.

Shortly, after AKAMAI.AMDOCS,ONAVO,and the apple-censorship thing are (accurately) revealed to be israeli-espionage operations (yeah, fuck the NSA!),
perhaps we can all *let freedom ring* over the festive season!
Happy Hunting!

Re:Crickets... (3, Insightful)

NotBorg (829820) | about a year ago | (#45529497)

It would help if he actually measured something worthwhile. In the 20+ years I've been coding, I've never once picked a compiler on the basis of how long it takes to spit out a binary. There are just so many other more interesting features and characteristics to consider.

Re:Crickets... (2)

ebno-10db (1459097) | about a year ago | (#45530359)

In the 20+ years I've been coding, I've never once picked a compiler on the basis of how long it takes to spit out a binary.

Then you don't compile big projects. Neither do I, but there are projects where a full build can take hours. Sure build farms and stuff help, but so does a 2x faster compiler.

Re:Crickets... (2)

Tallfeather (648452) | about a year ago | (#45533177)

I do work on large projects in the multi-hour range for a full rebuild, and the compile time is still pretty much the lowest priority in selecting a compiler. All things being equal, of course I'd like to have the fastest compiles possible. But more important than that is that I can write the code I need to write without dodging compiler bugs / shortcomings all day, and deliver a binary which is optimized well for the target.

You can adapt to slow compiles. Breaking the project up into libraries, for example. You can't readily adjust to other compiler problems.

butt goat fart (2)

For a Free Internet (1594621) | about a year ago | (#45528729)

Nobordy uses Windose anymorr, it is nowt a ipon.

Re:butt goat fart (1)

Anonymous Coward | about a year ago | (#45531159)

Dude, lay off the meth.

Here because I got timed out. (-1)

Anonymous Coward | about a year ago | (#45528783)

Richard Dawkiins is on Reddit and I'm her to pass the time while I'm \being timed out with some advertisement ärtivles and their sponsers...

So, all of you suck. No rrealy,. Can't get a job?

You suck.

Why do you suck?

Because hiring managers get a bonus to come under budget and hire cheap people (H1-B) and if you don't fall under that umbrella then you are unemployable.

Silicone Valley is for losers.

Salary to live in Silicone Valley - $300K - speaking as a native.

Re:Here because I got timed out. (0)

Anonymous Coward | about a year ago | (#45529531)

If you're from the states, it's a bit early for the bottle isn't it?

Calling home threw off the results? (5, Insightful)

iYk6 (1425255) | about a year ago | (#45528831)

Did calling home really throw off the results? Since that is something that ordinary users would have to put up with, I would think it should be part of the test. It might be difficult to get an average, but testing Intel's compiler only when it is at its fastest doesn't seem fair.

Re:Calling home threw off the results? (1)

Darinbob (1142669) | about a year ago | (#45532653)

Does the speed of compiling matter that much compared to the speed of the resulting code? I know that C++ compilers can be extremely slow to run but it shouldn't be a major concern unless it's so slow that an incremental build is wasting enough time that you're off getting another coffee or donut while waiting.

does the Intel one still slow down on AMD systems? (3, Interesting)

Joe_Dragon (2206452) | about a year ago | (#45528839)

does the Intel one still slow down on AMD systems and or trun out code with AMD slow down blocks?

Re:does the Intel one still slow down on AMD syste (4, Interesting)

fuzzyfuzzyfungus (1223518) | about a year ago | (#45529057)

My understanding is that they never explicitly 'slowed down' AMD systems; but that the binaries produced by their compiler refused to honor the capabilities flags of non-intel processors (eg. even if an AMD CPU lists 'SSE2, SSE3' among supported instructions, it would get the fallback to non-SSE instructions, while Intel CPUs would get whatever their supported instructions lists specified). No actual 'here be lots of NOPs for no reason'; but x87 on a machine that can do recent SSE is probably enough to achieve the same effect...)

Re:does the Intel one still slow down on AMD syste (2)

K. S. Kyosuke (729550) | about a year ago | (#45529425)

My understanding is that they never explicitly 'slowed down' AMD systems; but that the binaries produced by their compiler refused to honor the capabilities flags of non-intel processors

Oh, my. Just how many major non-Intel x86-64 CPU vendors are there? AMD, and...? It's suspiciously similar to the ACPI [slated.org] and SecureBoot affairs, don't you think?

Re:does the Intel one still slow down on AMD syste (1)

fuzzyfuzzyfungus (1223518) | about a year ago | (#45529525)

Well, our wiki overlords list 15 known CPU IDs [wikipedia.org] ; but one of them is intel, one is AMD, one is a VM, and most of the rest are the forlorn epitaphs of the fallen.

Re:does the Intel one still slow down on AMD syste (0)

Anonymous Coward | about a year ago | (#45529739)

If you're trying to figure out what features are available with only the vendor ID, you're doing it wrong. "GenuineIntel" and "AuthenticAMD" have never been enough to tell you anything about what SSE operations a chip can do. And if every other vendor can parse the rest of the CPUID data, Intel can too (or they can fuck off, IMO).

Re:does the Intel one still slow down on AMD syste (3, Insightful)

fuzzyfuzzyfungus (1223518) | about a year ago | (#45530805)

If memory serves, their argument varies (depending on whether the FTC appears interested or not) between 'fuck you, it's the Intel compiler collection, and it'll do what's best for Intel. Go suck an Opteron if you like AMD so much.' and 'Gosh, we sure know about the capabilities flags; but we can't be sure of the details of other vendors'(*cough*shoddy, probably reverse engineered illegally*cough*) implementations of certain complex features, and our customers expect our compiler suite to provide stable, correct output, so reverting to the x87 codepath is our only real option..."

Re:does the Intel one still slow down on AMD syste (1)

K. S. Kyosuke (729550) | about a year ago | (#45531825)

Why would they need to "reverse engineer" features (instruction set extensions) that are already publicly documented for the benefit of compiler writers and assembly language programmers?

Re:does the Intel one still slow down on AMD syste (0)

Anonymous Coward | about a year ago | (#45529549)

Oh, my. Just how many major non-Intel x86-64 CPU vendors are there?

Why is it Intel's job to waste time supporting processors that aren't their own?

Re:does the Intel one still slow down on AMD syste (1)

Anonymous Coward | about a year ago | (#45529775)

The reason there are these feature flags is that, they can be used to identify the capabilities of the processor. It's not supporting processors that aren't theirs, it's supporting the same features in all processors that support the features.

Re:does the Intel one still slow down on AMD syste (0)

Anonymous Coward | about a year ago | (#45532927)

Why is it Intel's job to waste time supporting processors that aren't their own?

You mean like all the CPUs based on the X86_64 (aka AMD64) instruction set? You know the instruction set invented by AMD that Intel licenced when it became apparent that it would kill Itanium.

Seriously, Intel devised a scheme to determine if a CPU supports advanced features. They documented the scheme and told programmers that it was the correct way to determine the feature set of a CPU. Then they create a compiler that does not use the documented method. In fact, their compiler went out of its way to ensure that only Intel CPUs would actually use the advanced features even if a CPU reported that it could.

F*ck Intel and f*ck any company that would waste my time by doing such underhanded things. I have no respect for Intel and I don't use their sh!t any more.

Re:does the Intel one still slow down on AMD syste (2)

Richard_at_work (517087) | about a year ago | (#45530341)

VIA was also one that was affected by Intels compiler behaviour.

Re:does the Intel one still slow down on AMD syste (1)

fuzzyfuzzyfungus (1223518) | about a year ago | (#45530879)

Oh, VIA... I'm honestly always a bit surprised to see them still trying.

Back before Intel got (slightly) serious about cheap, with 'atom' and AMD got slightly serious about low-power, with some of their APUs, they made more sense, (in particular, a number of rather interesting x86 embedded specialty boards were VIA based, for situations too low-power or cost constrained for a p3/p4); but lately they've been a much tougher sell. Still some interesting specialty stuff; but 'Unichrome' graphics are such a clusterfuck to deal with that they make AMD look like GPU driver gods, and Intel look (while slow) nearly infallible, and both Intel and AMD have put some rather more aggressive parts into what used to be VIA's playground.

Re:does the Intel one still slow down on AMD syste (2, Interesting)

Anonymous Coward | about a year ago | (#45531961)

Long, long ago some review site ran a Via CPU based system while spoofing the CPU ID to appear as an Intel CPU of similar capabilities.
They expected a few percent gain in the FP and INT benches, but oddly got an 8-fold increase in reported memory bandwidth. The other benchmarks appeared to reflect a real increase in memory performance.

Don't wipe your arse with Intel, they're so dirty you'll end up shittier.

Re:does the Intel one still slow down on AMD syste (1)

zubab13 (3445193) | about a year ago | (#45530773)

Just use libsimdpp ( https://github.com/p12tic/libsimdpp [github.com] ) or any of the myriad similar wrappers. With modest time investment you get almost optimal implementation for multiple instruction sets on any compiler you use.

Re:does the Intel one still slow down on AMD syste (1)

K. S. Kyosuke (729550) | about a year ago | (#45530893)

With modest time investment you get almost optimal implementation for multiple instruction sets on any compiler you use.

I'm using ClozureCL and SBCL. I don't think that this is going to work. :-)

Re:does the Intel one still slow down on AMD syste (1)

zubab13 (3445193) | about a year ago | (#45531259)

Sorry, my post was directed to the parent of your post. Somehow I misclicked somewhere and didn't notice.

Re:does the Intel one still slow down on AMD syste (1)

K. S. Kyosuke (729550) | about a year ago | (#45531419)

Doesn't matter, it's still an interesting thing to study. Maybe if I ported and macroified the whole thing for ClozureCL, some good use for me could come from it, too! :-)

Re:does the Intel one still slow down on AMD syste (1)

Billly Gates (198444) | about a year ago | (#45532599)

A lot.

Typing this in a AND Phemom II black edition which is very fast and not that far off from an i7 back in 2010 when I purchased this. True the newer ones are slower per ghz sadly.

But what if AMD's next chip kicked ass! Remember the Athlon and the later AthlonXPs were the fastest x86 chip you could buy a decade ago?

Tomshardware would include Skyrim and other Intel compiled apps and whine how slow their inferior AMD chips are and intel fan boys would gleam ... but regardless I have a problem with Intel.I do not want to pay more money for a CPU that provides less value.

Even if you do want to argue on this price/performance I get unlocked bios that can support virtualization. Not $900 computer to run VMware natively. Any AMD chipset can run it if you turn it on in the bios!

To answer the grandparent YES INTEL compilers DO CRIPPLE AMD if they do not include SSE3, registers, and other items and use non IEEE standard FPU x87 to make their cpus look better. It is like Nvidia crippling OpenCL to force developers to make CUDA apps who then go on how slow AMD's radeons are etc.

Re:does the Intel one still slow down on AMD syste (0)

Anonymous Coward | about a year ago | (#45530433)

There were checks for "GenuineIntel" in the cpuid result (http://www.agner.org/optimize/blog/read.php?i=49). The only thing that could possibly be worse would be *excluding* "AuthenticAMD". And that's debatable.

Re:does the Intel one still slow down on AMD syste (0)

Anonymous Coward | about a year ago | (#45530453)

Just use libsimdpp ( https://github.com/p12tic/libsimdpp ) or any of the myriad similar wrappers. With modest time investment you get almost optimal implementation for multiple instruction sets on any compiler you use.

Re:does the Intel one still slow down on AMD syste (1)

eulernet (1132389) | about a year ago | (#45533169)

My understanding is that they never explicitly 'slowed down' AMD systems

You are wrong:

"Overview of CPU dispatching in Intel software"
http://www.agner.org/optimize/blog/read.php?i=49#121 [agner.org]

Representative benchmarks? (4, Interesting)

jameson (54982) | about a year ago | (#45528891)

Based on his description, he is using a very synthetic benchmark:

The code I’m testing contains no #include directives, and makes use of only standard C++ code. It starts with one class, and then is followed by 6084 small classes derived from various instantiations of the template classes. (So these 6084 classes are technically not templates themselves.) Then I create 6084 instantiations of the original template class, using each of the 6084 classes. The end result is 6084 different template instantiations. Now, obviously in real life we wouldn’t write like that (at least I hope you don’t).

So in his own words, the code does not reflect realistic compiles. There is no reason to assume that the result generalise to any programs that anyone actually cares about.

Also, there are no error bars of any kind listed.

In other words, I have no reason to assign any meaning to these numbers.

Re:Representative benchmarks? (1)

OzPeter (195038) | about a year ago | (#45529139)

In other words, I have no reason to assign any meaning to these numbers.

Given the reaction to the previous article I don't know what this guy is even trying to do.

And why 6084? What is so special about that number?

Re:Representative benchmarks? (4, Insightful)

c++ (25427) | about a year ago | (#45529305)

In other words, I have no reason to assign any meaning to these numbers.

Given the reaction to the previous article I don't know what this guy is even trying to do.

And why 6084? What is so special about that number?

6084 / 2 % 100 == 42

That is meaning enough.

Re:Representative benchmarks? (3, Insightful)

Ravaldy (2621787) | about a year ago | (#45529447)

The article is alright but not one I would use to pick a compiler. IMHO the resulting EXE is more important than the compiler processing time. I've dealt with large sized applications and if structured properly, your build times on a modern computer should not be an issue.

Re:Representative benchmarks? (1)

Gothmolly (148874) | about a year ago | (#45529665)

Its like your average Moronix linux kernel benchmark, where they have bars and numbers with no scale. Basically, look at me, I'm an attention-whore nerd!

Re:Representative benchmarks? (2)

Anonymous Crowbar (692255) | about a year ago | (#45529757)

Not sure if anyone already mentioned this but my take is it is NOT how fast the project compiles but rather the performance of the executable. If one is building race cars it's not how fast they come off the assembly line, it's how fast the cars go on the track.

Re:Representative benchmarks? (1)

Kickasso (210195) | about a year ago | (#45531943)

You build race cars, we build family sedans, mmkay?

Re:Representative benchmarks? (2)

EMN13 (11493) | about a year ago | (#45531833)

Oh and one minor detail: did you see the final compiled code sizes and how much smaller the optimized versions are (esp. clang!). I'm willing to bet the entire benchmark just code "optimized away" by dead code elimination; and that's an entirely unrealistic situation... Also, where's the code? Is this reproducible?

The benchmark isn't worth anything.

Embarcadero cheap? (2)

TechyImmigrant (175943) | about a year ago | (#45529055)

I took a quick took at their website. It looks quite scammy, they only talk about how much you will save, not about how much it will cost.
After clicking through the buy-now buttons twice, I found the C++ version was $4000.

Re:Embarcadero cheap? (1)

Anonymous Coward | about a year ago | (#45529437)

Hmm, I looked at their website and see prices from $999 to $3999. The 64 bit compiler is included at the $999. Looks like all kinds of enterprise database stuff is in the $3999 version.

Re:Embarcadero cheap? (1)

TechyImmigrant (175943) | about a year ago | (#45529867)

Well there were lots of options. I clicked on the one with C++ in it.

Either way, $999 or $3999 is a barrier to me using their products. I could use it in production, but in production I'm going to use the tools that I'm fluent in because they're free and so I get to use them everywhere.

I was questioning TFA, because it implied that Embarcadero was cheap/free. It isn't.

Re:Embarcadero cheap? (0)

gbrandt (113294) | about a year ago | (#45529509)

Quoting a lower ranked answer because I use 64 bit Embarcaderos compiler and find it quite nice.

"Hmm, I looked at their website and see prices from $999 to $3999. The 64 bit compiler is included at the $999. Looks like all kinds of enterprise database stuff is in the $3999 version."

Executable size (1)

istartedi (132515) | about a year ago | (#45529063)

LLVM has got to be dynamically linking and stripped by default. There are switches on the other compilers that will let you do that, and it looks like they're being ignored.

Re:Executable size (0)

Anonymous Coward | about a year ago | (#45529593)

On the VC one he most certainly is leaving them in. VC puts an ass load of debug info in if you dont explicitly tell it not to. It also by default links in a few extra libraries you may or may not need. He is also using /Ox on a synthetic codebase. So depending on the code it may be /Ot == /Os. With some hand wavy of 'dont know why'. Then on the VC compiler he did not do /Od (disable any optimizations) for his disable test. At that point I concluded this dude was speaking from his ass.

Even comparing G++ to llvm can be weird. As they include different types of optimizations from O2 to O3 as each project considers something things more safe or not.

Most of this info can be retrieved from either the man pages or /? with the compiler.

These sorts of benchmarks are of the type of 'kind of interesting' but tell me little about what these switches will do to my code.

Many times if a C/C++ code base is taking a long time to compile it is because of header bloat. You can usually get a really decent speed up by just going thru your headers and making sure you *really* need them. Even doing the simple thing of boxing out the headers with #ifdefs/#ifndef patterns can help a bit.

Generated code speed (5, Insightful)

c++ (25427) | about a year ago | (#45529087)

This doesn't test the speed of generated code. I like to know which compiler produces faster code when looking at benchmarks.

Re:Generated code speed (0)

Anonymous Coward | about a year ago | (#45533841)

These benchmarks are worse than awful. Execution time? Runtime memory usage? STL and default allocator performance? These are things that actually matter. Compile time and executable size are useless. I haven't worried about executable sizes (except for PS3 spu or embedded code) for over a decade. Compile time is important, but this benchmark fails to give any indication of performance compiling large production systems.

A larger question might be... (-1, Troll)

roc97007 (608802) | about a year ago | (#45529117)

Why? Just... why?

Inaccurate test. (5, Insightful)

johnnys (592333) | about a year ago | (#45529119)

According to the fine article, "The Intel compiler occasionally âoecalls homeâ to an Intel-owned Website to check licensing information. When it does so, it prints out a message about when the current license expires. I didnâ(TM)t use the results when that happens, since it would add time and skew the timing results. " WRONG. The tester should not have excluded these results where time was wasted with this nonsense: If WE the users have to put up with it, it SHOULD be included in the benchmarks.

Re:Inaccurate test. (3, Insightful)

TheCarp (96830) | about a year ago | (#45529373)

while absolutely correct, and not just we put up with it.... if the license check is what the compiler does, then it is what it does. To leave those out is to be measuring something other than the real behaviour of the compiler in real situations.

Hell if this is the case, can you really call the testing complete if he didn't simulate network conditions like, the licensing server being unreachable, or having really high latency?

Re:Inaccurate test. (1)

wonkey_monkey (2592601) | about a year ago | (#45529391)

There also doesn't seem to be anything about how good the executables the compiler produces are. Y'know, the whole reason for the existence of compilers.

Useless Comparison (0, Insightful)

Anonymous Coward | about a year ago | (#45529267)

Wow, lets look at what's being measured here, COMPILE TIME, and EXECUTABLE SIZE...what about the performance of the generated application.

This doesn't measure optimizations, this just measures COMPILE TIME..I don't care if my applications takes 1 sec or 1 hour to compile, I care about the PERFORMANCE of the actual APPLICATION.

This is just crap.

Re:Useless Comparison (0, Insightful)

Anonymous Coward | about a year ago | (#45529339)

Wait until you grow up and are compiling systems with millions of lines of code that takes hours. We were happy when a compiler upgrade cut our times from 10 hrs, to 5. You may not care for your kiddie projects, but in the real world, time is money.

Re:Useless Comparison (-1)

Anonymous Coward | about a year ago | (#45529543)

in the "real world", we parallelize the makefile. you may not care about this for your 1 million line VisualBASIC program, but other people who have more than one file do.

Time is money, and clients expect an application to run fast.

When you need sub-ms response times, you begin to appreciate good compilers.

Than again, for the average Java Code Monkey, latency doesn't seem to matter for their little android apps.

My Application..stock trading, where time == money indeed...we test our algos for months, so we don't care abut compile times..it's all about the application response time. .The average Java idiot wouldn't understand, since "doing X is hard..let Java do it for you."

Re:Useless Comparison (-1)

Anonymous Coward | about a year ago | (#45529967)

I compile kernels and web browsers which are many millions of lines and it doesn't take hours. Adding more cores is cheaper than the cost analysis of using one compiler over another.

And if you can't keep your developers busy on something else while they're compiling... yer dumb and doing it wrong.

Re:Useless Comparison (0)

Anonymous Coward | about a year ago | (#45530127)

I can compile the Linux kernel in a few minutes, then again, I have 32 cores at my disposal.

When our devs are not compiling, they are testing, writing tests, or running tests.

Again, application runtime, not compile time is the true test of a compiler.

Re:Useless Comparison (0)

Guspaz (556486) | about a year ago | (#45530373)

Compiling millions of lines of code only takes hours if you're using a computer made in the 90s... and that's not even taking partial compiles into account.

Compile time hasn't been a relevant metric for many years. After all, you're only going to compile once, but your code might run millions of times. I'd rather shave 100ms off an execution than 100s of a compile.

Re:Useless Comparison (1)

gstoddart (321705) | about a year ago | (#45530545)

After all, you're only going to compile once

I've worked at places where everything gets built every night to ensure nobody has broken anything.

So, in some cases, you may well compile many many times.

I'm not sure it's how I'd primarily choose a compiler, but in some places it could certainly be a factor.

Re:Useless Comparison (0)

Guspaz (556486) | about a year ago | (#45530903)

And we use continuous integration such that we're always recompiling... but the compile speed is still not a major concern...

Compile speed might batter when you're talking about JavaScript or Java or any other JIT language, but c++? Not so much.

Re:Useless Comparison (0)

Hognoxious (631665) | about a year ago | (#45530485)

In the real world, if the number of times it's compiled isn't miniscule compared to the number of times it's run you're doing it wrong.

Embarcadero? (0)

Anonymous Coward | about a year ago | (#45529619)

never heard of Embarcadero.

I've heard of Borland, Watcom, Digital Mars and Bloodshed

Speed is always nice but... (2)

mark-t (151149) | about a year ago | (#45529669)

I'd just like to see a C++11 compiler for windows.

Re:Speed is always nice but... (1, Troll)

ebno-10db (1459097) | about a year ago | (#45530523)

Forget C++11 - switch to D. No, I can't do it either, but I can dream. C++11, for all that it has some nice features, continues the endless quest to make C++ ever more baroque, and to give it a syntax that makes the result of an obfuscated code contest look the same as any other code. It can be done so much more cleanly. In fact Walter Bright and Andrei Alexandrescu already have.

One of the interesting things about D is that both Bright and Alexandrescu are serious C++ experts. I don't think Bright decided to implement "C++ done right" because he didn't understand the features and nuances of C++.

Re:Speed is always nice but... (1)

gbrandt (113294) | about a year ago | (#45530599)

Embarcadero's 64 bit compiler uses CLANG 3.1, so it has most of C++11

Re:Speed is always nice but... (0)

Anonymous Coward | about a year ago | (#45530797)

The version of g++ currently distributed with MinGW has one of the largest lists of C++11 features. I have yet to hit a feature that I want to use but isn't implemented.

Re:Speed is always nice but... (1)

Billly Gates (198444) | about a year ago | (#45532657)

VS 2013 which just came out supports C++ 11. At least that is what MS is saying.

Re:Speed is always nice but... (1)

mark-t (151149) | about a year ago | (#45533135)

No, it doesn't. Most notable omissions include constexpr, user-defined literals, inheriting constructors, and attributes. There are others, but those are the big ones, IMO.

Re:Speed is always nice but... (1)

gbrandt (113294) | about a year ago | (#45533535)

Did you somehow miss the 'most' in my comment. It has what CLANG 3.1 supports which is quite a chunk.

Re:Speed is always nice but... (1)

jonwil (467024) | about a year ago | (#45533731)

They have just released a "tech preview" of the compiler that (per the Microsoft provided info) supports constexpr and inheriting constructors with a clear roadmap to supporting the rest of C++11/C++14 (including user defined literals and attributes)
http://blogs.msdn.com/b/vcblog/archive/2013/11/18/announcing-the-visual-c-compiler-november-2013-ctp.aspx [msdn.com] is the announcement from the Visual C++ guys about it.
 

I call BULLPUCKY! (0)

Anonymous Coward | about a year ago | (#45529741)

I guess they don't teach science in Computer Science, because the Linux post from 23 days ago doesn't state the compiler options that were used. Even after having the glaring omission pointed out by a commenter 20 days ago, the problem still hasn't been fixed. Fixing it should be a priority over moving on to Windows.

Invalid Benchmark - Who Cares (3, Insightful)

MerlynEmrys67 (583469) | about a year ago | (#45529905)

Benchmarking compilers on how long it takes to compile would be like benchmarking cars based on how long it takes to fill the gas tank.
There are so many things that can affect compile time more than the compiler - and the end customer really doesn't care anyway. Frankly, if you want a 3-5x speedup, just put the whole thing on an SSD and let it fly.

Re:Invalid Benchmark - Who Cares (1)

Guspaz (556486) | about a year ago | (#45530397)

benchmarking cars based on how long it takes to fill the gas tank.

Electric cars have made that an extremely relevant benchmark... and marketing stunts involving battery swaps have indeed benchmarked how long it takes to fill a tank.

Re:Invalid Benchmark - Who Cares (1)

MobyDisk (75490) | about a year ago | (#45530639)

Electric cars fill their gas tanks very quickly!

Businesses care (0)

Anonymous Coward | about a year ago | (#45532975)

Right now, I'm posting this as I wait for a brain-dead compiler to complete its task. This is productive time lost because we've chosen a compiler which can't figure out dependency management, takes a long time to compile, and needlessly recompiles unchanged files. Compile time is the single largest user-changeable component of the compile-edit-debug cycle.

It matters to those with a deadline to meet, or to those who'd like to see their families once in a while.

Compiler++ (1)

jones_supa (887896) | about a year ago | (#45530325)

Also Microsoft's Jim Radigan held a cool presentation [msdn.com] in GoingNative 2013 where he reveals some optimization tricks done by the MSVC++ compiler. It also shows some screenshots where Windows is being compiled on a monster multi-core machine.

I win! (0)

Anonymous Coward | about a year ago | (#45530423)

My compiler instantly produces a 0-byte executable for any codebase. You can't actually run the resulting file, but since all we care about here is compile time and executable size, I guess my compiler wins!

Re:I win! (1)

dyingtolive (1393037) | about a year ago | (#45530845)

I would argue that your executable returns with expected result nearly instantaneously.

VS version (0)

Anonymous Coward | about a year ago | (#45530691)

Am I blind or is every compiler version specified except for the Visual Studio one? Version 18 (which ships with 2013) is the latest.

Also, the VS compiler times are reversed. All of the others listed in the article are "No optimization" followed by "full optimization".

Final nitpick, /Ox favors execution speed

Fir5t (-1, Flamebait)

Anonymous Coward | about a year ago | (#45531183)

I know it s0x0Rs, maggot, vomit, shit

Intel compiler does not phone home for licensing (4, Informative)

stevel (64802) | about a year ago | (#45531677)

The Intel compilers do NOT "phone home" for licensing. What they do "phone home" for is to send anonymous usage data. When you install, you're asked if you want to opt in to this - it is not enabled by default. Licensing is done entirely locally for single-user licenses. See http://software.intel.com/en-us/articles/software-improvement-program [intel.com] for more information.

He forgot OpenWatcom (0)

Anonymous Coward | about a year ago | (#45532441)

I used that one a lot when I was younger. It is a free C/C++/Fortran compiler for Windows, OS/2, DOS and Linux. It has an integrated IDE and also does cross-compiling.

Too many words (0)

Anonymous Coward | about a year ago | (#45532527)

Given how a comparison like this is fairly objective for a variety of reasons, it would have been a much better use of the readers' time to just do a final chart which shows you which compiler, on average, and from his specific tests, performed the best. Then toss in another chart which compares binary size, even though people aren't really going to care as much about that.

Posting to cancel moderation. (1)

Shemmie (909181) | about a year ago | (#45533991)

Posting to cancel moderation.

Load More Comments
Slashdot Login

Need an Account?

Forgot your password?