Beta

Slashdot: News for Nerds

×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

cancel ×

298 comments

Money (5, Insightful)

TheRealMindChild (743925) | more than 5 years ago | (#24432213)

The reasons of this behavior of FutureMark product are not yet known

Easy. Intel paid them to make it that way.

Re:Money (5, Funny)

Shadow Wrought (586631) | more than 5 years ago | (#24432319)

Yep. They increased the L2 Cash size.

Re:Money (5, Interesting)

SimonGhent (57578) | more than 5 years ago | (#24432353)

Easy. Intel paid them to make it that way.

If anyone can come up with a better explanation I'd be interested to hear it.

Re:Money (4, Insightful)

Lord_Frederick (642312) | more than 5 years ago | (#24432435)

Even if this is an unintentional error, they have certainly lost some credibility.

Re:Money (1, Offtopic)

Lord_Frederick (642312) | more than 5 years ago | (#24432509)

After reading that post, just mod me -1 duh.

Re:Money (5, Funny)

Anonymous Coward | more than 5 years ago | (#24433589)

Mods, while you're at it, mod me +5 insightful for pointing out that the parent post was modded +4 insightful for pointing out that its parent was redunant...

Re:Money (5, Informative)

ArcherB (796902) | more than 5 years ago | (#24432481)

Easy. Intel paid them to make it that way.

If anyone can come up with a better explanation I'd be interested to hear it.

TFA offers the following:

At the very least, this suggests some incredibly sloppy coding on Futuremark's part, as the company may be enabling or disabling CPU optimizations based on a processor's vendor name in CPUID instead of actually checking CPUID for SIMD support. In this case, PCMark 2005's memory subsystem test doesn't appear to be aware that Nano supports SSE2 and SSE3, and is instead running a decidedly less-optimized code path. There are two factors, however, that make this explanation a bit difficult to swallow.

First, there's the issue of timing. PCMark 2005 was released (obviously) in 2005, and was obviously coded with an eye towards supporting current and future processors. This is standard operating procedure for Futuremark, which always builds benchmarks designed to last for at least a year, and often two. VIA's C5N-T (Nehemiah) core may have only supported MMX and 3DNow!, but the C7 launched in 2005, and that processor supported SSE2 and SSE3 from day one. Even if proper extension support wasn't built into the first version of PCM2K5, we tested version 1.2.0, and that patch was released on or around 11-29-2006.

Second, there's the issue of performance when Nano is identified as AuthenticAMD. If performance between the AMD and Intel CPUIDs was identical, there wouldn't really be a story here, but it isn't, and that's curious. Futuremark could plausibly argue that VIA's C3/C7 processors weren't exactly on the radar back in 2004-2005, but AMD and K8 certainly were, and K8 launched with full SSE and SSE2 support, with SSE3 added in 2005.

There's more, but I don't want to quote the entire article.

Re:Money (5, Insightful)

ShieldW0lf (601553) | more than 5 years ago | (#24432893)

Moral of the story is, when you're dealing with code like this, where it has the capacity to influence who receives billions of dollars and who doesn't, well, you can't trust it if it's closed source and not subject to public scrutiny.

Closed source test suites cannot be trusted, shouldn't even be considered by potential purchasers, and have been misleading the public for years and years. This is mute evidence to the fact.

troll? really? mod up again! (2, Interesting)

Albert Sandberg (315235) | more than 5 years ago | (#24433239)

you got a point there which is important to the discussion, if the source is closed, how can we know if the test is fair?

Re:Money (5, Interesting)

Fumus (1258966) | more than 5 years ago | (#24433693)

What I don't get is why game developers don't release freeware benchmark versions of their engines.
Saying that a config has 9000 points is pretty much useless. Saying that it gets an average of 40FPS in the UT3 benchmark at high details, and 1680x1050 is much more informative.

Unfortunately, this also is a little bit more complicated, and as we know everything simpler is more popular with the dumb masses.

Re:Money (5, Informative)

Eponymous Cowboy (706996) | more than 5 years ago | (#24432917)

I'll give 10:1 odds that Futuremark simply compiled their benchmark with Intel's C++ compiler.

I wrote a detailed explanation [slashdot.org] back in 2005 about how the Intel C++ compiler generates separate code paths for memory operations to make AMD processors appear significantly slower, and how you can trick the compiled code into believing your AMD processor is an Intel one to see incredibly increased performance. See this article [slashdot.org] for additional details.

Re:Money (4, Interesting)

JamesP (688957) | more than 5 years ago | (#24433219)

Yes, I remember that...

But why would icc make AMD better than "no name" beats me.

Re:Money (1)

cgenman (325138) | more than 5 years ago | (#24433623)

The devil you know?

Re:Money (0, Offtopic)

interiot (50685) | more than 5 years ago | (#24433259)

Damn, I've got moderator points, but where is the "this is the only post you need to read" option when you need it?

Re:Money (2, Insightful)

DrWho42 (558107) | more than 5 years ago | (#24433263)

If the authors of this benchmark test were competent they would have written the code for low-level tests like memory bandwidth in assembly language, so compiler choice would not impact them.

Re:Money (-1, Troll)

Anonymous Coward | more than 5 years ago | (#24433403)

Parent post is FUD. It is simply not true to say the compiler generates seperate code paths "to make AMD ... appear significantly slower". The Intel C++ compiler generates seperate code paths to avoid Intel needing to test on every non-Intel CPU. Poster refers to this fact in the linked article. Fact was disclosed by Intel.

Poster's development priorities may be different from Intels but there is no evidence of wrongdoing on Intel's part.

Nor do I see any evidence that poster's "fix" for the issue will not cause untold problems in various other parts of your code. The whole point is that the compiler's code will *support* all x86 processors and be *very fast* on Intel processors. The "fix" almost certainly prevents the support of all x86 processors while creating faster code on *some* AMDs.

This is just a classic example of amateur (poster) vs professional (Intel dev team).

Flame over.

Re:Money (5, Insightful)

clickclickdrone (964164) | more than 5 years ago | (#24433633)

This is just a classic example of amateur (poster) vs professional (Intel dev team).

Writes an (anonymous) Intel representative.

Re:Money (4, Insightful)

Kamokazi (1080091) | more than 5 years ago | (#24433573)

And this is partly why I generally ignore benchmark scores, and look at real-world performance. It's possible for the benchmark or the hardware being benchmarked to 'cheat' or at least behave very differently and produce bogus scores. If i'm looking for a new video card, I don't look at 3DMark scores, I look at framerates in games that I play (or that use the same engine). If I'm looking for a CPU, I'll look at RAR compression times or video encoding speeds. If I'm looking for a storage solution at work, I look at file copy speeds of similar file quantities and sizes, or I/O performance of a similar database.

Re:Money (4, Funny)

Chrisq (894406) | more than 5 years ago | (#24432817)

Easy. Intel paid them to make it that way.

If anyone can come up with a better explanation I'd be interested to hear it.

OK, far-fetched it maybe but what if VIA paid them to do it so that the expose would generate a lot of free advertising and ram home the information that the Nano is faster.

Alternatively the US military could have engineered it to distract us from the possibility that they are working for aliens and have files full of UFO data on their systems. Gosh, i'd better hack in and take a look....

Re:Money (0)

Anonymous Coward | more than 5 years ago | (#24432479)

Easy. Intel paid them to make it that way.

Good god sir, you must be some kind of CIA detective!

Case closed.

Re:Money (4, Funny)

Chrisq (894406) | more than 5 years ago | (#24432725)

It is just what you'd expect with "Intel" inside ..... even inside another manufacturer's processor!

Re:Money (1, Funny)

aikodude (734998) | more than 5 years ago | (#24432905)

It is just what you'd expect with "Intel" inside ..... even inside another manufacturer's processor!

<Larry the Cable Guy>I don't care who y'are. That thar's funny!</LTCG>

(sorry i'm outta mod points. that was funny!)

Re:Money (0)

Anonymous Coward | more than 5 years ago | (#24432959)

Easy. Intel paid them to make it that way.

Everything is easy, when you're stupid.

GenuineIntel (5, Funny)

Plantain (1207762) | more than 5 years ago | (#24432219)

I'm a GenuineIntel, mod me 47% higher!

GenuineSlashdot (0)

rarel (697734) | more than 5 years ago | (#24432245)

HA! As a GenuineSlashdot post, I should get even better than that!

Re:GenuineIntel (5, Funny)

Metorical (1241524) | more than 5 years ago | (#24432383)

People running Intel can get Score: 7.35, Funny?

Re:GenuineIntel (4, Funny)

despe666 (802244) | more than 5 years ago | (#24432833)

If it's an intel processor doing the math, then yes [wikipedia.org]

Re:GenuineIntel (1)

inasity_rules (1110095) | more than 5 years ago | (#24432911)

Calculated on a P1-90MHz in excel 2007...? *runs and hides*

The explanation is simple... (0)

Anonymous Coward | more than 5 years ago | (#24432227)

Intel is faster. The commercials say so, and commercials don't lie.

Money? (3, Insightful)

IWantMoreSpamPlease (571972) | more than 5 years ago | (#24432265)

Seems obvious, but follow the money trail, does PCMark get backing from Intel?

Compiler Optimization? (2, Insightful)

dlgeek (1065796) | more than 5 years ago | (#24432281)

A VIA Nano CPU has had its CPUID changed from the original VIA to fake GenuineAMD and GenuineIntel. An improvement of, respectively, 10% and 47% of the score was seen.

It sounds to me like this could possibly be explained by some kind of conditional optimization that the compiler puts in for various chips, to take advantage of differences in their designs that can improve performance.

Then again, probably not.

Re:Compiler Optimization? (3, Insightful)

Plantain (1207762) | more than 5 years ago | (#24432305)

This could all be explained if they compiled with something silly like ICC

http://www.theinquirer.net/en/inquirer/news/2005/07/13/intel-compiler-nobbles-amd-chips-claim [theinquirer.net]

Re:Compiler Optimization? (2, Insightful)

Anonymous Coward | more than 5 years ago | (#24432947)

Yeah, except that ICC Intel optimizations frequently improve AMD scores as well (over generic optimizations). Not always as much as it helps Intel processors, but it does help some.

Re:Compiler Optimization? (2, Interesting)

k_187 (61692) | more than 5 years ago | (#24432325)

Exactly, what happens when you run an AMD chip under both IDs? or an Intel? If the Intel optimizations are faster on all three, then there might be a problem, as it stands the via chip might just benefit more from the Intel optimizations than the AMD ones.

Re:Compiler Optimization? (4, Insightful)

MBCook (132727) | more than 5 years ago | (#24432581)

You can't. That's why it was discovered now. Intel and AMD don't let you change the CPUID results on their CPUs. Via DOES let you change it. (You could hack the benchmark to change the checks, but then your results are invalid because you changed the benchmark code)

Either way, that's not an excuse. As Ars points out, if it is just checking for something like SSE2 the Nano has that. If you want to make an optimized code path it should be based on if a feature is reported as present or not, not who made the CPU.

It's just really REALLY fishy.

Re:Compiler Optimization? (1)

fitten (521191) | more than 5 years ago | (#24433079)

Either way, that's not an excuse. As Ars points out, if it is just checking for something like SSE2 the Nano has that. If you want to make an optimized code path it should be based on if a feature is reported as present or not, not who made the CPU.

Not true at all... you may also want to know who made it. The chip implementation may give better performance of, say, integer or floating point divide on AMD than it does on Intel, so if AMD, use divide but on Intel, you may want to use inverse + multiply. Some sequences of instructions on Intel may be faster than the same exact sequence on AMD because the Intel made chip may have any number of implementation differences... more rename buffers so it can stall less, more decode resources for wider simultaneous execution, or any number of other things. If you're *really* out to test performance (and need that performance) you *will* optimize on a clock count basis and to know that, you have to know the implementation specifics of *the* processor you're on.

Re:Compiler Optimization? (1)

k_187 (61692) | more than 5 years ago | (#24433185)

I'll give you that its fishy, but never attribute to malice what can be explained by sloppy coding.

Re:Compiler Optimization? (3, Insightful)

brunascle (994197) | more than 5 years ago | (#24432585)

Exactly, what happens when you run an AMD chip under both IDs? or an Intel?

As TFA mentions, we cant test it. AMD and Intel lock the CPUIDs on their chips. VIA doesnt. I do think AMD should do some testing in-house though, as I'm sure they could change the CPUID themselves. Though I wouldnt be surprised if they'd already tried this long ago. I know I would have. And if there were major discrepancies, we probably would have heard about it by now.

Re:Compiler Optimization? (5, Informative)

afidel (530433) | more than 5 years ago | (#24432967)

CPUID can be intercepted so it shouldn't be a big deal to grab the call and return whatever you want.

Re:Compiler Optimization? (4, Insightful)

mikeabbott420 (744514) | more than 5 years ago | (#24432359)

Code that only used SSE3 or some such on the basis of the CPU ID might explain it but conspiracy is more fun to believe. Lies, Damn Lies and Benchmarks.

Re:Compiler Optimization? (0)

MBGMorden (803437) | more than 5 years ago | (#24432475)

But the CPUID doesn't convey features. GenuineIntel has been used on chips going back well before SSE existed. So assuming and using any features based on the CPUID is foolish at best.

Re:Compiler Optimization? (1)

DJProtoss (589443) | more than 5 years ago | (#24432613)

True, but it could be a lazily programmed first test - i.e. if it doesn't mark itself as genuineintel it can't have SSE2/3 (whichever was just out at the time), so why bother looking for it?

Re:Compiler Optimization? (1)

MBCook (132727) | more than 5 years ago | (#24432981)

Because by the time the benchmark came out, the AMD K8/Hammer series was out which had SSE2. SSE3 was added in the next update, which was before the most recent patch to the benchmark.

Ether way there is a set of flags returned by CPUID that specifically lists what features a chip has (like SSE1/2/3). To not check that would be moronic.

Re:Compiler Optimization? (1)

Bert64 (520050) | more than 5 years ago | (#24432887)

Well, it might not be SSE2 based at all...
It could be that it recognises the nano chip as an older variant VIA chip, and thus follows a codebase optimized for *that* chip...
Seeing an Intel it follows a codebase optimized for whatever chip Intel had out at the time, and thus performs better.

As an example, cyrix processors used to have very weak floating point support, to the extent that it was sometimes quicker to run software floating point emulation. The older VIA chips may have had a similar issue perhaps with SSE, an issue which has been rectified in the current models but which the benchmark is unaware of.

That said, benchmarks should have a single code path, or if they have more than one they should be entirely under user control to ensure accurate benchmarks.

Personally i think all benchmarks should comes with source, as should the compiler, so you can be sure what code is going in and what's coming out, and thus what's being executed. Same for any libs that are linked in.

Re:Compiler Optimization? (1)

dlgeek (1065796) | more than 5 years ago | (#24432943)

The older VIA chips may have had a similar issue perhaps with SSE, an issue which has been rectified in the current models but which the benchmark is unaware of.

I think it's much more likely that this should read "which the compiler is unaware of".

Re:Compiler Optimization? (1)

LWATCDR (28044) | more than 5 years ago | (#24432469)

That is actually a pretty good guess.
An extreme example is the Intel compiler which used to do great optimization on Intel CPUs but really bad optimization on AMD.
The VIA chip is really new and the code the compiler generates treats it as the a P3 or P2.

Re:Compiler Optimization? (4, Insightful)

neokushan (932374) | more than 5 years ago | (#24432487)

In all likelihood, this probably IS the case, but that still goes a long way to discredit Futuremark as it shows their benchmarks were certainly NOT fairly tested.

Re:Compiler Optimization? (1)

geckipede (1261408) | more than 5 years ago | (#24432501)

The article mentions this. The results aren't just different depending on whether the ID string is correct or not, but also on what incorrect value it is set to.

Closed Source Benchmarks? (2, Insightful)

bill_mcgonigle (4333) | more than 5 years ago | (#24432845)

It sounds to me like this could possibly be explained by some kind of conditional optimization that the compiler puts in for various chips, to take advantage of differences in their designs that can improve performance.

People are trusting closed-source benchmarks? Well, golly gee, who'd'a thunk there'd be errors, oversights, or shenanigans?

If this was used for anything more than entertainment value, any methodical person would have at least compared multiple closed-source benchmarks. If that proved to be inappropriately favoring a vendor, then, OK, start calling 'conspiracy', but this just sounds like an error in a tool that was never validated.

Re:Compiler Optimization? (1)

Kegetys (659066) | more than 5 years ago | (#24433031)

If this is the case then wouldn't changing the cpuid give you better "real-life" performance too because similarily compiled (non-benchmark) apps would also be utilising the optimisations? TFA doesn't seem to mention testing that...

Re:Compiler Optimization? (2, Insightful)

poot_rootbeer (188613) | more than 5 years ago | (#24433421)

It sounds to me like this could possibly be explained by some kind of conditional optimization that the compiler puts in for various chips, to take advantage of differences in their designs that can improve performance.

I suppose it's possible that a VIA chip running code optimized for what the benchmark believes is an Intel CPU might perform better than the same chip running the benchmark's unoptimized code path, but as I understand it the VIA Nano is pretty entry-level; any optimizations present in it should probably be available to most CPUs across the market.

Perhaps the unoptimized code is exceptionally sub-optimal, which is itself a way to skew results to make the Genuine* results look better.

Do I understand this correctly? (1, Interesting)

RandoX (828285) | more than 5 years ago | (#24432317)

Is this like changing the user agent in a browser?

Re:Do I understand this correctly? (4, Informative)

chalkyj (927554) | more than 5 years ago | (#24432347)

Is this like changing the user agent in a browser?

Pretty much, yes.

Re:Do I understand this correctly? (0)

Anonymous Coward | more than 5 years ago | (#24433191)

Just like putting too much water in a balloon!

Possible semi-benign explaination? (5, Insightful)

davidwr (791652) | more than 5 years ago | (#24432345)

This definitely requires clarification from the creator of the benchmark.

It is possible that the benchmark uses the CPUID to change how the benchmark works, for example, to work around known flaws in a given chip. If this is the case, then the problem is not "omyghoshitplaysfavorites" but rather lack of full disclosure that the benchmarks are not directly comparable across different chips. In the most benign scenario, this could be someone at the benchmark creator's shop forgetting to tell the documentation team. This is still a very serious issue, but it's not fraud.

Re:Possible semi-benign explaination? (2, Interesting)

SimonGhent (57578) | more than 5 years ago | (#24432453)

That's an insightful explanation, but IMO the benchmark is then only valid is the operating systems people use make the same allowances for the different chips.

One thing that doesn't seen to have been investigated is the permutations of the test - marking an Intel chip as a VIA, etc. If the differences are the same drop in performance as the improvement in marking a VIA as an Intel then your explanation has effectively been disproved.

Re:Possible semi-benign explaination? (1)

DJProtoss (589443) | more than 5 years ago | (#24432559)

Unfortunatly, intel/amd chips don't let you set cpuid

Re:Possible semi-benign explaination? (1)

moteyalpha (1228680) | more than 5 years ago | (#24432797)

I am pretty sure that I can go into debug and switch it in the code. I assume that would be considered committing a crime and so if I find the time to do that, I would post it anonymously. It seems that they could easily do that with the source and I agree with others that it does put their benchmark in a bad light.

Re:Possible semi-benign explaination? (1)

MBCook (132727) | more than 5 years ago | (#24432603)

Try reading the article. You can't do that. Only VIA lets you change what the CPU reports. Intel and AMD lock it down so it can't be changed making those tests impossible to run.

Re:Possible semi-benign explaination? (1)

Loibisch (964797) | more than 5 years ago | (#24432641)

They probably just use code compiled with optimizations for each given chip family...if they didn't then people would be shouting how not using special features of a certain cpu would be unfair, etc.

So what now if Intel was the biggest desktop CPU manufacturer in the market (I know it's a stretch, but bear with me for a minute) and profiles for their CPUs (either through ICC or maybe even GCC) were just better optimized because more people put time and effort into them?

I can certainly see this being true for VIA...not so very much with AMD though...but it's still a possible explanation.

And as other people hav already noted this is pretty much saying nothing without testing all the other permutations.

Re:Possible semi-benign explaination? (3, Insightful)

MadKeithV (102058) | more than 5 years ago | (#24433289)

But nothing changes in the CPU operation if you only change the reported CPUID. In the best case that means the 3DMark developers have spent a lot more optimizing for Intel specifically, applying a number of techniques that would have been just as valid for an AMD or VIA processor. They spent that effort without bothering to check the effect on the AMD and VIA specific paths, thus they did not get the same treatment as intel.
And that's simply Intel favoritism.

Re:Possible semi-benign explaination? (1)

MadKeithV (102058) | more than 5 years ago | (#24433325)

Reply to myself: another possibility is that the calculation results with GenuineIntel turned on in a VIA processor are actually wrong. Maybe the VIA doesn't support those instructions after all, defaulting to a no-operation, which is of course pretty fast.
It will be interesting to see how this pans out. It FutureMark playing favourites with Intel? Or did they actually try to avoid some substandard implementations or bugs in the VIA and AMD processors?

The obvious answer (-1, Redundant)

Anonymous Coward | more than 5 years ago | (#24432371)

Intel bribed them.

The more obvious answer (2, Interesting)

WindBourne (631190) | more than 5 years ago | (#24432977)

The benchmarks is looking at the ID and making assumptions. These benchmarks run on Windows. So another possibility is that MS does an optimization that few know about. Any of these are plausible. Simplest answer should rule until proof shown otherwise; Bad assumptions made in OS or program.

MMX/SMD Extensions (4, Insightful)

Cassini2 (956052) | more than 5 years ago | (#24432377)

Could it be that FutureMark uses the GenuineIntel and AMD flags to enable processor specific extensions? and then does a whole bunch of math with those extensions and never bothers to check the result?

This would indicate some really terrible code on FutureMarks part, and VIA should be flagging those op-codes as illegal op-codes, but it might be possible that something like this could happen. It is even possible that the CPUID checks are duplicated in some library somewhere that actually gets the correct code sequence right, and the main FutureMark code disables the advanced functions of the library whenever the GenuineIntel and AMD flags are missing. Thus FutureMark may feature both code sequences that work and those that don't, and the resulting incompatibilities are what causes the issues.

Why are they using a benchmark they can't read? (4, Insightful)

Ed Avis (5917) | more than 5 years ago | (#24432379)

Why would you even consider running a benchmark program you don't have source code for and cannot compile yourself? (If you are worried about random compiler differences messing up the results, you can check an MD5 sum of the final binary against the published one, but it is important that you can reproduce the binary from source and you can read the source to find out what it does.)

If compilers like ICC cripple their code depending on CPUID, that will just lead all manufacturers to set CPUID to GenuineIntel, just as moronic websites (with help from Microsoft) ensured that all browsers call themselves 'Mozilla'.

Re:Why are they using a benchmark they can't read? (1, Insightful)

Anonymous Coward | more than 5 years ago | (#24432471)

quit talking about shit you don't understand.

Two different compilers will almost always produce two different executables. They're just different, that's just how it is. So an md5 comparison will always fail.

Benchmark (3, Insightful)

Rinisari (521266) | more than 5 years ago | (#24432389)

Well, PC Mark 2005 is no longer good for testing processors against processors of another maker, i.e. only good for intra-AMD, etc.

Additional instructions (0)

Dan East (318230) | more than 5 years ago | (#24432395)

Could the difference be that the benchmark program is utilizing additional processor instructions typically found only on those types of processors? The VIA's CPU obviously supports those instructions, but perhaps the typical generic CPU does not.

Re:Additional instructions (4, Informative)

CTho9305 (264265) | more than 5 years ago | (#24432497)

The CPUID instruction provides feature bits that software should use to determine which instructions are available. Using the vendor string is not a reasonable way of detecting the presence/absence of instruction set extensions like SSE.

Re:Additional instructions (0)

Anonymous Coward | more than 5 years ago | (#24432625)

Querying for vendor id would be a really stupid thing to do when you can query for specific extensions (such as SSE3, HT,etc) as easily.

Re:Additional instructions (0)

Anonymous Coward | more than 5 years ago | (#24432681)

Why is it obvious that VIA's CPU supports those additional processor instructions? Does the PCMark memory benchmark also check for correctness? If not, maybe VIA's CPU interprets unknown instructions as NOP, which is why it runs so much faster.

That should be AuthenticAMD... (4, Informative)

LiquidFire_HK (952632) | more than 5 years ago | (#24432427)

That should be AuthenticAMD, not GenuineAMD.

But that would be expecting editors to actually, you know, edit.

Strange thing... (1)

TheDarkMaster (1292526) | more than 5 years ago | (#24432429)

I will not say anything about possibilities here without my anti-conspiracy-haters-shield online (needs a lot of power), but is really strange for a benchmark (supposed to be neutral) Well, I do not really expect neutrality for a benchmark with sponsorship (or partnership?) from hardware makers like nVidia.

GenuineAMD? (1, Redundant)

Naqamel (1138771) | more than 5 years ago | (#24432477)

I think you mean 'AuthenticAMD'.

Processor Optimizations (-1, Redundant)

Anonymous Coward | more than 5 years ago | (#24432491)

Yes, because AMD specific optimizations run so fast on an intel chip....

I think this is less a conspiracy and more that the software is using optimizations specific to the CPU returned by the CPUID. Perhaps someone could do the same test with and AMD chip reporting its CPUID as an intel chip?

Anyone remember the old distributed.net client that had multiple cores for each different processor type if it could guess correctly for your processor type?

All about money (1)

ohxten (1248800) | more than 5 years ago | (#24432561)

It's all about money, ain't a damn thing funny.

Numerology. (3, Funny)

cp.tar (871488) | more than 5 years ago | (#24432563)

V+I+A == 224
G+e+n+u+i+n+e == 715;
Genuine+A+M+D == 925
Genuine+I+n+t+e+l == 1223

The bigger the number, the faster the processor. And you get 20% extra when you pass 1000.

Re:Numerology. (-1, Flamebait)

Anonymous Coward | more than 5 years ago | (#24432831)

Good work, Corporal Tard. Very insightful. Keep it up.

I'm not Hungarian (0)

krkhan (1071096) | more than 5 years ago | (#24432645)

A VIA Nano CPU has had its CPUID changed from the original VIA to fake genuine_amd and genuine_intel. An improvement of, respectively, 10% and 47% of the score was seen. The reasons of this behavior of future_mark product are not yet known.

There, fixed that.

Benchmarking SW must be open source from now on (0, Troll)

denis-The-menace (471988) | more than 5 years ago | (#24432697)

The only way the maker of PCMark can EVER get their credibility back is if their future releases are open source.

Re:Benchmarking SW must be open source from now on (1)

Creepy Crawler (680178) | more than 5 years ago | (#24432991)

And why do we care about some e-penis benchmark?

If it's fast enough to play h.264 at nice high resolutions, and plays some of the fun games out, that's all I care about.

If I need CPU power, I'd go get 4 quad cores on a server with big memory and big hd. A 2d gfx card would be god enough for that server... See if they can do passive cooling on that gfx card, I bet not. And if I needed more cpu still, I'd use a dual backplane mobo with a bunch of ATI/AMD graphics cards with AIXGL (or whatever the acronym) and write applications for the GPUs.

Re:Benchmarking SW must be open source from now on (1)

clickclickdrone (964164) | more than 5 years ago | (#24433299)

>And why do we care about some e-penis benchmark?
Agreed. Back in the day when overclocking or m/board chipsets made a tangible difference in a world where PC power trailed software requirements, benchmarks were a useful way of ensuring you were wringing the max out of your hardware. These days, almost everything is fast enough and unless you're playing frame rate willy waving on Crysis or whatever, it's really of no real interest. The broad brush approaches of CPU speed and/or number of CPUs are all you need to worry about e.g. Word processing/Internet stuff? Anything will do. Video work? fast CPU/twin/quad CPU. Games? Max it all out. worrying about if an AMD X2 6000 beats an Intel Core Duo2 whatever is really no longer of any real value.

CPUIDs (4, Interesting)

bestinshow (985111) | more than 5 years ago | (#24432925)

VIA's is "CentaurHauls"
AMD's is "AuthenticAMD"
Intel's is "GenuineIntel"

There's no "VIA" nor is there "GenuineAMD".

Clearly PCMark2005 is buggy (at the best) and cannot be used to compare different CPU families in this test. At the worst it is intentionally flawed, and shouldn't be used at all.

It's a shame that not one VIA Nano review benchmarked the built-in Padlock functionality. Not one OpenSSL benchmark.

So? (1, Redundant)

mandark1967 (630856) | more than 5 years ago | (#24432931)

This isn't the first time they've been caught doing something "odd" with their code and it likely won't be the last.

That said, keep in mind it's a 3 year old benchmark. Whatever relevance this benchmarking program has today is far more lessened by its age than by any results shown from this research. Don't get me wrong. I'm not defending Futuremark at all. I don't particularly like their suite of benchmarking tools, and not just because of the "odd" results.

How well a platform scores in Futuremark is less relevant than how well it plays your games or movies or compiles your code or rips your movies/CDs. It's my humble belief that a proper benchmark of a system is how well it will perform in the role you want to use the computer.

If I can play GRID at 1920x1200 at the maximum settings possible with playable frame rates I'm happy.
If I can play Crysis at the same resolution and settings, cool.
If AOC runs well at those settings, then I built a nice system.

If Futuremark runs well...so?

Am I missing something? (3, Insightful)

clickclickdrone (964164) | more than 5 years ago | (#24432955)

It's a 3+ year old benchmark being let loose on 2008 vintage CPU's and making mistakes on it's optimisations. I wouldn't expect anything else. It's going to have a 3 year old view on the kind of things these CPUs can do and will act accordingly.

Do your own benchmarks (0)

Anonymous Coward | more than 5 years ago | (#24432979)

That's why I never use any of those becnhmark software. I run my own programs, say Crysis demo for FPS, or rending a specific image for render time, then look at results before and after upgrades and make my own mind.

Re:Do your own benchmarks (1)

AndrewNeo (979708) | more than 5 years ago | (#24433357)

Crysis demo for render time, or rending a specific image for FPS

Fixed that for you.

It's not GenuineAMD... (0)

Anonymous Coward | more than 5 years ago | (#24433039)

but AuthenticAMD!

So umm (0)

Anonymous Coward | more than 5 years ago | (#24433049)

wut is a benchmark

Careful fraud could be much harder to spot (2, Insightful)

tucuxi (1146347) | more than 5 years ago | (#24433143)

If I were an evil fraudster at PCMark, paid for Intel to deliver worse scores to rivals, I would make sure that these rivals had no easy way of uncovering the fraud. Testing for an ID looks much more like bad code paths than like "sneaky fraud".

There is no shortage of alternative quirks that can be used to see whether a given processor belongs to one family or another. Should enough of these quirks be combined, it would be *very* hard to discover an evil-related cause.

Of course, choosing the 'bad' path given an ID may just be blatant enough to provide plausible deniability for the developers that "messed up". However, being a firm proponent of Hanlon's Razor, I would rather call it a bug than a "sponsored feature".

On the other hand, kudos to the guys at Ars who thought of changing the ID and, when the numbers did not add up, make further tests to nail down the argument. Instead of just forgetting about the problem and performing a "review as usual", which would have doubtlessly required less effort. Yay for inquisitive hacker - reviewers.

47%? (1)

Jager Dave (1238106) | more than 5 years ago | (#24433213)

So why don't Intel's chips cost 47% more than VIA's?

Oh...wait....nevermind.

Look at Intel's track record (1)

pak9rabid (1011935) | more than 5 years ago | (#24433237)

Given Intel's track record involving anti-competitive practices, I have no doubt in my mind that Intel paid off PCMark.

And here's the code: (2, Funny)

Ihlosi (895663) | more than 5 years ago | (#24433311)

if(cpuid == "GenuineIntel")
{
Run_really_fast();
}
else if(cpuid == "AuthenticAMD")
{
Run_no_so_fast();
}
else
{
Run_slow();
}

There are lies, damn lies, (1)

spyrochaete (707033) | more than 5 years ago | (#24433437)

... and synthetic benchmarks.

tro4ll (-1, Troll)

Anonymous Coward | more than 5 years ago | (#24433557)

FutureMark's Website (1)

achenaar (934663) | more than 5 years ago | (#24433641)

I just hopped over to FutureMark's website and in the community section there's a list of "Most Popular Processors in 3DMark Vantage (Last 7 Days)"
Could be a coincidence but at the moment, they're all Intel [futuremark.com] .

buggy instruction set in earlier via cpus? (0)

Anonymous Coward | more than 5 years ago | (#24433657)

iirc, via cpus have been buggy even in their 686 instruction set implementation (ubuntu-686 kernel chrashing on via cpus). There might be something similar here.
Load More Comments
Slashdot Account

Need an Account?

Forgot your password?

Don't worry, we never post anything without your permission.

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>
Create a Slashdot Account

Loading...