Beta

Slashdot: News for Nerds

×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

New Analysis Casts Doubt On Intel's Smartphone Performance vs. ARM Devices

Soulskill posted 1 year,17 days | from the keep-on-marking-those-benches dept.

Cellphones 94

MojoKid writes "A few weeks ago, the analyst company ABI Research published a report claiming that Intel's new CloverTrail+ platform (dual-core Medfield) for smartphones was significantly faster and more power efficient than anything ARM's various partners were shipping. If you follow the smartphone market, that was a very surprising claim. Medfield was a decent midrange platform when it launched in 2012, but Intel made it clear that its goal for Medfield was to compete with other platforms in its division — not seize the performance crown outright. Further investigation by other analysts has blown serious holes in the ABI Research report. Not only does it focus on a single, highly questionable benchmark (AnTuTu), the x86 version of that benchmark is running different code than the ARM flavors. Furthermore, the recently released Version 3.3 of the test is much faster on Intel hardware than on any of the other platforms. But even with those caveats in place, the ABI Research report is bad science. Single-source performance comparisons almost inevitably are."

cancel ×

94 comments

Queue (0)

Anonymous Coward | 1 year,17 days | (#44262197)

The arm fanboys

Re:Queue (4, Funny)

K. S. Kyosuke (729550) | 1 year,17 days | (#44262275)

Why would you have to queue them? I'm sure each of them has his own device, it's not like they have to contend for them with each other.

Re:Queue (-1)

Anonymous Coward | 1 year,17 days | (#44262361)

You misunderstand. He wants the arm fanboys to queue up so he can suck them off. He only has one mouth, you know. Afterwards, he'll queue up the intel fanboys so he can suck them off as well.

Re:Queue (1)

Anonymous Coward | 1 year,17 days | (#44262605)

So Intel comes second once again.

Re: Queue (0)

Anonymous Coward | 1 year,17 days | (#44264297)

This is the funniest post I've ever seen on /. Made my day. Thanks.

Re:Queue (0)

Anonymous Coward | 1 year,17 days | (#44262435)

Perhaps you mean "cue"? As in "to give a signal or prompt"?

Re:Queue (4, Insightful)

ShanghaiBill (739463) | 1 year,17 days | (#44263195)

The arm fanboys

We are not "ARM fanboys". We are "Intel-haters". There is a difference.

It just kind of seems DOA (-1)

Andrio (2580551) | 1 year,17 days | (#44262211)

Total number of iOS apps: 900,000
Total number of Android apps: 782,000
Total number of WP apps: 150,000

So what do all of these nearly 2 billion total apps have in common? They're all compiled for ARM, not x86.

Re:It just kind of seems DOA (1)

Anonymous Coward | 1 year,17 days | (#44262245)

Android apps (if they're made correctly,) run on Dalvik, which is platform indapendent. The virtual machine is compiled for ARM & x86 both, and it in turn runs the android apps.

Re:It just kind of seems DOA (4, Insightful)

h4rr4r (612664) | 1 year,17 days | (#44262269)

Not if you use the NDK, which most games and video applications will use for performance reasons.

You can of course compile for both.

Re:It just kind of seems DOA (0)

Anonymous Coward | 1 year,17 days | (#44262365)

I know of the NDK, which is why I stated "if they're made correctly." Google put out documentation very clearly stating the disadvantages of doing NDK, and plenty of creators ignored the best practice and did it anyway; which is the WRONG way to do it.

Re:It just kind of seems DOA (1)

h4rr4r (612664) | 1 year,17 days | (#44262393)

If that is the only way to get the performance you need, it is the right way.

Using the NDK is not wrong, it is just an option. The play store should really demand x86 and ARM executable for all apps.

Re:It just kind of seems DOA (1)

evilviper (135110) | 1 year,17 days | (#44263027)

The play store should really demand x86 and ARM executable for all apps.

And why not MIPS? It's a pretty popular CPU in China, and has certainly been used in Android based phones and tablets.

PowerPC is a pretty good-performing mobile CPU, too. And I'm sure SuperH CPUs still have their adherents somewhere.

Re:It just kind of seems DOA (1)

h4rr4r (612664) | 1 year,17 days | (#44263271)

Personally I think the best case would be the code is uploaded not any binaries. That way the store can make binaries as needed for new architectures.

Re:It just kind of seems DOA (1)

Bert64 (520050) | 1 year,17 days | (#44265131)

I always thought MIPS would be a better choice than ARM for low power servers... MIPS already has a 64bit variant which is well tried and tested.

Re:It just kind of seems DOA (1)

Anonymous Coward | 1 year,17 days | (#44262457)

So how do you propose writing code which is portable between Android and iPhone?
I'm REALLY curious about this one...

Re:It just kind of seems DOA (1)

PRMan (959735) | 1 year,17 days | (#44262933)

Phonegap [phonegap.com]

Re:It just kind of seems DOA (5, Insightful)

phantomfive (622387) | 1 year,17 days | (#44262587)

If you write your code in C, you can port it relatively easy to iPhone, Android, WP8, and Blackberry (depending how much UI code you have). If you write it in Java, avoiding the NDK, you have to do two-four times as much work to port it.

Which would you rather do, use the NDK and recompile, or write once for each platform? "The right way" isn't always a single choice, it's usually a compromise.....

If Intel processors become popular in Android phones, Google will probably introduce a multiple-architecture executable format, much like iPhone does with FAT and MACH (currently around 70% of apps for the iPhone have two architectures, one for ARM7 and one for ARM7s).

Re:It just kind of seems DOA (0)

Anonymous Coward | 1 year,17 days | (#44262609)

Every technology has advantages and disadvantages. NDK's no exception. Downside: has to be compiled on a per-architecture basis. Upside: Improved performance, and the possibility of a lot of your codebase compiling on other, non-Android devices.

Re:It just kind of seems DOA (1)

gman003 (1693318) | 1 year,17 days | (#44262449)

The last Atom/Android phone I read about had an ARM emulator for running NDK apps. Performance suffers, but it's better than not running at all (particularly since Atom is more powerful than the common Cortex-A9 cores, so it has some performance to burn).

ndk has x86 support (1)

gl4ss (559668) | 1 year,17 days | (#44264723)

Not if you use the NDK, which most games and video applications will use for performance reasons.

You can of course compile for both.

extend the if done correctly to enabling x86 compiling for the native parts.
I really doubt that there's that many apps there that have hand crafted assembly in them...

though this myth that it doesn't have x86 support might have been pushed forward by the action from intel of writing an arm translator for those apps that haven't been ndk compiled for x86.

Re:It just kind of seems DOA (1)

h4rr4r (612664) | 1 year,17 days | (#44262253)

Android supports compiling for x86.
So not all of those are ARM exclusive.

Re:It just kind of seems DOA (1)

SuperKendall (25149) | 1 year,17 days | (#44262335)

Every iOS app was compiled for x86 also (iOS simulator), but it doesn't mean any games ship with that binary.

Just because you can easily cross compile does not mean a platform will not have a huge hurdle out of the gate just getting someone to re-compile a build.

Re:It just kind of seems DOA (2)

JDG1980 (2438906) | 1 year,17 days | (#44262381)

App stores make this less of an issue. If Apple wants to switch the iDevices to x86, they'll tell developers that they must have an x86 build posted by such-and-such date, or their app will be dropped from the store.

Which they do not do (5, Informative)

SuperKendall (25149) | 1 year,17 days | (#44262657)

App stores make this less of an issue. If Apple wants to switch the iDevices to x86, they'll tell developers that they must have an x86 build posted by such-and-such date.

Apple DOES NOT do that for older apps. It will often place restrictions on app updates or new app submissions (like must include iPhone5 screen shots) but they don't ever force older apps to do any kind of update - nor should they.

both binaries can be included (1)

gl4ss (559668) | 1 year,17 days | (#44265129)

Every iOS app was compiled for x86 also (iOS simulator), but it doesn't mean any games ship with that binary.

Just because you can easily cross compile does not mean a platform will not have a huge hurdle out of the gate just getting someone to re-compile a build.

if you enable the x86 portion then the app ships with both x86 and arm ndk compiled binary parts... and mips too.

all major apps would have that shit working instantly if they got even 10% of the android share.. or sooner, if it makes a good showcase.

Re:both binaries can be included (1)

SuperKendall (25149) | 1 year,17 days | (#44266975)

if you enable the x86 portion then the app ships with both x86 and arm ndk compiled binary parts... and mips too.

But it requires every app to re-compile, so it's not like you can launch with a full selection.

all major apps would have that shit working instantly if they got even 10% of the android share

No way, it would take a lot of testing and even for a 10% share a lot of companies would approach it cautiously. It would take years to get 50% of the app market onto a new platform even IF (Huge, huge IF) you could simply re-compile. Any app that used the NDK (as in: every single game of any quality) would need more done than a simple re-compile, at the very least the game engines have to be ported to the thing first.

Re:It just kind of seems DOA (0)

Anonymous Coward | 1 year,17 days | (#44262259)

Well, most android apps aren't compiled at all. Java JIT remember?

Re:It just kind of seems DOA (0)

Anonymous Coward | 1 year,17 days | (#44262723)

Bytecode, remember?

Re:It just kind of seems DOA (0)

Anonymous Coward | 1 year,17 days | (#44262329)

interesting though that you included WP apps in the list, since surely there are far more x86 apps that could be opened up to that platform.

Apple too might benefit from a shared architecture between the desktop and phone, though honestly if they said tomorrow "We're switching to SPARC processors, everyone re-write your apps" Everyone would giddily proclaim the dawn of the new sparc era and get to work. (Oh, how I wish. It's my 2nd favorite processor behind the 680x0)

Re:It just kind of seems DOA (1)

BasilBrush (643681) | 1 year,17 days | (#44262379)

Apple too might benefit from a shared architecture between the desktop and phone

It'd have to be a hell of a lot more than a "might benefit" for them to do it. It'd have to be the inability of one of the platforms to support future requirements in their existing niche. Which is a very far distant prospect in each case.

Re:It just kind of seems DOA (0)

Anonymous Coward | 1 year,17 days | (#44262399)

2 billion?

Re:It just kind of seems DOA (1)

Anonymous Coward | 1 year,17 days | (#44262543)

2 billion?

For small values of 2 billion.

Re:It just kind of seems DOA (1)

james_shoemaker (12459) | 1 year,17 days | (#44262515)

Most android apps are compiled for dalvik and don't care what processor is running dalvik.

Re:It just kind of seems DOA (1)

obarthelemy (160321) | 1 year,17 days | (#44263251)

Actually, no: a large plurality of Android apps are coded to Dalvik, which is basically (pun intended) java bytecode. Very few Android apps use the alternative (Native something something), which allows ASM or whatever, and then only for small snippets. That's why Android apps run indifferently on ARM, x86, and MIPS... various versions of those actually.

Optimization is another matter, but that's mostly fro GPU stuff.

Re:It just kind of seems DOA (1)

Marxist Hacker 42 (638312) | 1 year,17 days | (#44267305)

Java Android apks run just fine on Medfield.

Clearly this can't be true (2)

NoNonAlphaCharsHere (2201864) | 1 year,17 days | (#44262297)

Intel cooking the books on a benchmark? Never happen.

Re:Clearly this can't be true (1)

djohnsto (133220) | 1 year,17 days | (#44262377)

Where, in any of the linked articles, was any data provided by Intel?

Re:Clearly this can't be true (2)

BasilBrush (643681) | 1 year,17 days | (#44262443)

Whenever an analyst produces a dishonest research report, look to the company that benefited from the dishonesty for the source of funding of the report.

Re:Clearly this can't be true (4, Insightful)

NoNonAlphaCharsHere (2201864) | 1 year,17 days | (#44262523)

Intel has a VERY long history of questionable ;) benchmarks, all the way to tweaking processor designs to run benchmark code faster. Microsoft's "Get The Facts" propaganda is just a pale imitation of Intel's history.

Re:Clearly this can't be true (2)

djohnsto (133220) | 1 year,17 days | (#44262937)

Intel has a VERY long history of questionable ;) benchmarks, all the way to tweaking processor designs to run benchmark code faster. Microsoft's "Get The Facts" propaganda is just a pale imitation of Intel's history.

Supposedly, benchmarks are written to simulate real workloads. It seems to me that tweaking processor designs to run benchmarks faster is a good idea. If you have a better idea for what applications to design a processor for perhaps you should join a processor company in the workload analysis or planning group.

I agree that Intel, or any company with enough "muscle" (see Nvidia, ATI/AMD, MSFT, IBM, HP, Oracle, etc.), will try to influence the press and show their products in the best possible way, even dishonestly on occasion. I'm not saying that the fact that "everyone does it" makes it right, just that everyone does do it.

I also think that these companies receive a lot of blame for articles / "research reports" like this one that they have absolutely no involvement with. To think that EVERY positive Intel article is funded or influenced by Intel would:

1) Ignore the fact that Intel does make fairly good products most of the time
2) Greatly overestimate Intel's marketing prowess. :)

Re:Clearly this can't be true (1)

DigitAl56K (805623) | 1 year,17 days | (#44264447)

Supposedly, benchmarks are written to simulate real workloads. It seems to me that tweaking processor designs to run benchmarks faster is a good idea.

That may be true, *if* you don't break the benchmark in the process. This link, shared by another posted further down the /. thread, accuses AnTuTu of exactly that:

http://forums.anandtech.com/showthread.php?t=2330027 [anandtech.com]

In summary, while the compiled ARM code performed a series of bitwise operations as written in order to excercise the CPU instructions, the Intel compiler seems to have applied some compile-time smarts to effectively bypass a lot of the work but achieve the same end result. In a real-world program that's good, because you speed the program up by cutting down on the work the processor has to do, but in a benchmark that's supposed to make the processor perform all those individual operations it's bad and gives misleading results your readers end up unwittingly comparing apples to oranges but assuming that they're comparing apples to apples.

Essentially, "Which CPU is faster and/or more efficient? Who knows, they were asked to do different work."

Re:Clearly this can't be true (3, Insightful)

makomk (752139) | 1 year,17 days | (#44265157)

Wow. What's more, apparently that optimisation was added by Intel after the benchmark was developed:

What's more, this optimization wasn't present in ICC until a recent release. Somehow I don't think that they just now discovered it has general purpose value. More likely case is that they discovered is they could manipulate AnTuTu's scores. Seems to coincide well with this third-party report appearing showing how amazing Atom's perf/W is - using nothing but AnTuTu. Or the leaked scores seen for CloverTrail+ and now BayTrail that are AnTuTu. Is this really a coincidence?

So basically they modified their compiler to optimise away the actual benchmark, then got someone to release a third-party report based solely on the benchmark they'd just manipulated the results of.

Re:Clearly this can't be true (0)

Anonymous Coward | 1 year,17 days | (#44266323)

That may be true, *if* you don't break the benchmark in the process.

as long as program gives same result when run and looks the same (or very similar at least - nvidia/amd shader optimisation) its not broken

while the compiled ARM code performed a series of bitwise operations as written in order to excercise the CPU instructions, the Intel compiler seems to have applied some compile-time smarts to effectively bypass a lot of the work but achieve the same end result.

exactly that is what good compiler does, if you use intel compiler and libraries to compile your program (street price $3,499) it will detect a lot of unnecessary parts of program and most programs will run faster, intel does have best (from performance standpoint) compilers in industry.
I would agree with you its unfair if intel compiler accelerated only benchmark and was useless on real-world apps, but whatever you compile on intel compiler including almoust any kind of app ( GPU limited apps are exception ) will run faster

so intel made their CPU faster by providing compiler that optimizes additionally apps only for their processor, nothing is stopping ARM or Apple to provide their own even better compiler that optimizes only for arm or apple processors (except few trillion dollars in R&D money intel has as advantage but that should not be something we care about, results are important, even if its David (arm) vs Goliath (intel) situation)

Essentially, "Which CPU is faster and/or more efficient? Who knows, they were asked to do different work."

as long as result they give is same it can be easily answered, and nothing is stopping any of contenders to optimise more their version of compiler to make apps faster (as long as it makes all apps faster like intel and not only benchmark)

Re: Clearly this can't be true (1)

DigitAl56K (805623) | 1 year,17 days | (#44267143)

The interpretation of whether the benchmark is broken or not depends on what the benchmark was trying to ascertain. If it was trying to ascertain which CPU could process instructions fastest and most efficiently then it could be argued that it was broken because the result is heavily influenced by the logic of the compiler rather than the performance of the CPU. If it was trying to ascertain which platform had a better result allowing for toolchain-based optimizations, then you could argue the benchmark was fair *except* if this benchmark had specific optimizations applied to it that are not representative of what could be expected in a typical setting, and if that was not declared, because that would set the wrong expectations for readers, and it seems that is part of the allegation.

Re:Clearly this can't be true (1)

bd32322 (571576) | 1 year,17 days | (#44263937)

This is ABI Research (not affiliated with Intel). http://en.wikipedia.org/wiki/ABI_Research [wikipedia.org]

So in this particular case Intel is not reporting wild benchmarking results

Re:Clearly this can't be true (2)

Synerg1y (2169962) | 1 year,17 days | (#44262461)

In addition, how much did Intel pay ABI for the report I wonder?

With Intel wanting to get a bigger share in the mobile market as bad as they do (they restructured around it), paying for a favorable research report doesn't sound all that out of scope.

Re:Clearly this can't be true (1)

Anonymous Coward | 1 year,17 days | (#44262875)

The intel code was just optimized with a smarter icc compiler that unrolled a loop doing trivial bitops, which is a fairly standard thing for compilers to do. If this breaks the benchmark because optimization wasn't in the spirit of what it was trying to measure, fix the benchmark.

Re:Clearly this can't be true (1)

makomk (752139) | 1 year,17 days | (#44265237)

It was so standard that the ICC compiler apparently didn't bother to do it until after the benchmark was released, probably because it's unusual for anyone to write code that benefits from that optimisation outside of benchmarks.

Re:Clearly this can't be true (1)

Technician (215283) | 1 year,17 days | (#44262941)

The article is on face value self admitted FUD. If, maybe, possibly. Due to the lack of hard facts, the article is raising concerns of what might be happening.

Has anyone tried to use one of the phones hands on? I had the oppertunity to try one of the earlier Intel phones, but not the new one, so I don't have a comparison between the older Intel phone and the newer one. A benchmark between the new and older model would be interesting. I know how the older Intel phone works. It worked very well. The only issue of concern with the older phone was the limitations on the radio. Since Intel purchased the portfloio from Motorolla, I expect the big improvements will be in the radio technology. With SOC designs, benchmarking end user app space is tricky at best as software modem, graphics, etc are all sharing processor cycles.

Overall, I really liked the Intel Phone.

The big expected performance gains are in battery life as much physical hardware is replaced with software on a processor that has the headroom to do the processing on less power.

Refrence Intel buys Motorola Mobility http://www.intc.com/releasedetail.cfm?ReleaseID=638502 [intc.com]

Clover Trail vs Medfield (5, Interesting)

Anonymous Coward | 1 year,17 days | (#44262303)

I design chips and work for intel.

Clover Trail is significantly more power efficient than Medfield because it has a lot more power control stuff in it, so more of it is turned off most of the time. This is not a big secret as far as I know.

The rest of the phone consumes power too, particularly the screen. So YMMV.

Surprised? (1)

willthiswork89 (2885827) | 1 year,17 days | (#44262339)

ARM has been focusing on mobile platform architecture for much longer than intel, its like Honda trying to make a truck, sure they did do it but its nothing like a company making nothing but trucks from the get-go. Intel needs to stay right where they are king and keep that crown and stop trying to follow rabbit holes thinking they might find money where ARM is the clear leader down in those places.

Re:Surprised? (1)

tlhIngan (30335) | 1 year,17 days | (#44262509)

ARM has been focusing on mobile platform architecture for much longer than intel, its like Honda trying to make a truck, sure they did do it but its nothing like a company making nothing but trucks from the get-go. Intel needs to stay right where they are king and keep that crown and stop trying to follow rabbit holes thinking they might find money where ARM is the clear leader down in those places.

Problem is, Intel's kingdom is shrinking [arstechnica.com] and while it's the bread and butter, it won't be large enough to sustain them in the long term.

Like buggy whip manufacturers, Intel's having to evolve ,and that's what they're doing. There's tons of crap that's wrong with x86, and perhaps Intel's stubbornness to the architecture is the wrong approach, but they have to try something.

Right now, Intel's trying to shoehorn x86 into ARM's territory. Will they be successful? So far, not so much, but who knows?

And a little competition for ARM can't be too bad a thing. Heck, ARM's trying to do a bit of the same by going 64-bit and entering the server room.

Re:Surprised? (1)

Anonymous Coward | 1 year,17 days | (#44262751)

Well, Intel tried to go away from x86 once. They probably don't want a second Itanic.

Re:Surprised? (0)

Anonymous Coward | 1 year,17 days | (#44263831)

There's tons of crap that's wrong with x86, and perhaps Intel's stubbornness to the architecture is the wrong approach

Intel has made at least 3 attempts to move away from x86 that I'm aware of, the market kept them at x86.

Once at the move from 16-bit to 32-bit processing - iAPX432. They hedged their bet here and did the 32-bit 386 processor as a fall back.

There was the i860 which was a RISC chip around the time of the Pentium. Both options were on the market and the market chose x86 chips, so what they could they moved to their Pentium chips (SIMD/MMX Instructions)

Once at the move from 32-bit to 64-bit processing - Itanium. They didn't hedge their bet here and were forced to play with AMD's 64-bit implementation.

One might be able to argue their StrongARM line was also an attempt although that was never marketed towards servers / workstations that I'm aware of.

Re:Surprised? (1)

Bert64 (520050) | 1 year,16 days | (#44268691)

x86 is a horrible kludge and always has been, it survives not because it's any good but because there is a large amount of closed source code out there compiled specifically for it.
Trying to shoehorn x86 into another market tho, one that isn't previously locked in to it just seems ridiculous... Even if Intel manage to become competitive with ARM by using more efficient fabbing processes, this is only detrimental to end users as an ARM chip fabbed on the same process would be better still.

Re:Surprised? (1)

marcosdumay (620877) | 1 year,17 days | (#44263885)

Well, the best thing Intel may do to society is just dying... But I see how they'll think differently.

Anyway, one of the basic tenets of evolution is that it can only happen when you change things. Intel is trying to evolve without changing anything... They'll stay dependent of Microsoft, they'll keep the centralized product development, they'll stay compatible with the power wasting x86 applications, and looks like they are keeping the dishonest marketing department.

By the current trends, even if Intel create an entire family of mobile processors better than the ARM ones, I wouldn't be surprized if they still fail to get traction in the mobile devices. (But I'll be quite surprized if they create better mobile processors than ARM.)

Re:Surprised? (1)

gtall (79522) | 1 year,17 days | (#44264855)

It depends upon how their architecture is licensed. ARM sells a licenses and then licenses on steroids. The latter allow you much latitude in how you design your SoC. Intel so far has an "I know best" attitude. The other problem for Intel is their chips are too expensive. To fight on price means their profit margins shrink and that makes chasing ARM less of profitable proposition. On the other hand, Intel might feel their future is threatened by ARM so much that they must go after ARM.

Intel, like Microsoft, defines themselves in killing the opposition, not competing with it. It is a mindset that is not easily given up.

Re:Surprised? (0)

Anonymous Coward | 1 year,17 days | (#44266523)

in the end intel will win i am sure, here arm is mom&pa shop on corner working for over 100 years, always friendly and doing a lot of good for neighbourhood, and intel is k-mart store chain with trillions dollars behind it just opened last month, how many months you give to that mom&pa store?
one in my town lasted 2 months until they got bankrot and got bought by k-mart for extra parking space.
don't underestimate power of money

Re:Surprised? (1)

marcosdumay (620877) | 1 year,16 days | (#44269839)

Yep, the power relation is similar, but ARM was successfull competing with them up to now. In fact, that's the main reason they have a monopoly at mobiles, because Intel killed everybody else... And time is playing at the ARM team, not Intel. With the end of Moore's Law (that's not a certainty yet, but a possilbe outcome for the next generation of fabs), Intel's lead in fab technology becomes less relevant.

Re:Surprised? (0)

Anonymous Coward | 1 year,17 days | (#44266423)

arm might work on low power (mobile) chips much longer than intel, but intel has 100 times more scientists and 1'000 times bigger R&D budget and several years of advantage in silicon technology ( they soon expect 14nm chips on market and other players like arm are not yet at 22nm)

intel will definitely win there is no doubt in that, even if they have morons as employees (and they actually have really smart scientists on their team) they can always buy arm for chump-change arm is small/cheap.

Biased benchmarks endemic in chip industry (4, Insightful)

OneAhead (1495535) | 1 year,17 days | (#44262345)

News at 11.

I don't think there's any chipmaker (CPU, GPU or otherwise) who hasn't been caught doing it. Not that that makes it right, of course.

For the quick readers, note that this is about Clover Trail, not to be confused with the recently announced Bay Trail. Though it does cast doubts on Intel's claims about the latter's performance [extremetech.com] ...

Re:Biased benchmarks endemic in chip industry (2)

evilviper (135110) | 1 year,17 days | (#44262977)

note that this is about Clover Trail, not to be confused with the recently announced Bay Trail

I anxiously await Intel's new "Oregon Trail" CPU.

Re:Biased benchmarks endemic in chip industry (3, Funny)

Guppy (12314) | 1 year,17 days | (#44265425)

I anxiously await Intel's new "Oregon Trail" CPU.

Might be a long wait, I heard a bunch of their engineers mysteriously died of dysentery while working on the design.

Re:Biased benchmarks endemic in chip industry (1)

OneAhead (1495535) | 1 year,17 days | (#44264625)

Here's an excellent analysis of the implications for Bay Trail [seekingalpha.com] and its competitiveness against its big rivals Snapdragon (Qualcomm) and Tegra (Nvidia).

TL;DR version: the latter aren't shaking in their boots yet.

You know how to waste power ! (0)

Anonymous Coward | 1 year,17 days | (#44262363)

Use something other than a well engineered and architected Microsoft compiler

Re:You know how to waste power ! (1)

axonis (640949) | 1 year,17 days | (#44262567)

Even if there waz 100 million + imperative Masters grade programmers, a small declarative based team using top Microsoft tools could wipe the floor with them all and put that social union into a sustained Hyper V partition for beer training, Im sure apple would give up their $150+Mill 90's sub from Microsoft to comply and compete at a soverign software level ;)

Re:You know how to waste power ! (0)

Anonymous Coward | 1 year,17 days | (#44266581)

Intel compilers are even better than Microsoft, best in industry-end of story (as long as you don't try to run program on AMD it makes code that is very fast on intel and intentionally slow on AMD to make people buy more intel chips).

only reason that everyone does not use intel compilers and libraries is that licence for one developer ( 1 year ) is $3'500 (plus $20'000+ licence for using runtime library per each program you compile/sell) and libraries are not open source compatible (you can open your own code but not headers/intel libraries)

Re:You know how to waste power ! (0)

Anonymous Coward | 1 year,17 days | (#44263833)

Exactly! Never mind the development time you'll waste using gdb or printf debugging.

Enlightenment (0)

Anonymous Coward | 1 year,17 days | (#44262453)

Benchmarks and polls have to be funded so the people behind them can feed their children.
That's why results ofthen don't match realiy.

Single-source performance comparisons ....? (1)

HerculesMO (693085) | 1 year,17 days | (#44262513)

Single-source performance comparisons almost inevitably are, yet this point is being made by none other than a single person. I love the irony.

Its. (0)

Anonymous Coward | 1 year,17 days | (#44262583)

Its. As in "its goal", not "it's" as in "It's...Monty Python's Flying Circus!"

Streisand effect in action! (3, Interesting)

Anonymous Coward | 1 year,17 days | (#44262637)

What's really hilarious is that this supposedly "rigged" benchmark got almost zero press (never posted on Slashdot or mentioned on major tech websites) when it first came out. Now the ARM religion has to wage a jihad on anybody who claims that it is physically possible to use silicon that doesn't include royalty payments to the Church of ARM to run your smartphone.

Using my own ARM-based smartphone and the Dexplorer app, I looked at Antutu and other common Android benchmarks. One interesting thing that stood out was that Antutu uses the NDK (i.e. there are C-compiled libraries for both ARM + x86 ABIs). The other benchmarks like quadrant and Linpack were pure java based. Basically what I'm seeing is that the Clovertrail+ x86 hardware absolutely is in the same ballpark as the snapdragon 600 for performance, but that the ARM vendors and Android developers have been optimizing Dalvik for the ARM architecture since 2009 while little has been done for x86. Once a real compiler like GCC gets to generate C-code for the x86 Atoms though (and also for the ARM parts BTW, it's a level playing field), then we see that Clovertrail+ puts up decent competition for modern ARM chips in the same power envelope.

As for the usual: "Intel cheats using compilers!" whine, please take a big dose of vitamin STFU. I've been bored to death for over a decade by the drone of the ARM jihadists who claim that their architecture is so magical that you can literally knock back a fifth of Tequila and vomit up a perfectly optimized web browser before breakfast. If ARM is truly so beautiful and if the engineers at ARM are truly such geniuses, then it should be trivial for them to implement compilers that blow away x86.

Re:Streisand effect in action! (0)

Anonymous Coward | 1 year,17 days | (#44263599)

They both suck. There was a time when ARM had a very nice tight concise instruction set that enabled very compact code (ALU behind the barrel shifter was a cool design). Then it became a mess due to the demands of their customers, much like the x86 instruction set.

Re:Streisand effect in action! (0)

Anonymous Coward | 1 year,17 days | (#44266177)

Yes, us chip designers know this. It's because modern software responds to business demands, not engineering ones and thus is in no way 'clean'. Then we design the chip against those use cases since that's what sells.

Re:Streisand effect in action! (1)

cupnoodleboy (856625) | 1 year,16 days | (#44268615)

You misunderstand what Streisand effect is. No one is trying to delete or remove the report published by ABI Research. If you want to know what Streisand effect really means, a quick trip to wikipedia would show you the common usage of this term. Discussing the merits or shortcomings of such reports is something anyone with a brain would like to do. Only those ignorant or lazy would blindly accept everything they read without questions. Your attempt to defend Intel from accusation of cheating using compiler is misleading. You keep saying that if ARM engineers are good at making compiler, programs would run faster in ARM CPU than in Intel CPU. However, whether ARM engineers are good or bad at making compiler is completely irrelevant to the quesition whether Intel is cheating or not. Appying your way of thinking to the case of a sports race, when one of the athletes in a race is suspected of cheating, your answer would be that the other athletes should run faster than the suspected cheater if they are good athletes. This is ridiculous. Unless you believe cheating is good, as some marketroids do, those who cheats should be exposed and shamed in the public. To find out if cheating has occured or not, the only valid method is to do a technical analysis of the output of the compiler and the benchmark programs, which is what some other analysts are doing. In case you haven't heard it before, there is a well known saying, âoeWhen you point your finger at someone, three are pointing back at youâ. By calling people who supports ARM CPU as "ARM jihadists", you have revealed your bias and prejudice, and it is now reasonable to believe that you are yourself an "Intel jihadists".

Re:Streisand effect in action! (2)

Bert64 (520050) | 1 year,16 days | (#44268719)

When it comes to compiled code the situation is reversed, gcc has been heavily optimized for x86 whereas other architectures although supported have had far less work done on them. There's also Intel's compiler which generally produces faster code than gcc.

What happened to "don't be evil"? (0)

arcite (661011) | 1 year,17 days | (#44262711)

Or right forgot, it was just a marketing slogan.

Re:What happened to "don't be evil"? (1)

Anonymous Psychopath (18031) | 1 year,17 days | (#44262839)

Or right forgot, it was just a marketing slogan.

...for Google, not Intel.

Where to buy? (1)

smprather (941570) | 1 year,17 days | (#44262791)

If you can't buy one of these, who cares?

AnTuTu assembly code analysed (5, Informative)

edxwelch (600979) | 1 year,17 days | (#44263031)

Some guy on the Anandtech forums analysed the AnTuTu code and found that it indeed had been tweeked to favor x86 processors:
http://forums.anandtech.com/showthread.php?t=2330027 [anandtech.com]

hardly, understand it (0)

Anonymous Coward | 1 year,17 days | (#44264767)

"This is what we call breaking the benchmark. Where the compiler applies some logic that makes the benchmark much faster by doing a set of operations that the benchmark identifies as correct (if it even checks) but are not performing the intended function of the benchmark."

Better instruction set in x86 and better compiler. it might not have been intentional. What happens is that when compiled for x86 it produces faster code for what the benchmark actually does. and that it uses vastly less instructions should come as no surprise to anyone!

More than a processor... (0)

Anonymous Coward | 1 year,17 days | (#44264891)

The overall state of the tooling (gcc+arm v. icc+x86) is highly relevant. Intel invests a lot in things like icc in order to make software naturally take best advantage of their design. To dismiss that value would be unfair as well.

In this case, there was a sign tht some recent gcc optimizations were not used, but a follow up said when those were applied, the results in this case didn't vary as much as perhaps hoped.

Re:AnTuTu assembly code analysed (0)

Anonymous Coward | 1 year,17 days | (#44265591)

Thanks for the link. The link basically says AnTuTu is using nbench. nbench is the BYTE benchmark.

The BYTE benchmark is notorious for being sensitive to compiler optimizations. This is not a problem if you are aware that the benchmark is sensitive to compiler optimizations. Once you are aware, you will realize the benchmark is not a hardware test, but rather a test of the compiler's ability to perform those compiler optimizations.

It does not look like AnTuTu portrays their benchmark as a test of the compilers' optimization capabilities, so therefore the results are purposefully misleading.

To claim so otherwise (i.e. Intel is cheating) is missing the point -- the benchmark is not suitable as-is for CPU hardware benchmarking.

Re:AnTuTu assembly code analysed (0)

Anonymous Coward | 1 year,17 days | (#44266487)

That ToggleBitRun function is just so poorly written that if it's representative of what AnTuTu does, it should be barred from being an benchmark to begin with (especially when it's coming up in a performance comparison, meaning it's a significant part of the computation time). The whole "set the entire word optimization" is SO FUNDAMENTAL that EVERY optimized memory management library has a function that does almost exactly the same thing -- an implementation of memset that loop unrolls and sets an entire max address-aligned register width at a time. In this case, we're just dealing with bit granularity instead of byte granularity.

The surprising thing for me is that ICC didn't have this optimization until now.

"New analysis" (0)

Anonymous Coward | 1 year,17 days | (#44263165)

No new analysis has occurred. No additional benchmarks have been run, no new power consumption measurements have been taken. A previous analysis has been, if not discredited, at least had its limitations explicitly described.

hierarchy (0)

Anonymous Coward | 1 year,17 days | (#44263949)

lies damn lies benchmarks manufactures' benchmarks

ALL positive Intel reports are paid for by Intel (0)

Anonymous Coward | 1 year,17 days | (#44265611)

Intel and Microsoft soak all the technical websites with hundreds of millions of dollars every year, buying favourable coverage for their products.

For instance, Anandtech (one of the very worst for taking Intel pay-offs) recently ran a 'gaming' battery-life benchmark that consisted of allowing a free-running GPU benchmark program that works by timing how long it takes to complete a certain number of rendered frames of pseudo game data. Anandtech CLAIMED that their benchmark measured battery life in gaming situations. Of course, the weaker the GPU (and Intel's GPUs tend to be very weak indeed) the less work the GPU does per hour, and thus the less power the GPU benchmark takes.

The bent Anandtech benchmark showed Intel as a big winner, of course. But then again, when AMD launched its first combined CPU+GPU chips in laptops, Anandtech refused to publish the graph showing the gaming life of the device, because that one benchmark doubled Intel's score. However, Anandtech did include 20 other graphs in their review which all showed LESSER performance than Intel.

Most people only read the headlines or skip to the conclusions of the review. Often the body of text is a lot more honest than the weasel words at the top and bottom of the review, allowing the reviewer to ease their conscience somewhat.

Intel chips are a LOT worse than most of you realise, being only for the most part a little better than AMD if cost is excluded as a factor. Factor cost in, and AMD often wins. (To be fair, factor power use in, and AMD tends to comprehensively lose on the mains powered high-end platforms).

Against ARM, and Intel is a total joke. Even if Intel were equal at a tech level (and it is most certainly not- Intel's disastrous dalliance with FinFET is leading them to be crucified by ARM devices using FD-SOI), the Intel Tax will always make Intel an unacceptable platform.

What is the Intel Tax?
1) Massive increase in die area for units that translate the rotten x86 ISA to the internal RISC one.
2) Massive power usage by that translation block
3) Massive IP costs for everything that makes x86 'special' (special in the short-bus sense)
4) Massive coding inefficiency overhead for having to control the true internal RISC ISA with code written to the x86 ISA (usually emitted by a compiler, of course)

ARM misses all of this legacy nonsense. ARM, since version 2 (version one was used in a mains powered desktop computer) has been designed from the ground up for low power use.

Intel has one last, incredibly dishonest, trick up its sleeve- and you are seeing sites like Anandtech exploit this lie. Intel mobile chips are thermally throttled, and this means they can burn desktop like power amounts for very short periods. If a benchmark can be completed in this period (once), the throttling is avoided, the chip can auto-clock to obscene speeds, and the benchmark score will seem amazing.

So Anand at Anandtech is encouraging the use of mobile benchmarks of the shortest possible duration to inflate Intel's mobile scores. Now a game, for instance, needs to be continuously doing work. A game will suffer maximum continuous thermal throttling, and will show the real performance of the system. None of Intel's power saving tricks help them with real apps/games.

In fairness to Anandtech, they run a fascinating comparison of gaming performance across tablets AND older PC system, showing the best ARM tablets as having the gaming performance of high-end PCs from not all that long ago (but today's gaming PCs are vastly more powerful).

Anyway, FD-SOI is ringing the death knell for Intel. Intel bet the farm on FinFET and that bet has not paid off. All Intel can now do is pay millions for good PR, and prey their 2015 or later parts with Nvidia graphics will turn things around. In the mean time, Intel is going to make ZERO worthwhile progress in the mobile space.

Re:ALL positive Intel reports are paid for by Inte (1)

Tough Love (215404) | 1 year,16 days | (#44268895)

1) Massive increase in die area for units that translate the rotten x86 ISA to the internal RISC one.
2) Massive power usage by that translation block
3) Massive IP costs for everything that makes x86 'special' (special in the short-bus sense)
4) Massive coding inefficiency overhead for having to control the true internal RISC ISA with code written to the x86 ISA (usually emitted by a compiler, of course)

5) Massive cost of supporting multiple overlapping compute models such as x87, mmx, sse, sse2, blah, blah, blah...
6) Massive cost to support stupid number of obsolete and legacy instructions that nobody care about but still need to perform well to avoid performance regressions on old binaries

Of course they're running different code. (1)

rsaxvc (2383898) | 1 year,17 days | (#44266771)

I hope this doesn't surprise anyone, but they're running different code. They're also running different instruction sets. And they likely have different memory controllers and caches sizes. I think we can say that Intel is pretty darn good an running AnTuTu, but that's all that graph says.

Re:Of course they're running different code. (0)

Anonymous Coward | 1 year,16 days | (#44269627)

I hope this doesn't surprise anyone, but they're running different code. They're also running different instruction sets. And they likely have different memory controllers and caches sizes. I think we can say that Intel is pretty darn good an running AnTuTu, but that's all that graph says.

the issue isn't that they are "running different code". The issue is they are "doing different things". ARM is actually executing the test while Intel plugs in a number, doesn't run the test, and therefore uses no power during the test...giving Intel a much performance per watt score. Read the Anandtech analysis. This is like a guy that uses a cab to win a foot race.

Great Article (0)

Anonymous Coward | 1 year,16 days | (#44267925)

This was a great article , if you want to read some other great article come at http://easycode.me I read the history of x86 and I really love this processors...

Re:Great Article (0)

Anonymous Coward | 1 year,16 days | (#44267949)

This was a great article , if you want to read some other great article come at http://easycode.me [easycode.me] I read the history of x86 and I really love this processors...

Why are we talking about old tech? (0)

Anonymous Coward | 1 year,16 days | (#44270439)

Clovertrail+ is old tech. It's just a rehash of the old atom line. It was never going to be anything more than a stepping stone, or something to flesh out Intel's product line. It's nothing to be excited about.

Baytrail, with silvermont cores, however is going murder Arm in it's crib if it's even half as good as early reports claim. Intel's latest 22nm fabrication tech. Based off of latest gen haswel "core" cpu tech (So real out of order execution). Real intel graphics, not an awful powerVR IP core (Important if you're going to run windows). Completely designed from the ground up for power efficiency. (All pre-haswell intel cpus have bolt-on, after the fact power managment tech)

Baytrail is supposed to enable sub-200 dollar windows 8 tablets. Not windows RT. Full windows 8.
The silvermont cores will be going in to phone oriented SoCs that should also be impressive.

One of the problems I see with arm SoCs meant for anything other than smartphones is that they're a disorganized mess. Each chip is damn near an architecture of it's own. That's why there is no such thing as a universal android system image. You have to custom compile for EACH DEVICE. For all the problems with x86/x64/PC arch, it sure is easy to throw your windows or linux distro CD in and install a working OS.

Check for New Comments
Slashdot Account

Need an Account?

Forgot your password?

Don't worry, we never post anything without your permission.

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>
Create a Slashdot Account

Loading...