kidjan's Journal: Dual Core Mumbo Jumbo

The other day, I came across a very nasty article which claimed (among other things) that many hardware review sites were biased and bought out by industry.

This came as little surprise to me; however, the reasoning behind all of this had to do with the recent reviews of Intel dual core chips.

The article in question:

http://www.theinquirer.net/?article=22332

Charlie Demerjian states: "To step sideways a bit, the current dual core chips are all going to suck on games regardless of whether they come from Intel or AMD. Both are heat limited and will debut several clock bins below their single core counterparts. The Intel side also takes a step backward in bus speed because of the added loads on the bus. All these are engineering realities, and in no way diminish the really great jobs both companies are doing to bring dual cores to the masses.
It does mean however that until software catches up, most likely not this year, that gaming is going to suck on them. They will cost more, take more power, and be a status symbol for the rich and stupid, but their frame rates will blow dead goats. On multitasking and multithreaded apps, they will shine like the sun, but how many of these are there? How many times do you encode a movie while typing a document, zipping your C drive, doing some heavy CFD work all while listening to a few MP3s? Yeah, me neither, but at least 3DSMax and photoshop will rock on the new chips.
Getting back to reality, imagine my surprise when I saw that this new preview studiously avoided games. They are testing two of the most popular gaming chips out there, and the heir to the throne, and they did not put in one single game benchmark. Not one, think about that.
In the rebuttal to this, there will be the usual cries of 'we were not testing gaming performance' or some such bullsh*t ass covering, but here is the truth, if you are going to multitask and do and do anything that tasks both of the CPUs, one of those is going to be a game.
If you read up on the benchmarks posited by the current crop of reviews, how many are things you do regularly? How many fit the a scenario that you have ever found yourself in? How many of you would do seven things concurrently if you had seven things to do rather than do one or two at a time, and probably end up at the finish line first? The human mind does not multitask well, so 19 active windows is 17 or 18 more than you really can use at once."

The only real-world evidence provided to support the claim that reviewers are being bought out is regarding dual core processors, so that's what I'll address.

Charlie claims that because benchmarks for dual cores tested things no typical user would ever do regularly, and reviewers avoided benchmarks where dual cores performed poorly, the reviewers must have been bought out.

Regarding the multitasking that no users would ever do: first Charlie points out that multi-core CPUs cannot be fully utilized until the software industry as a whole begins threading properly, and then moves on to browbeat reviewers because they didn't use multithreaded software to test, instead using suites that don't properly emulate real-world behavior? This is a ridiculous catch-22. Reviewers are doing a bad job because they didn't test real-world software usage, despite there being very few applications currently available that can actually make use of a dual-core properly? I'm sorry, but to me that just smacks of hypocrisy.

Regarding reviewers not testing benchmarks that make a product look bad: This is, again, a half-truth at best. Given that:

1. Applications that are single threaded aren't going to benefit from dual cores, and 2. Most games are currently single threaded (mostly, anyways)

...why would anyone benchmark current single threaded games? It's pretty obvious what the results would be. The dual-core is going to be slower than the single core that's clocked faster. There isn't any point in benchmarking a single threaded application on both a dual core and a single core chip running the same core where one is clocked faster than another. It'd be like comparing a 2.8 GHz P4 to a 3.6 GHz P4--the result should be pretty obvious.

One site that is probably guilty in the eyes of this article would be Anandtech. Their first dual-core article clearly omitted gaming performance, and they state the following:

Anandtech wrote: (For plain-jane single threaded application performance), the Pentium Extreme Edition or the Pentium D will simply perform identically to the equivalently clocked Pentium 5xx series CPU. The second core will go unused and the performance of the first core is nothing new. Given the short lead time on hardware for this review, we left out all of our single threaded benchmarks given that we can already tell you what performance is like under those tests - so if you're looking for performance under PC WorldBench or any of our Game tests, take a look at our older reviews and look at the performance of the Pentium 4 530 to get an idea of where these dual core CPUs will perform in single threaded apps. There are no surprises here; you could have a 128 core CPU and it would still perform the same in a single threaded application. Closer to its launch, we will have a full review including all of our single and multithreaded benchmarks so that you may have all of the information that will help determine your buying decision in one place.

Did Anandtech omit benchmarking games? Yes! Is there a good reason to do so? Yes!

However, in Anandtech's second review, they compare apples to apples from a price perspective--a dual core 2.8 GHz proc and a 3.0 GHz single processor P4. The dual core is clearly a better processor in this scenario, given the price. Applications that are threaded properly are going to utterly smoke the faster clocked single core processor.

An example of this is the application I'm currently writing, which is heavily threaded and includes multiple video encodes and decodes. The things I could do with a dual core architecture makes me giddy like a school boy. Not only would my application be able to re-encode content on the fly, but it would be capable of doing it with minimal interference to the rest of the computing being done by the computer. If I can get dual core hyperthreaded performance, that's even better.

It's going to take time for the software industry to catch up with dual cores, but when they do, the end user is going to see noticable benefits.