Comments on: It’s the Season for Benchmarking

By: Loyd Case

Loyd Case — Tue, 25 Aug 2009 17:15:43 +0000

Thanks, Mark. I’ve fixed the contact page. The email is actually loyd (at) improbableinsights (DOT) com.

By: Mark

Mark — Tue, 25 Aug 2009 16:58:05 +0000

Sorry to post this here — delete after reading:

Loyd, I tried to send you an email this morning to the address listed on your contact page (loydcase@improbableinsights.com). A few mintues ago I got a ‘failed permanently’ message kicked back to me:

Technical details of permanent failure:
Google tried to deliver your message, but it was rejected by the recipient domain. We recommend contacting the other email provider for further information about the cause of this error. The error that the other server returned was: 550 550 5.1.1 … User unknown (state 14).

Hope that helps.

By: Loyd Case

Loyd Case — Tue, 25 Aug 2009 16:44:24 +0000

The other thing I’ll be doing will be percentile comparisons to a reference system. Right now, it’s a very high end system (Core i7 975). But I plan on keeping that around, and we’ll see systems that exceed that over time.

By: trip1ex

trip1ex — Tue, 25 Aug 2009 14:58:59 +0000

I used to pay attention to benchmarks.

Nowadays all I need is a rough number. 10% faster or 15% faster. And that’s it.

Your article could really just be a number. Take after videogame reviews and just assign the hardware a speedscore A 10 means it’s 10% faster. 15 means 15% faster. etc.

This would leads to lots of debate because assigning one overall number is judgement call. And imagine if all the various hardware sites started doing this. I can see the flames and discussions about why this or that site gave this or that piece of hardware such a high or low (speed)score.

Anyway benchmarks could use a little bit of fun and a judgement call like that. They’ve become stale.

Faster hardware is overkill for many tasks.

Pcgaming isn’t what it once was. It is not like new pcgaming hardware is even needed. DX11? It’s as if a million people yawned simultaneously.

Sure i7 gave video encoding a nice boost. Still takes too long and still can’t do video encoding very well at the same time you’re doing other tasks at least not that well on my 2.66ghz intel c2d cpu.

Software really has to come full front and take advantage of this hardware. Intel should just bury themselves in the lab for 5 years before announcing another cpu. By that time they’ll have a truly new and useful cpu for everyone.

Anyway applaud the effort to try and make benchmarks more relevant. HOpefully you’re not swimming upstream.

By: Carlo

Carlo — Tue, 25 Aug 2009 08:51:53 +0000

I like your approach. An approach shared by a UK magazine called Custompc, who have designed their own (I think) similar battery of tests, which you can download and use. Take a look at http://bit.ly/oolIs for the explanation and download, you may find it useful.

By: Markeyse

Markeyse — Tue, 25 Aug 2009 05:53:26 +0000

I can’t wait to see SSD Performance. This is definitely the way of the future and will be very valuable.
I do audio engineering and I am starting to dabble in video, and my hard ware is everything to me.

By: Brandon Champion

Brandon Champion — Tue, 25 Aug 2009 04:10:10 +0000

“I care about digital photography and video encoding performance. I care about games performance.”

Finally! No more irrelevant stuff. I wanna see things like… encoding time in Vegas and FPS in ArmA2. Something I can do on my OWN machine to compare.

By: Derek

Derek — Tue, 25 Aug 2009 00:35:26 +0000

I get all that. Just making the point that single-threaded performance is getting less meaningful. Since it is hardly possible to buy a single-threaded processor, nor to configure a machine to not have multiple uncontrollable processes it seems an academic discussion.
We’ll care that a task takes 60 minutes on the new box and 73 minutes on the old box. But until the software makers re-write a bunch of their core code, upgrading from a dual-core 9300 (for example) to an effective 8-core i920 possible just means that I will have a lot more idle time. Yes, identical tasks will be somewhat faster, but not 4 times as fast.
Ah, the good old days when upgrading from 25Mhz to 90Mhz actually meant almost 3x in performance boost.

By: Loyd Case

Loyd Case — Mon, 24 Aug 2009 23:28:25 +0000

I’ll bet if you looked at your performance meter in task manager, you’ll see that all the cores of your CPU is about 98% unused. Those tiny OS tasks take up very little CPU usage, and are often idel.

When you launch something CPU intensive, that also happens to be single-threaded, Nehalem will run that task in one of the cores at a higher frequency than the other cores. That’s when you’ll see at least one of the CPU meters peg.

Finally, Windows 7 is smarter than Vista and much smarter than Windows XP at assigning and maintaining a single, intensive task on one of the cores, rather than doing a lot of core swapping, which slows things down and wastes resources.

By: Derek

Derek — Mon, 24 Aug 2009 22:53:45 +0000

While single-threaded testing might be important for all the reason you mentioned, I have a hard time imagining a true single-threaded usage in everyday life. No matter what else I might be doing, there is always something else. Ventrilo during the gaming session, or Pigdin while watching a movie. When giving the computer a hours-long task like transcribing video (and walking away) you are virtually guaranteed to have the system come along and do something under the covers. Defragging the disk, a home server backup, the iPod deciding that NOW is a good type to sync.

Right at this moment I have 88 processes running on my Windows 7 box. I have complete control over maybe 20 of them. Is a single-thread test even meaningful?