CPU Tests: Legacy and Web

In order to gather data to compare with older benchmarks, we are still keeping a number of tests under our ‘legacy’ section. This includes all the former major versions of CineBench (R15, R11.5, R10) as well as x264 HD 3.0 and the first very naïve version of 3DPM v2.1. We won’t be transferring the data over from the old testing into Bench, otherwise it would be populated with 200 CPUs with only one data point, so it will fill up as we test more CPUs like the others.

The other section here is our web tests.

Web Tests: Kraken, Octane, and Speedometer

Benchmarking using web tools is always a bit difficult. Browsers change almost daily, and the way the web is used changes even quicker. While there is some scope for advanced computational based benchmarks, most users care about responsiveness, which requires a strong back-end to work quickly to provide on the front-end. The benchmarks we chose for our web tests are essentially industry standards – at least once upon a time.

It should be noted that for each test, the browser is closed and re-opened a new with a fresh cache. We use a fixed Chromium version for our tests with the update capabilities removed to ensure consistency.

Mozilla Kraken 1.1

Kraken is a 2010 benchmark from Mozilla and does a series of JavaScript tests. These tests are a little more involved than previous tests, looking at artificial intelligence, audio manipulation, image manipulation, json parsing, and cryptographic functions. The benchmark starts with an initial download of data for the audio and imaging, and then runs through 10 times giving a timed result.

We loop through the 10-run test four times (so that’s a total of 40 runs), and average the four end-results. The result is given as time to complete the test, and we’re reaching a slow asymptotic limit with regards the highest IPC processors.

(7-1) Kraken 1.1 Web Test

Google Octane 2.0

Our second test is also JavaScript based, but uses a lot more variation of newer JS techniques, such as object-oriented programming, kernel simulation, object creation/destruction, garbage collection, array manipulations, compiler latency and code execution.

Octane was developed after the discontinuation of other tests, with the goal of being more web-like than previous tests. It has been a popular benchmark, making it an obvious target for optimizations in the JavaScript engines. Ultimately it was retired in early 2017 due to this, although it is still widely used as a tool to determine general CPU performance in a number of web tasks.

(7-2) Google Octane 2.0 Web Test

Speedometer 2: JavaScript Frameworks

Our newest web test is Speedometer 2, which is a test over a series of JavaScript frameworks to do three simple things: built a list, enable each item in the list, and remove the list. All the frameworks implement the same visual cues, but obviously apply them from different coding angles.

Our test goes through the list of frameworks, and produces a final score indicative of ‘rpm’, one of the benchmarks internal metrics.

We repeat over the benchmark for a dozen loops, taking the average of the last five.

(7-3) Speedometer 2.0 Web Test

Legacy Tests

(6-5a) x264 HD 3.0 Pass 1(6-5b) x264 HD 3.0 Pass 2(6-4a) 3DPM v1 ST(6-4b) 3DPM v1 MT(6-3a) CineBench R15 ST(6-3b) CineBench R15 MT

CPU Tests: Encoding CPU Tests: Synthetic and SPEC
Comments Locked

210 Comments

View All Comments

  • bji - Tuesday, January 26, 2021 - link

    Well I guess it's a difference of opinion about what "general availability" means. If most people who are not within direct driving distance of a MicroCenter are having a hard time finding these chips, then I would call that not "general availability". I did admit that I overstated the low number of people who are in driving distance to a MicroCenter, but I still contend that it's a low number relative to the total number of people who want to buy these chips. We could start arguing now about who is TRULY within reasonable driving distance of a MicroCenter -- if you have to drive 1 hour to get the chip, is it really still generally available? If your time is even worth $15/hr minimum wage, that's adding a significant cost in the dollar value of your time to the cost.

    But I really don't want to argue about it any more. I still think these chips are not generally available and posting an article that compares one to to another when one or both are not easy to get for the vast majority of people, is disingenuous as it presents choices that do not practically exist. If you disagree, then that's fine.
  • Spunjji - Wednesday, January 27, 2021 - link

    It's not a difference of opinion, it's a difference in perspective. I don't even care about MicroCenter because I'm not from the USA. These chips have "general availability" in the EU and UK at the very least, so I appreciate the comparison - as will many other people

    This article will be around longer than demand continues to outstrip supply in the USA, and maybe even long enough to see $15 an hour become the minimum wage there. 👍
  • brucethemoose - Thursday, January 21, 2021 - link

    Maybe they'll go the route of GPUs and just stop advertising TDPs entirely?

    This whole system works because most desktop users don't seem to care about TDP or power consumption. Performance at the absolute extreme end of the frequency/voltage curve is all that seems to matter.
  • Mr_Spock - Thursday, January 21, 2021 - link

    I appreciate the testing, showing that with most other things 'equal', there is effectively little difference between the 10700/K.

    Would you consider running any tests with a 65W-class cooler to demonstrate exactly how performance is damaged by only matching the TDP on the box?

    Having the full-bore tests for 10700 is quite useful if you can't find the K at a decent price, but if we're talking about quoted TDP 'limits', let's try limiting to that heat dissipation and see how quickly it flops.

    I built a few Haswell systems back in the day, and whoo boy, were those stock coolers ineffective if you got any decent chip under it.
  • DominionSeraph - Thursday, January 21, 2021 - link

    Eh? The stock cooler on the 4770 was fine. They only pulled 50W. I have no idea what you mean by "any decent chip under it" since there was only the 4770 and 4790 at the top and their wattage was about the same. (my 4790 was undervolted a tad and was 50W)
  • Mr_Spock - Thursday, January 21, 2021 - link

    Running prime95 on a stock cooler with a 4770 sent temperatures skyrocketing in my experience (80+C). I might have had too-high standards for cooling, since I always recommended at least a 212 EVO or better when it would fit into the cases we used. My employer used cheap cases that didn't come with an exhaust fan by default - installing 1+ case fans was always a recommendation for anything generating a lot of heat if I could convince the customer how much it was needed.

    Sounds like our use cases might be different? Or, you might have superior case cooling. All I know is we always had more tolerable temps once dumping the included cooler for something more suited to very heavy work.
  • vegemeister - Friday, January 22, 2021 - link

    80°C is safe. Read the Haswell datasheet. The embedded controller won't even tell you to start spinning up the fan until 80°C idle. https://www.intel.com/content/dam/www/public/us/en...
  • quiq - Sunday, January 24, 2021 - link

    now using a 4590 whit aftermarket cooler ambient temperature now is 34°C/93°F
    https://ibb.co/NKYXx6T

    I have to upgrade because whit the itb heatsink run over 60°C/140°F all the time
  • geokilla - Thursday, January 21, 2021 - link

    One reason to go for the K series is for faster RAM and the possibility of overclocking it to 5GHz. Shame there's no overclocking vs turbo results.
  • Naine - Thursday, January 21, 2021 - link

    You keep talking about how the TDP means nothing in the context of peak power draw, but it has nothing to do with peak power draw. That is only restricted by PL2, which is probably well over 200W for that chip, and like you say, motherboards can come up with their own PL2 values, especially if you slot this into the same high-end Z490 board you bought for your K CPU. Different story if you put it in a low-end B460 board.

    Here's what "65W TDP" means: If you pair this CPU with a 65W cooler, it will work.

    That's literally what it means.

Log in

Don't have an account? Sign up now