Intel Investors Should Not Care About The 5s' Benchmarks

Nov. 5.13 | About: Intel Corporation (INTC)

With personal computer sales in a clear downtrend, the burden of driving future growth has fallen on Intel's mobile processor division. Investors seemed optimistic about the prospects of Intel (NASDAQ:INTC) making a successful entry into the smartphone and tablet market with their upcoming Silvermont Atom system on chip processors, but sentiment took a turn for the worse after the release of the iPhone 5s. To penetrate the mobile CPU market, Intel would need to make a processor that is significantly better than its competition, and there was Apple (NASDAQ:AAPL) beating them in certain key benchmarks - Many Intel bulls, myself included, reacted badly to this.

Click to enlarge

The Browsermark results in particular were troubling, as it is a holistic benchmark that more accurately mimics the type of load you would see in real world usage. Once the initial panic subsided, however, I realized that these benchmark results really don't matter for two reasons.

  1. You cannot actually tell whether Apple's A7 processor is more powerful than Intel's Atom Z3770 based on these benchmarks, especially at this stage.
  2. Even if the A7 was marginally more powerful than the Atom Z3770, it really wouldn't matter for Intel.

FFRD vs. 5s? Don't you mean Atom Z3770 vs. Apple A7?

Believe it or not, there is actually a very specific reason I titled the chart above using the handset names rather than processor names, and it makes a huge difference. A common misconception regarding benchmarks is that they measure the performance of a CPU or GPU. In reality, because these benchmarks are applications which run on devices, they measure the performance of the entire device.

It is common to see reputable sources associate PC benchmarks with specific graphics cards or CPUs, and they are not wrong in doing so. They key difference is that these benchmarks are done holding all other factors constant, i.e. they use the same computer and software for each benchmark, but simply swap out the GPU. This is, however, impossible to do with smartphone processors.

This distinction is crucial because a device's performance is equal to the performance of its weakest component, so if a smartphone scores lower on a benchmark than another, it does not necessarily have a slower processor. Its performance could just as easily be bottlenecked by the RAM, storage, motherboard, etc.

The hardware in Intel's Bay Trail FFRD (form factor reference design) is more than dissimilar enough from that of the iPhone 5s to influence benchmarks. As it is, manufacturers create and design their own motherboards, and often access different suppliers for other components, which partially explains why you see smartphones running the exact same processor scoring differently on benchmarks in the chart below. Now when you consider that the Bay Trail FFRD is a demo device that was never meant for commercial use, it's reasonable to assume that the differences in hardware may be immense.

Click to enlarge

The Software Side of Things

Differences in software, from the kernel to the application level, would also dramatically influence performance, and by definition benchmark scores. The iPhone 5s and the Bay Trail FFRD run different operating systems, OSX and Android respectively, and different kernels. I will not delve too deep into the technical nitty-gritty here, but know that the level of optimization at this level affects how well the system interfaces with and utilizes the CPU's available throughput.

At a higher level, both handsets also use different browsers, which run on different rendering engines. Even within the same device, a different browser can dramatically affect performance. I ran the SunSpider JavaScript benchmark on two different browsers on my smartphone to illustrate this point.

Click to enlarge

In layman's terms, all web browsers produce the same result - the translation of web code into a visual interface - but some will accomplish certain tasks in far fewer operations, thereby using the processor's available throughput more efficiently. It is very likely that the browser on the Bay Trail FFRD was not as optimized for the processor architecture of the device, like most browsers are, because it was only a reference design. Since all of the benchmarks in question are browser based, their scores were all likely skewed downward because of these differences.

The Bottom Line

One of the fundamental tenets of empirical analysis is that external variables must be held constant. Thus, in order to draw any meaningful conclusions from these benchmarks, the processor has to be the only independent variable. However, I have just listed a myriad of factors that prove that this is not the case, so it is far too early to panic. In fact, all the evidence suggests that the benchmarks' scores would be skewed downward, and will improve as the new Atom SoCs approach release.

Disclosure: I have no positions in any stocks mentioned, and no plans to initiate any positions within the next 72 hours. I wrote this article myself, and it expresses my own opinions. I am not receiving compensation for it (other than from Seeking Alpha). I have no business relationship with any company whose stock is mentioned in this article.