The slowdown of Moore's law turns out to have significant implications, perhaps especially for Intel (NASDAQ:INTC). The company already is suffering from losing out in mobile and seeing PC sales slump, but so far it has been rescued mostly from its stranglehold on servers, where it still holds a near monopoly in CPUs.
The latter have come under attack from Nvidia's (NASDAQ:NVDA) GPUs, which execute some tasks, more specifically those related to artificial intelligence where massive amounts of data are crunched in parallel.
Nvidia's rise in datacenters has been nothing less than spectacular. Last year it clocked a growth rate of 126% in this segment.
Intel is fighting back though. It's now giving datacenter chips priority on new process platforms (nodes) and it has acquired Altera, which provided field programmable gate array (FPGA) technology to accelerate computing in the cloud. Xilinx (NASDAQ:XLNX) is another player taking this route.
FPGAs is one answer to the slowdown of Moore's Law hitting general purpose CPUs, as they are "field programmable" that is, they can be reconfigured to specific end user needs after having been produced. From Intel:
Intel® FPGAs can be used to accelerate the performance of large-scale data systems. Intel FPGAs enable higher speed data processing by providing customized high-bandwidth, low-latency connections to network and storage systems. In addition, Intel FPGAs provide compression, data filtering, and algorithmic acceleration. With the Intel® FPGA SDK for OpenCL™, you can now rapidly develop acceleration solutions for computer and storage systems. The Intel FPGA SDK for OpenCL enables even software developers to easily design with FPGAs by allowing them to utilize a high-level programming language for developing acceleration functions.
It now seems that Google (NASDAQ:GOOG) (NASDAQ:GOOGL) has come up with another way to solve this problem. The route taken is not general purpose chips like CPUs and GPUs, nor programmable chips like FPGAs, but highly specialized chips, like dedicated accelerators.
Google's Tensor Processing Unit beat Intel's Xeon and Nvidia GPU in machine-learning tests by more than an order of magnitude, the web giant reported. A 17-page paper gives a deep dive into the TPU and benchmarks showing that it is at least 15 times faster and delivers 30 times more performance/watt than the merchant chips.
One might retort that while impressive, the benchmarks used are not cutting edge (the paper acknowledges that, since the 2015 tests, Intel has delivered 14-nm CPUs and Nvidia has new 16-nm GPUs) as the tests were conducted in 2015(!), but then again, neither is the TPU itself particularly cutting edge (in terms of frequency and processor node):
The 40-W TPU is a 28-nm chip running at 700 MHz, designed to accelerate Google's TensorFlow algorithm. Its main logic unit packs 65,536 8-bit multiply-accumulate units and a 24-Mbyte cache, delivering 92 tera-operations/second.
Just moving production to a smaller node would materially improve performance, and there are other tricks available:
If we compared newer chips, Section 7 shows that we could triple performance of the 28-nm, 0.7GHz, 40W TPU just by using the K80's GDDR5 memory (at a cost of an additional 10W).
If anything, it seems likely that as the TPU matures, its performance profile might actually widen its advantage over its competitors (from EE times, quoting Google's paper):
The order-of-magnitude performance advantage of the TPU is rare and could "lead to the TPU becoming an archetype for domain-specific architectures. We expect that many will build successors that will raise the bar even higher," the paper said... A relatively short 15-month design schedule prevented inclusion of many energy-saving features in the TPU, Google noted.
The TPU is already used in Google's datacenters, but the company doesn't provide any information on how widespread that use is, nor what the update path is or whether it will sell the TPU to third parties.
There are distinct warnings for Intel, and to a lesser extent Nvidia:
The TPU project began in 2013 with experiments with FPGAs. "We abandoned them when we saw that the FPGAs of that time were not competitive in performance compared to the GPUs of that time, and the TPU could be much lower-power than GPUs while being as fast or faster," the paper said.
Not so long ago, Intel enjoyed a virtual monopoly in the fast-growing server market. But with the slowdown of Moore's law and, especially, the shift in work done in the cloud from mere storage and simple applications towards complex AI, its CPUs are not able to do all this work efficiently, and the search was on for more efficient solutions.
Early alternatives were Nvidia's GPUs and FPGAs by Altera (now Intel) and Xilinx. These provided not a head-on threat to Intel, as servers still run on CPUs, almost invariably from Intel, but even here there is some emerging competition from AMD (NASDAQ:AMD), IBM (NYSE:IBM) and ARM-based designs.
But as more and more processing power is executed by GPUs and FPGAs, Intel is going to benefit less from the growth in the datacenter market.
And now there seems to be a new kid on the block in the form of dedicated accelerators like Google's TPU. How much a threat does this constitute?
For starters, one has to realize that just like GPUs and FPGAs, the TPU isn't a replacement for the CPU in servers. It provides additional processing capacity by executing some tasks (way) more efficiently.
As such, one could say that it's perhaps more of a threat to GPUs and FPGAs, but it's likely that each of these have their own advantages depending on the exact user needs.
But like GPUs and FPGAs, the TPU shifts processing power from the CPU where Intel is dominant.
So for Intel, the best-case scenario is that its usage remains limited to Google's cloud itself. As of yet, there is no indication whether Google is planning on marketing the TPU to third parties. It could be using the TPU to gain performance advantage for some of its own cloud applications and/or to limit its dependence on third party suppliers.
But advanced chip business benefits from enormous economies of scale so we don't think it's likely that Google will keep this as a proprietary solution.
What's more, it seems to have opened the door to a whole new class of data center solutions and Google was able to develop this in a stunning 15 months. How long before others follow?
Disclosure: I/we have no positions in any stocks mentioned, and no plans to initiate any positions within the next 72 hours.
I wrote this article myself, and it expresses my own opinions. I am not receiving compensation for it (other than from Seeking Alpha). I have no business relationship with any company whose stock is mentioned in this article.