Last year, Amazon.com's Graviton CPU wasn't competitive.
That was because Amazon.com was using 4 generations-old technology.
The Graviton2 uses more recent technology. The result is different.
One year ago, I published an article on Amazon.com (AMZN) titled “Amazon.com's Graviton CPU: A New War, A Lost Battle.” In that article, I detailed why Amazon.com’s then-new Graviton CPU was not likely to be competitive. The main reason was that it was based on ARM’s old A72 core. Here’s my conclusion at the time:
This particular battle, this particular ARM implementation, will likely show itself to not be competitive against present day x86 instances.
However, the war has clearly started. It won’t end anytime soon, and given what ARM and Apple are currently achieving this war will become a lot more difficult for Intel (NASDAQ:INTC) and AMD (NASDAQ:AMD) in the near future.
Amazon.com and cloud customers in general will eventually emerge winners, as processing power will become cheaper for all. Just not quite yet.
In that article, I also predicted that things could turn much more different once Amazon used technology based on ARM’s latest core (at the time), the A76.
Today, Amazon announced the successor to the original Graviton CPU, the Graviton2. The Graviton2 is based on ARM’s Neoverse N1 technology, which is itself based on the A76 core I said would make things substantially different. In this article, we’ll see how different, using a similar approach to the original article.
This Battle Will Go Very Differently
Unlike during its first attempt, this time Amazon will be deploying technology based on ARM’s latest design. The jump is huge. We’re talking about a 4-generation jump (A72 – A73 – A75 –A76). On top of this, ARM’s core performance took a large jump from the A75 to the A76 as well.
So what did this all mean in terms of performance? Amazon is saying the following (interestingly, these are comparisons to the Intel Xeon Platinum 8175 processor, whereas in my original article, I compared to the Intel Xeon Platinum 8180, which runs at the same base frequency):
All of these performance enhancements come together to give these new instances a significant performance benefit over the 5th generation (M5, C5, R5) of EC2 instances. Our initial benchmarks show the following per-vCPU performance improvements over the M5 instances:
- SPECjvm 2008: +43% (estimated)
- SPEC CPU 2017 integer: +44% (estimated)
- SPEC CPU 2017 floating point: +24% (estimated)
- HTTPS load balancing with Nginx: +24%
- Memcached: +43% performance, at lower latency
- X.264 video encoding: +26%
- EDA simulation with Cadence Xcellium: +54%
As for our own exercise, it goes like this:
- The A76 core is a recent ARM core. It’s used in many top Android phones carrying the Snapdragon 855 CPU. In the Snapdragon 855, the fastest core (which typically will set a CPU's peak single-core performance) runs at 2.84Ghz. In the Neoverse N1 used on the Graviton2, it will likely run at 3.1Ghz, among other changes. When used in smartphones like the Xiaomi Mi9 or Galaxy S10, the Snapdragon, supported by its high-frequency A76 core, can get a Geekbench 4 single-core score of ~3,500. Since the Neoverse N1 runs at a higher clock, plus has other changes, we can linearly expect it to print at least ~3,820 in this benchmark.
- Of course, Geekbench is just one of many possible benchmarks. And each will showcase different strengths or weaknesses for a given CPU. However, originally we already were able to show that the Graviton would be weak because our estimate for its adjusted Geekbench 4 score was just 1,909. Now, notice the following slides comparing the A76 core to the A73 core (which is already superior to the A72 used in the original Graviton):
- These are all very consistent with what we found: That the new architecture can deliver a ~100% improvement in performance vs.the previous A72 core (and thus the previous Graviton single-core performance).
- Now, the CPU Amazon compared to (Intel Xeon Platinum 8175) is basically the same as we compared to in our original work. So its numbers will remain constant. At the time I estimated we were looking at a 3,084 single-core Geekbench 4 score.
- So what did we see here? From the Xeon being 61% faster than the Graviton, we’re now back to a place where the Graviton2 is ~24% faster than the Xeon. That is, the ARM-based CPU went from losing the battle to winning it.
Of course, you’ll say that Intel and AMD won’t sit still. But neither will ARM. The next ARM core (A77) already promises another large (20%) single-core jump. This exceeds what’s to be expected by Intel or AMD in the short term.
Of note, I concentrate on single-core performance, because that’s mostly what server instances sell. Instances sell that or a few up to many cores. ARM tends to excel on multi-core performance as well, but this isn’t as obvious from smartphone benchmarks. This is so because of power and thermal constraints as well as the fact that only a single A76 core is clocked at 2.84Ghz in the example I gave. Neither of these constraints exist on a server chip.
While Amazon’s first attempt at ARM-based chips seemed too weak, the second attempt – the Graviton2 – already seems fully competitive.
This is yet another shot across the x86 bow. This also is yet another move toward the commoditization of computing power, and might result in more bargaining power for Amazon when buying chips from Intel or AMD. Obviously, later on it's likely that other players also will use ARM's Neoverse N1 and subsequent designs.
The pressure from this trend will not abate, because the next ARM core will, again, show faster performance improvement than the competing x86 solutions.
Idea Generator is my subscription service. It's based on a unique philosophy (predicting the predictable) and seeks opportunities wherever they might be found, by taking into account both valuation (deeply undervalued situations) and a favorable thesis.
Idea Generator has beaten the S&P 500 by around 33% since inception (in May 2015). There is a no-risk, free, 14-day trial available for those wanting to check out the service.
Disclosure: I/we have no positions in any stocks mentioned, and no plans to initiate any positions within the next 72 hours. I wrote this article myself, and it expresses my own opinions. I am not receiving compensation for it (other than from Seeking Alpha). I have no business relationship with any company whose stock is mentioned in this article.