The True Contender
Tesla (NASDAQ:TSLA) is fond of making fantastical claims which its followers often take at face value. Such claims include the impending ability to provide self-driving technology as well as selling cars which are supposedly already full self-driving capable in hardware terms and are just waiting for the appropriate software to (soon) arrive.
These claims are very hard to dispel. For the most part it's hard to get concrete data showing reality leans this or that way when it comes to self driving. In the absence of such data, Tesla fans always default to believing everything Tesla says, and then some. For them, Tesla is the undisputed self-driving champion, no proof required. Just look at all those self-driving learning miles Tesla has versus all others. No chance. Not a contest.
It so happens that at times, hard data becomes available. And when such happens, these wild beliefs have to be confronted with such data. This is one of those times. The California DMV just published the Autonomous Vehicle Disengagement Reports for 2016. This article will extract facts from those reports, and show how the reports paint an ugly picture for Tesla's actual prowess.
I had already covered the 2015 reports in my article titled "Tesla Has A Self-Driving Car, Here's How It Compares." Back then, Tesla hadn't yet filed anything with the California DMV showing that it wasn't even testing autonomous cars in California during 2015. I had to extrapolate Tesla's numbers and compare them to what the other competitors were achieving nearly one year earlier. The results didn't look good for Tesla.
The 2016 Report
That was then and this is now. Now, as California's DMV put out the 2016 report, we have actual and updated information on all the brands testing in California public roads, including Tesla. So here we go, first a summary of all the reports:
In graphical form (excluding BMW and Ford, likely unrepresentative because of being highway-only):
What observations can we already draw? Several:
- Google/Waymo (NASDAQ:GOOG) (NASDAQ:GOOGL) kept its massive testing and performance advantage over all others.
- Three other automakers are doing substantial testing in California: GM/Cruise Automation, Delphi and Nissan.
- Of those three other competitors, GM/Cruise automation is working the hardest streets. All of its testing is in urban environments. Arguably, GM/Cruise's testing is in an even harder environment than Google/Waymo's.
- BMW and Ford's testing seems preliminary and done just on highways, where the challenge is minor. The results are not comparable to everyone else's.
- Tesla is hopelessly behind. It is the brand with fewer testing miles in spite of being based in California (meaning, several other of these companies also are testing elsewhere including other states and countries). It had a disengagement rate which only bested Bosch and Mercedes, but is at least behind Mercedes in practice, as all of its miles were urban. Arguably and taking into account all data, Tesla is ahead of Bosch, Ford and BMW, and behind everybody else by a large margin.
- On General Motors' (NYSE:GM) effort, there's something the aggregate averages don't fully capture: GM/Cruise is improving at an amazing rate. Arguably, between the environment it tests at and the improvement, GM/Cruise is already the main contender to Google/Waymo. The following chart depicts the miles/disengagement ratio attained by GM/Cruise as the year developed:
General Motors/Cruise Automation
As I observed, it is now apparent that GM/Cruise might already be the main contender to Google/Waymo, and improving rapidly.
We already had reason to believe GM/Cruise was onto something special when it posted a demo video shot in the streets of San Francisco:
However, looking at the hard data and its progression makes it more impressive still. The data shows that the video wasn't a mere demo but actually what one would already expect to happen on most trips taken by GM/Cruise test cars. The same cannot be said of Tesla.
This also brings memories of something else. GM beat Tesla to the market with an affordable EV by one year. In what regards autonomous driving, GM looks to be even farther ahead of Tesla.
The Leader, Though, Is Still The Same
The report also makes something clear. Google/Waymo remains the clear leader. Its average miles per disengagement now exceed 5,000 miles, a significant improvement from last year, when it was barely above 1,000 miles per disengagement.
I would however add that GM/Cruise's rate of improvement right now exceeds even Google's, and that the rate of improvement is taking place on an arguably harsher environment.
A Final, More Subjective, Interpretation
There's something which Google and GM/Cruise have that Tesla doesn't have. Both use LIDAR in their self-driving approaches as well as high-resolution maps of their environments. Tesla, on the other hand, is building its currently massively inferior system based on the thesis that vision is enough to get the job done (though supplemented by a front-facing radar and 12 ultrasonic sensors).
Theoretically, vision alone is indeed enough to get the job done. We can say this with certainty because we do it when driving - we base ourselves overwhelmingly on vision, with just accessory inputs from our notion of balance as well as hearing.
However, there's a massive difference between something being possible and it being attainable on any reasonable timeframe. It is here that LIDAR comes in. LIDAR produces certainty regarding the environment. It does this by shooting out a massive number of lasers, timing their reflection, and thus building a very exact point cloud of the environment, which can then easily be translated into a very reliable 3D map of everything around the car. On top of this, calculating this map over time yields exact information about the movement of all surrounding objects, their exact positions, their exact speeds and approximate their trajectories.
With the knowledge of where everything is around the car that's being self driven, the exact trajectory the self-driven car will follow in that high resolution map, the exact trajectory of everything else, it becomes a much more limited problem to not run the risk of hitting anything. Of course, a gigantic task remains ahead to build a self-driving car. Recognition of the actual objects surrounding the car helps in defining their likely behavior, which helps understanding their probable changes in trajectory. Then there are markings, signs, road rules and a thousand other things to reach self driving.
But the point is a tremendous task which faces the development of a self-driving car: knowing the exact surrounding physical world and all objects in it, is made simple by LIDAR to a very high degree of confidence (which I would term "certainty" under favorable weather conditions). When it comes to relying on video alone, that entire task still remains ahead of the self-driving candidate. Video feeds don't directly give exact information about everything surrounding the car. It's possible to recreate a 3D world from a few cameras or even one camera in motion, but the information from that reconstructed world is much less certain in all regards, from the identified objects, to their positions, to their trajectories.
What does this mean? It means that Tesla by foregoing the use of LIDAR puts itself at a significant disadvantage. The problem Tesla needs to solve, to attain the even the same level of performance as the others, is a much more complex problem. The results, up until now, confirm this.
Now that there's hard data to go by, we can conclusively say Tesla lags other self-driving competitors dramatically. It's not even in the same league, neither in terms of total testing, nor in terms of how well its cars behave, nor even on the environments it's able to face.
The leader, by a massive margin to all others, remains Google/Waymo. However, GM/Cruise is also doing an extraordinary job and improving much faster than other players. Nissan also merits a mention.
It can also be argued that Tesla's material disadvantage is structural on account of its hardware options. Not having LIDAR simply makes the problem harder to solve. As others will approach self-driving deployment, Tesla might well be forced to adopt LIDAR. Of course, if Tesla does that it then creates problem - all of the cars it's selling as having full self-driving hardware would then be a liability. After all, their owners would have a legitimate claim against Tesla in that they'd have been wronged into buying a car on false promises.
Disclosure: I am/we are short TSLA.
I wrote this article myself, and it expresses my own opinions. I am not receiving compensation for it (other than from Seeking Alpha). I have no business relationship with any company whose stock is mentioned in this article.