California Department of Motor Vehicles recently published the Autonomous disengagement reports for 2016. These reports provide data on which companies tested autonomous automobiles on California public roads in 2016 and what type of issues these cars faced during the testing process.
This data is closely followed by the industry, investors, and autonomous driving enthusiasts because it gives a good view of the status of autonomous driving technologies and how the participants are faring. For manufacturers and service providers, participating in this exercise is critical as it allows companies to collect field data in the all-important California market and provides for a mechanism to find real life problems with their systems. Companies use the field data to refine their autonomous systems and make them less error-prone.
Some manufacturers do not like testing in California as making the autonomous test data public gives the industry and investors visibility into the status of their technology. Some companies would likely prefer not to participate, if possible, given their technology will not fare well compared to the market leader Google (NASDAQ:GOOG) (NASDAQ:GOOGL). Uber (Private:UBER), for example, tried to circumvent the testing process as the company did not want to share its performance data. However, as a result of this non-compliance, Uber autonomous trials in California have been shut down by the DMV.
The 2016 reports show (image below from Wired), in a very telling way, that Google (GOOG) (GOOGL) / Waymo are in a compelling technology leadership position when it comes to autonomous driving. With about 636 thousand miles of field test data, Waymo has done almost two orders of magnitude more testing than its nearest competitor (less than 10 thousand miles of testing by GM (NYSE:GM) and Cruise Automation).
All other players, with the exception of Nissan (OTCPK:NSANY) and Delphi, have less than 1000 miles of autonomous testing data. While the low number of miles does not provide enough statistically significant data, it is clear from the available data that the technologies deployed by these companies are not mature.
Waymo also stands alone in the robustness of its technology with its cars disengaging from autonomous mode once every 5,000 miles. Google leads other vendors by almost two orders of magnitude even in this key metric.
With these statistics, Waymo appears to be on a path for commercial autonomous deployment in 2018 - likely a year or more ahead of all other players. Waymo's lead should not be surprising since it was the first to demonstrate a fully autonomous car, without a steering wheel, in 2014. It is quite telling that no other player, outside of Waymo, has reached a stage where it can deploy and test a system without a steering wheel.
It now appears that Google/Waymo is now on the verge of leveraging its technology lead and realizing the fruits of its nearly decade-long autonomous car research.
Setting aside Waymo, the most surprising revelation from the disclosures is that Tesla (NASDAQ:TSLA), the Wall Street darling, and perceived leader of autonomous technology, had only done 550 miles of testing on California roads. Of these 550 miles, 530 were driven in October and 20 were driven in November. There were 182 disengagements in these 550 miles. That is an abysmally high disengagement rate of once every 3 miles. This testing was done by a small fleet of four Model Xs.
Given the timing of these tests and media leaks on the subject, it appears that this testing was done for demonstration videos the company used to market its AP 2.0 technology. The lack of data, beyond this 550 miles, implies Tesla has done no real public testing of autonomous technology outside of what it took to develop a demo video.
These are stunning statistics and facts. They are stunning because the company announced its AP2.0 technology in early October and promised a future Level 4 and Level 5 autonomous operation. In other words, Tesla promised deployment of a technology that it did not even properly test in live conditions before it made such promise. Investors and customers should let the impact of this sink in.
It should be noted that the company failed to deliver on many of its AP 1.0 promises before obsoleting that hardware and moving on to AP 2.0. Given the lack of sufficient public testing and the poor disengagement data, we expect the history to repeat. We do not believe the company's 2.0 hardware is sufficient to ever reach Level 5 autonomy. Given the feeble test data and the lack of LIDAR in the hardware set, even Level 4 autonomy is questionable.
On the matter of AP 2.0 autonomy, bulls argue that the testing was done using an early stage system and the high disengagement rate should be expected. While there is some merit to the argument, some key questions for Tesla investors and customers to ask are:
- Why does Tesla not have public trials before or after this demo?
- Given the poor data from the testing, what gives Tesla the confidence to promise functionality without additional testing?
- How long will Tesla take to catch up to someone like Waymo whose disengagement rates are better by three orders of magnitude?
- What is the disengagement rate that Tesla would consider acceptable before full autonomy can be claimed?
- Why does the company believe it can reach full autonomy by the end of 2017, when it appears to have no data to support its claims?
- Why is Tesla marketing demoware? Why is it that Tesla competitors which have more and superior test data are not marketing vehicles with the promise of autonomy?
- If the data available does not support management claims to investors and promises to customers, should Tesla continue to make these claims?
- What liability does Tesla have to customers who leased or purchased cars based on Tesla's promises if the functionality is not ready by the end of 2017? Or 2018? Or 2019?
Bulls willing to accept Tesla's claims should also consider additional data. Some important tells on the status of autonomous technology are the feedback from the guinea pig customers of the current revision of the AP 2.0 software. This AP 2.0 software, which was forecasted by Tesla to be available in December, has been nothing but a dangerous science fair experiment.
The reports from these hard core Tesla fans should be terrifying to investors and customers. See: Buyer Beware: AP2 Extremely Dangerous
Considering the information presented so far, does it appear that Tesla will reach full autonomy by the end of 2017 as it is promising to do?
We cannot imagine a responsible company putting out such a buggy software in a safety critical software on the street. This software not only endangers the life and property of adventurous Tesla owners but other citizens on the road.
What we see here is that there is certainly no basis for Tesla to be making the grandiose claims it makes. We believe that Tesla has already lost the battle for autonomous driving leadership and the results will become evident over time.
Tesla management may feel that it has the competence to deliver on the goal but the data presented, and not presented, does not give any reason to trust this management.
It also begs the question when a management should revise its guidance when the current guidance is patently false?
Note that the highly exaggerated claims are not just limited to autonomous performance. For example, Tesla has not updated its prior guidance about producing 100 thousand to 200 thousand Model 3s in 2017. Given the company has yet to demonstrate even a production prototype of Model 3, the current guidance in place is impossible to meet. When will the management deem fit to update guidance on this matter?
Unfortunately, gullible customers and analysts are being duped with Tesla's bogus metrics and stale guidance. Analysts such as Adam Jonas of Morgan Stanley (NYSE:MS), run with this fictional "zillions of miles of autopilot miles" statistic and ascribe multi-billion dollar valuation to Tesla's autonomous ride share business which depends on the autonomous technology (it is altogether a different matter that Morgan Stanley has this caliber analysts on staff). The irony is that Morgan Stanley, a likely underwriter of Tesla's next capital raise, will use such patently bogus data to sell the secondary to unsuspecting investors.
A few conclusions:
- Tesla management has never been shy about boasting its real or imagined technology competencies. Given this, the poor disengagement data and lack of testing on public roads in the key California market imply that Tesla is likely far behind its own schedule and other players in achieving Level 4 autonomy.
- There is no evidence that the current sensor set will lead Tesla to Level 4 autonomy, let alone Level 5 autonomy
- Reports from customers who agreed to be beta testers for the AP 2.0 technology indicate that the technology is immature and dangerous and certainly unfit to be on public roads
- Tesla investors should ask for management to present running disengagement data to verify the status of its autonomous technology instead of the meaningless "millions of miles of AP miles" metric that the management uses.
Before it is here, it is on the Solar Insights subscriber platform. For timely and in-depth research and analysis of solar and battery industry stocks and developing news, please consider subscribing to our Solar Insights platform.
Disclosure: I am/we are long GOOG, GOOGL.
I wrote this article myself, and it expresses my own opinions. I am not receiving compensation for it (other than from Seeking Alpha). I have no business relationship with any company whose stock is mentioned in this article.