<Object Recognition for Autonomous Driving>
Tesla's approach to autonomous driving development differs from that of Waymo and GM Cruise. While Waymo implements autonomous driving using LiDAR and precise maps, Tesla employs reinforcement learning (imitation learning) based on camera-acquired video to implement self-driving technology. That is, Tesla makes autonomous driving possible using only stereo cameras (vision) and radar, doing away with LiDAR technology. Tesla CEO Elon Musk is quoted as having said that “anyone relying on LIDAR is doomed,” because of its high price and battery consumption.
Current LiDAR technology is not cost-effective. A battery electric vehicle (BEV) is a device wherein efficient use of electrical power by a computer to realize autonomous driving is important; high power consumption by elements other than the on-board computer is undesirable. Further, high power consumption by LiDAR systems can negatively affect the range of self-driving vehicles.
From Tesla’s viewpoint that the EV must become a device that is optimized for autonomous driving, LiDAR is a primary culprit behind decreased EV efficiency.
As mentioned above, the Tesla approach to autonomous driving does not involve precision maps. As such, LiDAR, which senses the environment surrounding a vehicle and creates a 3D map of the vehicle’s surroundings, is less important to Tesla than it is to other companies, and can be replaced by vision systems.
Source: ACS Paper - Life Cycle Assessment of CAVs, Meritz Securities Research Center
Tesla’s current “camera-radar combination” may be less accurate than a LiDAR-high definition (HD) map combination in environments where insufficient data collection has been carried out.
However, once data is accumulated from test vehicles and Tesla EVs already sold and on the road, with the advancement of deep learning techniques, accuracy may be improved to the point that autonomous driving is possible even in environments for which there is no map data available. This is why Tesla's autonomous driving performance has become far superior to that of other companies.
<Tesla Autopilot: Tesla has accumulated massive amounts of data from EVs on the road>
Recently, Tesla announced that it would go one step beyond foregoing LiDAR, even removing radar from its vehicles. For some models (Model 3 and Model Y vehicles delivered in North America from May 2021 onwards), cameras will replace radar in sensing a vehicle's surroundings, which is essential for autonomous driving. The move represents a complete reliance on ‘vision’ through the vehicle's cameras.
While unlike LiDAR, radar is inexpensive, its distance and object detection accuracy is lacking, and processing radar places an unnecessary burden on the vehicle's computing capacity.
<Elon Musk’s Twitter post on removing Radar>
Tesla’s commitment to implementing self-driving using vision only to the exclusion of LiDAR and radar is evident in a patent (US 1095655) granted to Tesla on May 21 of this year. (Insert claims from US patent here)
(US 10956755 도면 5)
Reference: Diagram illustrating an example of capturing auxiliary sensor data for
training a machine learning network (US 10956755, FIG. 5)
The patent was granted on the characteristic of training using a data set of training images and distance sensor measurements, then calculating distance only image data acquired through a vehicle camera. In particular, this patent is associated with the implementation of pseudo-LiDAR, which fulfills the role of LiDAR through distance measurement based on images acquired through stereo cameras.
The application for the patent was filed in the U.S. on February 19, 2019, with a PCT application filed on February 7, 2020. As of August of this year, Tesla is set to secure rights in other PCT states as well. It is expected that the claims granted in other countries will be similar to those granted in the U.S.
It is likely that Tesla will implement autonomous driving using a vision system only, without LiDAR or radar, then adopt LiDAR and radar technology later on when prices decrease and battery consumption efficiency improve. That is to say, once LiDAR and radar have a negligible effect on range and optimized self-driving, which are essential to autonomous EVs, there would be no reason not to use these technologies for safer and more seamless self-driving.
It is considered to be important that the ways real human drivers think are implemented as-is in autonomous driving, and Tesla’s directions for self-driving development appear to be along these lines.
Even without maps or knowledge of the terrain, a human driver can drive a vehicle based on judgments made exclusively through visual data received through the eyes.
Human drivers who have better spatial perception and judgment in emergency situations are better drivers.
Further, on frequently traveled roads whose environments they understand better, drivers can drive even better, aided by visual data acquired in real-time. Human drivers are able to drive even more safely when aided by ADAS (Advanced Driver Assistance Systems).
While humans use just two eyes – usually facing in a single direction – to acquire visual data for judgment in the brain, autonomous vehicles analyze images acquired through cameras facing in multiple directions. So long as clear visual images can be acquired regardless of weather and the surrounding environment, and the deep learning model which corresponds to the human brain is capable of fast and accurate judgment, safe autonomous driving will be possible. (Humans also have a hard time driving or making out the road in rainy weather.)
Also, if radar and LiDAR can assist real-time judgment based on camera images, safer self-driving will be enabled, similar to the manner in which ADAS systems help humans drive more safely. If, additionally, an autonomous vehicle could determine its surrounding environment based on high-definition maps, the vehicle could drive more safely and accurately as human drivers do when driving through frequently driven and well-known environments.
While the safest autonomous driving will be possible when a vehicle employs all of the technologies currently being implemented for autonomous vehicles, what is most important is to select which current technologies will provide the most efficient mode of autonomous driving at given circumstances and points in time. For example, while individually owned vehicles will need to compromise between maximizing autonomous driving safety and achieving the highest possible range with a single charge, vehicles which can be charged frequently and in which battery consumption by sensors is less of an issue (for example, mass transit buses) could afford to make use of all learning models and sensors available to maximize autonomous driving safety.
Related Professional
Taekyun Chung
Managing Partner
<Object Recognition for Autonomous Driving>
Tesla's approach to autonomous driving development differs from that of Waymo and GM Cruise. While Waymo implements autonomous driving using LiDAR and precise maps, Tesla employs reinforcement learning (imitation learning) based on camera-acquired video to implement self-driving technology. That is, Tesla makes autonomous driving possible using only stereo cameras (vision) and radar, doing away with LiDAR technology. Tesla CEO Elon Musk is quoted as having said that “anyone relying on LIDAR is doomed,” because of its high price and battery consumption.
Current LiDAR technology is not cost-effective. A battery electric vehicle (BEV) is a device wherein efficient use of electrical power by a computer to realize autonomous driving is important; high power consumption by elements other than the on-board computer is undesirable. Further, high power consumption by LiDAR systems can negatively affect the range of self-driving vehicles.
From Tesla’s viewpoint that the EV must become a device that is optimized for autonomous driving, LiDAR is a primary culprit behind decreased EV efficiency.
As mentioned above, the Tesla approach to autonomous driving does not involve precision maps. As such, LiDAR, which senses the environment surrounding a vehicle and creates a 3D map of the vehicle’s surroundings, is less important to Tesla than it is to other companies, and can be replaced by vision systems.
Source: ACS Paper - Life Cycle Assessment of CAVs, Meritz Securities Research Center
Tesla’s current “camera-radar combination” may be less accurate than a LiDAR-high definition (HD) map combination in environments where insufficient data collection has been carried out.
However, once data is accumulated from test vehicles and Tesla EVs already sold and on the road, with the advancement of deep learning techniques, accuracy may be improved to the point that autonomous driving is possible even in environments for which there is no map data available. This is why Tesla's autonomous driving performance has become far superior to that of other companies.
<Tesla Autopilot: Tesla has accumulated massive amounts of data from EVs on the road>
Recently, Tesla announced that it would go one step beyond foregoing LiDAR, even removing radar from its vehicles. For some models (Model 3 and Model Y vehicles delivered in North America from May 2021 onwards), cameras will replace radar in sensing a vehicle's surroundings, which is essential for autonomous driving. The move represents a complete reliance on ‘vision’ through the vehicle's cameras.
While unlike LiDAR, radar is inexpensive, its distance and object detection accuracy is lacking, and processing radar places an unnecessary burden on the vehicle's computing capacity.
<Elon Musk’s Twitter post on removing Radar>
Tesla’s commitment to implementing self-driving using vision only to the exclusion of LiDAR and radar is evident in a patent (US 1095655) granted to Tesla on May 21 of this year. (Insert claims from US patent here)
(US 10956755 도면 5)
Reference: Diagram illustrating an example of capturing auxiliary sensor data for
training a machine learning network (US 10956755, FIG. 5)
The patent was granted on the characteristic of training using a data set of training images and distance sensor measurements, then calculating distance only image data acquired through a vehicle camera. In particular, this patent is associated with the implementation of pseudo-LiDAR, which fulfills the role of LiDAR through distance measurement based on images acquired through stereo cameras.
The application for the patent was filed in the U.S. on February 19, 2019, with a PCT application filed on February 7, 2020. As of August of this year, Tesla is set to secure rights in other PCT states as well. It is expected that the claims granted in other countries will be similar to those granted in the U.S.
It is likely that Tesla will implement autonomous driving using a vision system only, without LiDAR or radar, then adopt LiDAR and radar technology later on when prices decrease and battery consumption efficiency improve. That is to say, once LiDAR and radar have a negligible effect on range and optimized self-driving, which are essential to autonomous EVs, there would be no reason not to use these technologies for safer and more seamless self-driving.
It is considered to be important that the ways real human drivers think are implemented as-is in autonomous driving, and Tesla’s directions for self-driving development appear to be along these lines.
Even without maps or knowledge of the terrain, a human driver can drive a vehicle based on judgments made exclusively through visual data received through the eyes.
Human drivers who have better spatial perception and judgment in emergency situations are better drivers.
Further, on frequently traveled roads whose environments they understand better, drivers can drive even better, aided by visual data acquired in real-time. Human drivers are able to drive even more safely when aided by ADAS (Advanced Driver Assistance Systems).
While humans use just two eyes – usually facing in a single direction – to acquire visual data for judgment in the brain, autonomous vehicles analyze images acquired through cameras facing in multiple directions. So long as clear visual images can be acquired regardless of weather and the surrounding environment, and the deep learning model which corresponds to the human brain is capable of fast and accurate judgment, safe autonomous driving will be possible. (Humans also have a hard time driving or making out the road in rainy weather.)
Also, if radar and LiDAR can assist real-time judgment based on camera images, safer self-driving will be enabled, similar to the manner in which ADAS systems help humans drive more safely. If, additionally, an autonomous vehicle could determine its surrounding environment based on high-definition maps, the vehicle could drive more safely and accurately as human drivers do when driving through frequently driven and well-known environments.
While the safest autonomous driving will be possible when a vehicle employs all of the technologies currently being implemented for autonomous vehicles, what is most important is to select which current technologies will provide the most efficient mode of autonomous driving at given circumstances and points in time. For example, while individually owned vehicles will need to compromise between maximizing autonomous driving safety and achieving the highest possible range with a single charge, vehicles which can be charged frequently and in which battery consumption by sensors is less of an issue (for example, mass transit buses) could afford to make use of all learning models and sensors available to maximize autonomous driving safety.
Related Professional
Taekyun Chung
Managing Partner