Vehicles today have several hundred sensors that are used to monitor vehicular subsystems. Such monitoring can track the health of these subsystems and can also be used to control vehicular behavior. In this project, funded by the National Science Foundation and General Motors, we explored ways in which these sensors could be used to improve vehicular safety, comfort, and performance. Many of the sensors can be accessed through a diagnostic interface using proprietary protocols, so to conduct our research, we partnered with a major car company to obtain access to these sensors.
One outcome of our research was a suite of algorithms that are used on-board sensors to improve the positioning of the vehicle. Because GPS can be inaccurate in downtown areas, we developed techniques to use onboard sensors to obtain positioning with enough accuracy to locate the vehicle within a given lane. These techniques combine GPS readings with map information, but also use vehicular sensors to determine whether a vehicle traverses a speed bump, or turns right or left, or comes to stop at a signal, then uses these determinations to learn positional corrections. These positional corrections leverage the wisdom of the crowds: if multiple cars stop at the same light, then the average of their positional readings is likely to be more accurate.
A second outcome of the research built upon this idea and developed a collection of crowd-sourcing algorithms that use onboard sensors to determine driving behavior or various aspects of the environment. Because vehicles have hundreds of sensors, if drivers would share their sensor readings (in much the same way that drivers today share information on apps like Waze), we hypothesized that we could determine various aspects of the environment such as: stop signs, road curvature, road grade, and pothole positions. We developed sensor processing and aggregation algorithms and demonstrated that these algorithms could reliably detect these environmental features.
Our most recent outcome explored a novel capability that we call Augmented Vehicular Reality (AVR). Today, and in the foreseeable future, vehicles will have “3D sensors” like LiDAR and stereo cameras. Autonomous vehicles and driver assist systems rely on these sensors. These sensors can perceive depth, but their view is limited by obstacles. AVR enables a vehicle to obtain, using wireless communication, readings from other sensors, so that the vehicle can effectively “see through” obstacles. This capability can greatly increase the safety of autonomous driving, enabling a vehicle to plan its path more effectively. The code for AVR is publicly available (https://github.com/USC-NSL/AugmentedVehicularReality).
Concretely, these outcomes have resulted in two patents, several publications, and we have transitioned code to industry.
Autonomous vehicle prototypes today come with line-of-sight depth perception sensors like 3D cameras. These 3D sensors are used for improving vehicular safety in autonomous driving, but have fundamentally limited visibility due to occlusions, sensing range, and extreme weather and lighting conditions. To improve visibility and performance, we explore a capability called Augmented Vehicular Reality (AVR). AVR broadens the vehicle’s visual horizon by enabling it to wirelessly share visual information with other nearby vehicles. We show that AVR is feasible using off-the-shelf wireless technologies, and it can qualitatively change the decisions made by autonomous vehicle path planning algorithms. Our AVR prototype achieves positioning accuracies that are within a few percentages of car lengths and lane widths, and it is optimized to process frames at 30fps.
Like today’s autonomous vehicle prototypes, vehicles in the future will have rich sensors to map and identify objects in the environment. For example, many autonomous vehicle prototypes today come with line-of-sight depth perception sensors like 3D cameras. These cameras are used for improving vehicular safety in autonomous driving, but have fundamentally limited visibility due to occlusions, sensing range, and extreme weather and lighting conditions. To improve visibility and performance, not just for autonomous vehicles but for other Advanced Driving Assistance Systems (ADAS), we explore a capability called Augmented Vehicular Reality (AVR). AVR broadens the vehicle’s visual horizon by enabling it to share visual information with other nearby vehicles, but requires careful techniques to align coordinate frames of reference, and to detect dynamic objects. Preliminary evaluations hint at the feasibility of AVR and also highlight research challenges in achieving AVR’s potential to improve autonomous vehicles and ADAS.
