Schooling the AI ‘Drivers’ of Autonomous Vehicles
Self-driving cars and trucks promise quicker trips. Xianfeng “Terry” Yang is working to make those faster rides safer by training autonomous ‘drivers’ to navigate dangerous circumstances.

Self-driving cars and trucks promise quicker trips. Xianfeng “Terry” Yang is working to make those faster rides safer by training autonomous ‘drivers’ to navigate dangerous circumstances.

Accelerating, then braking for red light after red light. A blocked line of sight. A lapse in attention.
Driving can be tedious and dangerous. Autonomous (self-driving) vehicles (AVs) promise to save time and reduce crashes. Xianfeng “Terry” Yang, associate professor of civil and environmental engineering and director of the Maryland Transportation Institute, is working to make this technology truly roadworthy.
“Autonomous vehicles don’t make human errors, they don’t get distracted. But they definitely have their own problems,” Yang says.
The Society for Automotive Engineers ranks vehicles’ autonomy:
Level 0: No Driving Automation
Level 1: Driver Assistance
Level 2: Partial Driving Automation
Level 3: Conditional Driving Automation
(e.g. Tesla: requires supervision by human driver)
Level 4: High Driving Automation
(e.g. Waymo robotaxis: may need remote human assistance)
Level 5: Full Driving Automation
His lab uses artificial intelligence (AI) to maximize how efficiently AVs travel on the road system. Simultaneously, Yang and his team are addressing some of the technology’s problems, including critical gaps where the AV lacks training to make split-second, potentially life-or-death decisions.
An unlikely scenario? That’s exactly the point. Yang improves AVs by simulating extremely rare edge cases so they may be solved in the lab before wreaking havoc on the road.
As the technology matures, AVs may eventually communicate directly with the systems that manage traffic flow. Yang is working to help them use the information they receive to speed travel. For instance, given the timing of red lights along a high-traffic thoroughfare, an AV could adjust its trajectory to avoid stopping, according to Yang.
“It can calculate that, if a vehicle slows down by five miles per hour, then by the time it gets to the intersection, the light will turn from red to green,” he says.
His team uses reinforcement learning, a type of AI in which the algorithms improve their performance through trial and error, thereby teaching AVs to make these kinds of adjustments.
Yang uses computer models, miniature vehicles, and full-sized vehicles to test AVs.
AVs have the potential to significantly reduce car accidents and the loss of life and injury that they cause.
But first, researchers like Yang must tackle their shortcomings. With support from the U.S. National Science Foundation, projects in his lab seek to better simulate AVs’ performance on snowy or icy roads, and to allow AVs to detect hazards using information from roadside sensors.
The team is also focusing on preparing AVs for rare—but possibly deadly—scenarios. AVs receive standard training that introduces them to most of the circumstances on the road, but not to everything. It can miss edge cases, such as those involving bad weather, unpredictable behavior by pedestrians or bicyclists, or a crash that can’t be avoided.
Dianwei Chen, a graduate student in Yang’s lab, uses large language models to identify potential edge-case scenarios; he then verifies and replicates them in CARLA, an open-source 3D driving simulator built on the Unreal Engine. Using reinforcement learning, Chen looks for ways to prevent this situation from arising and, if a crash is unavoidable, ways to minimize injury and damage.
“We want to define the problem, and then solve it—not wait until the problem happens on the road,” Chen says. “That’s my final goal.”