Autonomy levels, operational design domains and system architecture

Self-driving cars, robotaxis, driverless trucks, delivery drones and autonomous shuttles are the most visible robots most people will ever meet. They are also among the hardest robots to build, because they move fast, share space with people who did not agree to be part of an experiment, and must cope with weather, traffic and road layouts that nobody can list in advance. This course brings together what you learned in earlier robotics courses, state estimation in ROB-310, perception in ROB-330, mapping and localization in ROB-340, motion planning in ROB-350 and learning in ROB-410, and applies it to vehicles on roads and in the air. This first lesson sets out the vocabulary used across the industry: the levels of driving automation, the operational design domain that limits where a system may drive, and the layered software architecture that almost every autonomous vehicle shares.

Why vocabulary matters

Words such as autopilot, self-driving and autonomous are used loosely in advertising, and the confusion has consequences. A driver who believes a car can drive itself may stop watching the road, while the system expects the driver to take over in an instant. Regulators, insurers and courts therefore need precise terms that say who, or what, is responsible for each part of the driving task at each moment. Engineers need the same precision to write requirements, test plans and safety cases. The most widely used vocabulary comes from SAE International's standard J3016, Taxonomy and Definitions for Terms Related to Driving Automation Systems for On-Road Motor Vehicles, whose current edition was published in April 2021 jointly with ISO as ISO/SAE PAS 22736. Transport Canada, the US National Highway Traffic Safety Administration (NHTSA), provincial and state regulators all refer to it.

J3016 describes the dynamic driving task, or DDT: all the real-time operational and tactical functions needed to drive, including steering, braking and accelerating, and monitoring the environment by detecting, recognizing and responding to objects and events. It excludes strategic functions such as choosing a destination or a route. The standard also defines the DDT fallback: the response when a system failure or an exit from the allowed conditions means driving cannot continue normally. Either a human driver takes over, or the system itself reaches a minimal risk condition, such as stopping in a lane or pulling onto the shoulder, in which the risk of a crash is reduced.

The six levels of driving automation

J3016 defines six levels, numbered 0 to 5. The levels describe the feature that is engaged at a given moment, not the vehicle as a whole: a car can have a Level 2 highway feature and a Level 4 parking feature. The key dividing line falls between Levels 2 and 3. At Levels 0 to 2 the human is driving whenever the features are engaged, even if their hands and feet are not moving, and must supervise constantly. At Levels 3 to 5 the automated driving system, or ADS, performs the entire dynamic driving task while engaged, so the person in the driver's seat is not driving.

Level 3 is the most delicate. The system drives and monitors the road, but a human in the seat must be receptive to a request to intervene and take over within a reasonable time. That person may read or watch a film, but cannot sleep or leave the seat. Level 4 removes the need for a fallback-ready human inside the domain: if something goes wrong, the system itself must bring the vehicle to a minimal risk condition. This is why robotaxis without a safety driver are Level 4 systems, not Level 5. They operate only in particular cities, on mapped streets, within certain weather and speed limits.

The operational design domain

Every ADS at Levels 3 and 4 works only within an operational design domain, or ODD: the operating conditions under which the system is designed to function. J3016 lists examples such as road types, speed ranges, geographic areas, times of day, and environmental conditions like rain, fog, snow or glare. The ODD is a promise and a limit. The developer promises that the system will perform the whole driving task safely inside the ODD, and the system must recognize when it is approaching the edge of the ODD and respond, either by handing over to a fallback-ready user at Level 3 or by reaching a minimal risk condition at Level 4.

Describing an ODD precisely is an engineering task in its own right. A useful structure, used in standards such as ISO 34503 and BSI PAS 1883, groups attributes into scenery (road types, lane markings, intersections, junction types, zones such as school zones and construction), environmental conditions (weather, illumination, road surface, connectivity) and dynamic elements (traffic density, presence of pedestrians, cyclists and animals, speed of other vehicles). Each attribute is either permitted, permitted with conditions, or excluded. A robotaxi ODD might include urban arterials and residential streets up to 70 km/h, light to moderate rain, day and night, but exclude freeways, heavy snow and flooded roads.

The layered software architecture

Almost every autonomous vehicle, from a robotaxi to a delivery drone, organizes its software into a pipeline of modules, sometimes called the modular or classical stack. Sensors produce raw data. Perception turns that data into objects, free space, lanes and traffic signals. Localization estimates where the vehicle is on a map. Prediction forecasts what other agents will do over the next several seconds. Planning chooses a route, a behavior such as yielding or changing lanes, and a precise trajectory. Control converts the trajectory into steering, throttle and brake commands, or motor speeds for a drone. Underneath all of these sit the vehicle platform, the computing hardware, and the safety and monitoring functions that watch the system and trigger the fallback when something is wrong.

The modular structure has practical advantages. Each module has clear inputs and outputs, so teams can develop and test them separately, measure their errors and set budgets. Engineers can trace a mistake to a module: did perception miss the pedestrian, or did planning ignore a correctly detected one? Interfaces also make it easier to argue about safety, because a monitor can check, for example, that the planned trajectory never enters space perception marked as occupied. The disadvantage is that errors and uncertainty can be lost at the interfaces. If perception passes on only its best guess, prediction and planning cannot reason about how confident that guess was.

An alternative is end-to-end learning, where a single neural network, or a few tightly coupled networks, map sensor data directly to a trajectory or to controls. ROB-410 described how such policies are trained by imitation and reinforcement learning. Several developers now use hybrid designs: learned models for perception, prediction and planning trained jointly, with classical safety layers and rule checks on the output. At the time of writing, the most widely deployed driverless services still keep explicit intermediate representations, such as tracked objects and planned trajectories, because these support testing, debugging and safety arguments.

Timing and the loop rate

An autonomous vehicle runs its stack as a set of loops at different rates. Control typically runs at 50 to 100 Hz for cars and several hundred hertz for drone attitude control, because vehicle dynamics change quickly. Planning commonly runs at 10 to 20 Hz, and perception at the rate of its sensors, often 10 Hz for spinning lidar and 20 to 30 Hz for cameras. The total time from a sensor seeing an event to the actuators responding is the end-to-end latency, and it directly affects safety: during that time the vehicle keeps moving at its current speed.

Where autonomous vehicles operate today

At the time of writing (October 2026), driverless Level 4 services carry paying passengers in several US cities. Waymo reported more than 400,000 paid trips a week in early 2026 across Phoenix, the San Francisco Bay Area, Los Angeles, Austin, Atlanta and Miami, and announced expansions to further cities in Texas, Florida and elsewhere. Other companies run smaller services or testing programs, and driverless trucks carry freight on some Texas and southwestern highways. Canada has no driverless public robotaxi service at the time of writing, but Transport Canada guidelines and provincial pilots allow testing, and automated haul trucks have operated in Canadian oil sands and mines for years. Lesson 9 covers the rules in detail, and lesson 11 looks at trucking, transit, mining and agriculture. Because deployment changes month by month, treat any list of cities or fleet sizes as a snapshot and check current sources.

# Latency budget: distance travelled before braking for different speeds and latencies
speeds_kmh = [30, 50, 80, 100]
latencies_s = [0.15, 0.30, 0.50, 1.50]    # last one ~ typical human reaction
print("speed   " + "".join(f"{l*1000:>8.0f} ms" for l in latencies_s))
for v in speeds_kmh:
    ms = v / 3.6
    print(f"{v:3d} km/h" + "".join(f"{ms*l:>10.1f} m" for l in latencies_s))

speed        150 ms     300 ms     500 ms    1500 ms
 30 km/h       1.2 m       2.5 m       4.2 m      12.5 m
 50 km/h       2.1 m       4.2 m       6.9 m      20.8 m
 80 km/h       3.3 m       6.7 m      11.1 m      33.3 m
100 km/h       4.2 m       8.3 m      13.9 m      41.7 m