Capstone: a safety case and compliance review

This capstone brings the course together by building the core of a safety case and a compliance review for an invented service robot deployment. A safety case is a structured argument, supported by evidence, that a system is acceptably safe for a specific use in a specific environment. Unlike a compliance checklist, which shows that listed requirements were met, a safety case explains why the system is safe, which matters most for novel systems where no checklist is complete. Safety cases have long been required for nuclear plants, offshore installations and railways, are central to the new global regulation for automated driving systems described in lesson 7, and are the organizing idea of UL 4600 for autonomous products. Robot companies increasingly use them for deployments in public and care settings. Everything in the scenario is invented, and the compliance points illustrate the method rather than giving legal advice.

The scenario

Harbourview Health Center, an invented 400-bed hospital, plans to deploy twelve autonomous delivery robots made by an invented company, Ferro Robotics. Each robot weighs 90 kg loaded, travels at up to 1.2 m/s, carries medications, specimens and supplies in locked compartments opened by staff badges, rides elevators through an integration with the elevator controller, and navigates corridors used by staff, patients, visitors and children. It uses safety-rated laser scanners and bumpers for protective stopping, cameras with a learned person-detection model for path planning, and a voice interface that answers simple questions. A cloud-hosted fleet manager assigns tasks, and Ferro updates software remotely. The hospital wants to start with two wards and expand.

Structuring the argument

A safety case has three parts: claims (what is asserted, such as the robots are acceptably safe for use in the defined wards), arguments (why the evidence supports the claims, such as by addressing every identified hazard) and evidence (test reports, analyses, certificates, procedures). Goal Structuring Notation (GSN), developed in the 1990s and maintained as a community standard, draws the argument as a tree: goals are claims, strategies explain how a goal is broken down, solutions are evidence, and context and assumption nodes record the conditions. A good safety case also records defeaters, reasons the argument might be wrong, and shows how they were addressed, which guards against confirmation bias.

UL 4600 sets out what a safety case for an autonomous product should cover, including hazards from the autonomy itself, the operational design domain, perception and machine learning limitations, data and testing, human interaction, cybersecurity and field monitoring, and it requires independent assessment. For this capstone, the safety case has three branches under the top claim: hazards are reduced to acceptable levels; legal requirements are met; and safety is maintained in operation.

Branch 1: hazards and safety functions

The hazard log uses task-based analysis and STPA, as in lesson 2. Each hazard is assessed, and safety functions are specified with required performance levels using the ISO 13849-1 risk graph, as in lesson 3.

# Residual-risk tally for the hazard log and a check of what field data can show (illustrative)
import math
hazards = {   # name: (annual frequency of hazardous event after measures, prob. of serious injury given event)
    "corridor collision":       (0.40, 0.02),
    "low obstacle collision":   (0.05, 0.05),
    "elevator trapping":        (0.02, 0.05),
    "fire exit blocked":        (0.01, 0.10),
    "lid pinch":                (2.00, 0.001),
    "battery fire":             (0.002, 0.30),
}
total = 0.0
for name, (f, p) in sorted(hazards.items(), key=lambda kv: -kv[1][0] * kv[1][1]):
    r = f * p
    total += r
    print(f"{name:24s} serious injuries per year {r:.4f}")
print(f"total {total:.4f} per year, about one per {1 / total:.0f} years")
# how many fleet-km without injury would show a rate below the total, at 95% confidence?
km_per_year = 12 * 25 * 365
rate_per_km = total / km_per_year
print(f"fleet km per year: {km_per_year:,}; km needed with zero events: {-math.log(0.05) / rate_per_km:,.0f}")

corridor collision       serious injuries per year 0.0080
low obstacle collision   serious injuries per year 0.0025
lid pinch                serious injuries per year 0.0020
elevator trapping        serious injuries per year 0.0010
fire exit blocked        serious injuries per year 0.0010
battery fire             serious injuries per year 0.0006
total 0.0151 per year, about one per 66 years
fleet km per year: 109,500; km needed with zero events: 21,724,019

The tally identifies the corridor collision as the dominant residual risk, which directs attention to speed limits and field design in busy areas. It also shows that years of operation would be needed to confirm the estimate from field data, so monitoring near misses, not only injuries, is essential: the fleet's safety-rated stops, close passes and bumper activations provide far more data than injuries ever will.

Branch 2: legal compliance

The compliance review lists every legal requirement, the evidence that it is met and the owner responsible. The robot is not a medical device, because it has no medical intended use, but the hospital setting brings health privacy, accessibility and elevator rules into play.

For a hospital in a US state, the main items are these. Workplace safety: the hospital's duties under OSHA standards and the General Duty Clause, or the state plan, with Ferro's risk assessment, validation records and training. Electrical approval: NRTL listing of robots and chargers. Elevators: the integration with elevator controls must comply with ASME A17.1 and be accepted by the state or local elevator authority. Fire safety: the Life Safety Code (NFPA 101) as applied by the authority having jurisdiction and by accreditation bodies, including keeping egress clear. Accessibility: the Americans with Disabilities Act requires that robots do not block accessible routes. Privacy: cameras will capture images of patients, which can be protected health information, so the hospital as a HIPAA covered entity needs a risk analysis under the Security Rule, a business associate agreement with Ferro if Ferro handles protected health information, and limits on collection, use and retention. State privacy and biometric laws may also apply. Product liability, insurance and the contract allocate the remaining risk.

Branch 3: safety in operation and ethics

A safety case is not finished at go-live. The third branch argues that safety is maintained: staff are trained; routes and speed zones are reviewed after the first month; near misses and safety stops are monitored weekly; software updates follow a change control procedure with Ferro, including regression testing of safety functions and notice to the hospital before deployment; incidents are investigated and reported as the law requires; and the safety case is reviewed when the deployment expands to new wards, when the robots or their software change significantly and at least annually.

The ethics review, which lessons 9 and 11 support, asks questions the law may not. Will robots shift workload in unexpected ways, for example by making nurses wait at doors to receive deliveries? Are patients and visitors told about cameras in plain language? Could confused patients be frightened, and how will staff respond? Will the robots be used to monitor staff performance? Who decides, and how are workers and their unions involved? Recording these questions and the decisions made strengthens the safety case and builds trust.

Deliverables

The capstone deliverable is a concise safety case report of perhaps 20 to 40 pages for a real project, with appendices of evidence. It contains the limits of the deployment, the GSN argument, the hazard log, the safety requirements specification with PLr and achieved PL for each function, the validation summary, the compliance matrix, the operational safety plan, the ethics review and the list of defeaters with their resolution. It is signed by the people responsible, reviewed independently and kept under version control. For the course project, produce the top two levels of the GSN argument, a hazard log with at least eight hazards, safety function specifications for three of them, a compliance matrix for your chosen jurisdiction, an operational safety plan and three defeaters with rebuttals.