Defining Operational Conditions for Safety-Critical AI-Based Systems from Data

Artificial Intelligence (AI) has been on the rise in many domains, including numerous safety-critical physical AI applications such as autonomous driving, aviation, and robotics. However, for complex systems in the real world, defining the underlying environmental conditions in which the AI-based system must operate---the Operational Design Domain (ODD)---is extremely challenging. This often results in an incomplete description of the ODD, which contrasts with the requirements of many domains for certifying AI-based systems, including deployed robots. Traditionally, the ODD is created in the early stages of the development process, drawing on sophisticated expert knowledge and related standards, an approach that scales poorly to embodied systems intended to generalize zero-shot across environments never explicitly specified by a designer. This paper presents a novel method to a posteriori define the ODD from previously collected data using a multidimensional kernel-based representation. This approach is validated through both synthetic benchmarks and a real-world aviation collision-avoidance use case, a safety-critical autonomy setting structurally analogous to physical robot deployment. Moreover, the paper defines similarity of two ODDs if they generate the same outputs up to Lebesgue-null input sets and proves convergence in volume of the calibrated representation under the stated assumptions. The novel, Safety-by-Design, deterministic kernel-based ODD representation is derived fully automatically, given documented assurance inputs, permutation-stable away from exact ties, bounded by construction, and, under affine-equivariant per-dimension normalization, invariant to the choice of units. Utilizing the proposed ODD representation supports runtime out-of-distribution monitoring and future certification of data-driven, safety-critical AI-based systems.