Defining Operational Conditions for Safety-Critical AI-Based Systems from Data

Artificial Intelligence (AI) has been on the rise in many domains, including numerous safety-critical applications. However, for complex systems in the real world, defining the underlying environmental conditions in which the AI-based system must operate---the Operational Design Domain (ODD)---is extremely challenging. This often results in an incomplete description of the ODD, which contrasts with the requirements of many domains for certifying AI-based systems. The ODD is also the applicability envelope of any evaluation protocol applied to such a system: an evaluation claim is only as trustworthy as the operational conditions under which it was measured, yet those conditions are traditionally fixed a priori by domain experts rather than derived from evaluation data itself. This paper presents a novel method to a posteriori define the ODD from previously collected data using a multidimensional kernel-based representation. This approach is validated through both synthetic benchmarks and a real-world aviation use case. Moreover, the paper defines similarity of two ODDs if they generate the same outputs up to Lebesgue-null input sets and proves convergence in volume of the calibrated representation under the stated assumptions. The novel, Safety-by-Design, deterministic kernel-based ODD representation is derived fully automatically, given documented assurance inputs, permutation-stable away from exact ties, bounded by construction, and, under affine-equivariant per-dimension normalization, invariant to the choice of units. Utilizing the proposed ODD representation supports future certification of data-driven, safety-critical AI-based systems, and offers a black-box, data-only construction of the domain over which any downstream evaluation protocol's conclusions may be considered valid.