Position: AI-for-science Systems Should Not Hide Consequential Analytical Decisions from Researchers

AI is increasingly used to automate complete analytical workflows, but existing approaches to human-AI decision-making largely evaluate how humans respond to system outputs, rather than how they participate in the analytical processes that produce them. We argue that scientific automation requires a process-oriented approach in which researchers can inspect, contest, and revise consequential intermediate decisions. Leveraging analytical provenance mechanisms, we propose that future AI-for-science systems learn to produce human-evaluable intermediate artifacts at meaningful milestones: representations of intermediate workflow states that present the information required for validation by researchers, and do so in a form appropriate to the underlying data and decision. We motivate this position through biomedical sensor-data analysis, where analytical choices frequently depend on visual evidence and contextual knowledge distributed across multiple domain experts. Because exhaustive human review would undermine the benefits of automation, we further argue that systems should learn when human judgment is sufficiently valuable to warrant feedback elicitation, and what artifact should be presented for that review. We formalize this as selective artifact-mediated human review within sequential decision-making, with human attention treated as a limited resource. This approach aims to preserve researcher autonomy while supporting systematically documented workflows in which consequential analytical choices remain accessible for researcher validation.