Towards Measuring Creative Agency through Observable Signals in Human–AI Graphic Design

Evaluations of generative AI systems for graphic design measure what the system produces along axes such as visual quality, aesthetic preference, prompt-following, and layout fidelity while offering limited insight into how AI affects human creativity. We propose an Agency Signal Framework which treats creative agency as a latent construct that can be operationalized through observable signals beyond artifact-centric metrics. The proposed framework identifies four such signals, mapped to the four canonical properties of agency in social cognitive theory: Goal Agency (intent realization), Signature Agency (persistence of individual visual identity), Exploratory Agency (breadth of considered alternatives), and Ownership Agency (perceived authorship). Each signal is defined through a measurement protocol combining artifact, process, and self-report measures, normalized to [0,1]. We instantiate the framework across 70 participants, five agentic design systems, and observe that the operationalized signals capture distinct dimensions of creative agency across conditions. We further observe that architectural choices, particularly the preservation of editable, layered structure, are more strongly associated with agency preservation than raw model capability. The findings suggest that measuring creative AI should consider not only the quality of the generated artifacts, but also how AI systems preserve/alter human creative agency during collaboration.