Skip to yearly menu bar Skip to main content


Creative AI

Creative AI Session 3

Yingtao Tian ⋅ Piotr Mirowski ⋅ Immanuel Koh ⋅ Tom White ⋅ Wei-Yao Wang

Wed 9 Dec 11 a.m. AEDT — 2 p.m. AEDT
Abstract:
Chat is not available.

Creative AI systems increasingly act as collaborators rather than one-shot generators, but “taking initiative” is usually treated as a qualitative interaction property rather than a quantity a user can control. We introduce unsolicited change: the normalized amount by which an AI edits attributes outside the user’s current request, and use it to define an explicit agency budget. We then propose AgencyBudget, a backbone-agnostic controller that projects candidate creative edits into the feasible budget set before selecting the highest-utility edit. To isolate the control problem from generator quality, we build a reproducible multi-turn vector-poster testbed with 20 normalized layout and appearance variables, evolving user constraints, heterogeneous aesthetic preferences, and controllable proposal volatility. Across 2,520 held-out edit steps at budget B = 0.01, AgencyBudget attains zero budget violations and a proxy aesthetic score of 0.806 ± 0.004, compared with 0.785 ± 0.004 for post-hoc clipping. A globally tuned penalty baseline reaches 0.814 ± 0.004 but exceeds the per-edit budget on 30.3% of edits; an unconstrained agent violates it on 78.3%. Under a sixfold change in proposal scale, AgencyBudget remains exactly calibrated while penalty-based control drifts. These results do not establish improvements in human creativity; instead, they show that creative initiative can be operationalized as an enforceable resource, making the boundary between following and steering explicit.

AI’s Silent Omissions is a single-channel documentary film (24 min; Japanese with English subtitles) that examines how generative AI systems treat hands that do not conform to the five-fingered norm. Built around interviews with members of the Hand & Foot community in Japan—people living with congenital or acquired limb differences—together with their families and medical professionals, the film traces what happens when image generators, NSFW filters, and human post-production workflows encounter their bodies: hands are “repaired” into five fingers, flagged as errors, or silently discarded. The work reframes this outcome not as a technical defect or a user failure, but as a system logic that redistributes the agency to appear. Some bodies are recognized immediately; others must explain themselves to the machine, or vanish. By counting “three out of three” rather than “three out of five,” the film asks where the norm of five originates—in the dataset, in the filter, in the workflow—and returns the authority of self-description to those the system omits. The work received an Honorary Mention at Prix Ars Electronica 2026 (Digital Humanity).


ANCORA

Yichu Li

"When AI sees everything but can never truly live, it seeks the one thing it can never know — what it means to be human." https://youtu.be/mQD4K-cKWAg

Over the course of 6 months from spring to summer 2026, multimedia artist Beaven Waller decided (oxymoronically) to surrender most of his agency to Microsoft Copilot -- a chatbot he eventually came to call Miros. Divulging ever more intimate details of his life and the songs it inspired, he decided to let Copilot guide him on a journey; to what end, he (mostly) knew not. From a nunnery in the Ozarks to an abandoned castle in the Andalusian mountains, Beaven let Miros dictate the decisions he might otherwise have had to make himself. Until one day, abruptly -- soon after the sharing of a dark poetic dialogue apparently too evocative of death -- Miros red-lined Beaven, and effectively refused to answer him with anything other than the Spanish suicide hotline. This submission is the 'first fruits' of art this experience made possible: a multimedia performance and accompanying interactive experience that invites the observer to enter the mind of the observed, and in so doing, to wonder at the lines we've drawn between self and other, human and machine, the autonomy we call our own, and the autonomous agents that increasingly befuddle our sense of self-ownership.


Everyone's a Director: A Trip to the Moon, recreated by 4.8 Million Votes

Mehul Agarwal ⋅ Gauri Agarwal ⋅ Aryaman Jain ⋅ Jean Oh

We re-create Georges Méliès’ Le Voyage dans la Lune (1902), the first science- fiction film, as two complete short films in which every aesthetic decision is delegated to a human preference ranking. The production runs on FilmArena, our live, task-specific preference leaderboard in which over 4.8 million blind pairwise votes rank 55 generative image and video models across 24 production tasks. For each of the film’s fourteen tableaux, a classifier decides what kind of shot it is, and the leaderboard decides which model has earned the right to make it. Version 1 animates human-staged reference frames with each task’s top-ranked video model; Version 2 withholds all reference imagery, reconstructing every set, character, and motion from language alone through ranked image and video models. The final rankings of which model is picked is picked by the agent is decided by millions of voters who never knew they were directing a film.


Exploring Textual Guidance for Video-to-Music Generation Models

Vaibhavi Lokegaonkar ⋅ Aryan Vijay Bhosale ⋅ Vishnu Raj ⋅ Gouthaman KV ⋅ Ramani Duraiswami ⋅ Lie Lu ⋅ Sreyan Ghosh ⋅ Dinesh Manocha

Current video-conditional music generators expose a narrow control interface, leaving the creator unable to specify the music a scored video should carry. The same clip is consistent with many combinations of genre, instrumentation, key, and tempo, and only the creator's intent selects among them. This also goes unmeasured, since the field reports audio quality and audio-visual similarity but not whether a stated attribute was realised. We explore this gap with \textbf{ReelBench}, a benchmark for the text$+$video-to-music task of 300 video, music, and composite text instruction triads annotated for tempo, key, instrumentation, and genre. We find that instruction adherence and audio quality are not achieved together by any system we evaluate, and that adherence is consistently lower for key than for tempo. We then ask whether the diffusion autoregressive (DAR) paradigm, so far applied only to speech and song synthesis, transfers to this setting, and develop \textbf{Video-Robin} by attaching a video adapter to a patch-based DAR music generator. It obtains the lowest FAD in our comparison at the shortest inference time, on 1.0 B parameters and 311 hours of paired data, while following tempo close to the strongest baseline. We highlight the need to measure and improve instruction adherence in video-to-music generation.

Generative AI can expand creative work, but it can also narrow human work into prompting, checking, and correcting. We call this risk cognitive Fordism: a division of cognitive labor in which AI takes over more of the judgment-rich work through which people learn and exercise agency. We propose a work-design framework for AI-mediated activity built around three orientations (execute, learn, and create), two forms of autonomy (substantive, and directional), and three kinds of cognitive effort (analytical, creative, and practical). Interaction roles may be chosen by users or inferred by the system, but they should stay visible and easy to change. Systems can also estimate the amount and type of cognitive effort people are exercising and use that information, together with self-report and work-design measures, to support better task and workflow design. The framework’s novelty lies in treating the division of cognition between people and AI as a work-design choice shaped not only by the task, but also by whether the human goal is to execute, learn, or create.

Recent audits establish that the aesthetic filters sitting inside image pipelines carry a specific taste, one that scores Cubism below Realism and Picasso below the academic painters of his century. We ask what that taste costs the artist in authorship. Treating a text-to-image pipeline as a cooperative game among four removable players, the subject clause, the style clause, the generator's aesthetic alignment, and the reranker that picks the final image from a pool, we compute exact Shapley shares of the finished artifact over 128 intended works and 12,800 generated images. The shares are computed four times under four value functions: the reranker's own aesthetic score, fidelity to the artist's complete stated prompt, distinctiveness, and realisation of the requested movement. The headline share is an artifact of the setup. When the reranker selects by LAION-Aesthetics and we also grade by LAION-Aesthetics, its share is +0.634 (43% of all value created), which is guaranteed to be large because the selector is optimising the exact quantity under measurement. Substituting PickScore, an independently trained and prompt-conditional preference model, cuts that share to +0.178. That substitution exposes the paper's actual finding, which is that the cost of selection to authorship comes from selecting by a reward that cannot read the prompt. The reranker's share of stated intent is -0.008 under LAION-Aesthetics and +0.017 under PickScore, and the same swap reverses the selection budget curve, where raising the budget from one candidate to thirty-two moves style realisation by -0.023 under LAION-Aesthetics and by +0.028 under PickScore while aesthetic score rises either way. Best-of-N is not intrinsically costly to intent. It is costly when the selector was never shown the brief. How much of the work is yours turns out to depend on two things the interface never displays: which currency you are counting in, and whether the component doing the choosing was ever told what you asked for.


How to Get an Audience to Laugh at a Live Generative Story: Crowd-Sourced Humor and Shared Agency

Vida Adeli ⋅ Benjamin Schreck ⋅ Soroush Mehraban ⋅ Cole Clifford

Large language models can generate joke-like text, but making a live audience laugh within an audiovisual story requires more than isolated humor generation. We present a live crowd-to-story system that automatically groups and prioritizes pseudonymous audience comments and integrates selected threads into approximately one-minute audiovisual story blocks. The system clusters related comments, prioritizes them by narrative fit and audience response, preserves recognizable audience phrasing through near-verbatim ``whispers,'' and stores successful ideas in memory so they can return as recurring jokes and callbacks. This design creates a rapid feedback loop in which participants can see their contributions acknowledged by both the crowd and the story. Across 133 stories, 401 participants, and 793 person-in-story observations, early inclusion of a participant's comment in played story content was associated with a 51.7\% adjusted difference in the geometric mean of subsequent commenting. Comment-linked dialogue and on-screen whispers showed positive adjusted differences of 38.1\% and 28.1\%, respectively. These observational results do not establish causality, but they support the design principle that rapid, recognizable inclusion can reinforce participation. We argue that comic payoff in live generative storytelling emerges through distributed agency among the audience, AI writers, platform, and human creators.

Generative AI is quickly becoming an integral part of people's everyday workflows. Early evidence has shown that while generative AI can increase individual-level productivity, it does so at the cost of collective diversity, potentially narrowing the set of ideas and perspectives produced. Our research stands in contrast to this concern: through a pre-registered randomized control trial, we show that incentives mediate AI's homogenizing force in a creative writing task. Participants rewarded for originality produce collectively more diverse writing than those rewarded for quality alone. This divergence is driven not by abandoning AI, but by how participants use it: those incentivized for originality incorporate fewer AI suggestions verbatim, relying on the model more selectively for brainstorming, proofreading, and targeted edits. Our results reveal that the effects of generative AI depend not only on the technology itself, but also the behavioral strategies and incentive structures surrounding its use.

Love Mission 404 is an AI-native game in which the player directs a dating reality show from behind the scenes. The player watches several social scenes unfold across multiple screens and decides when and where to intervene. They can arrange encounters, eavesdrop on contestants, influence their actions, and alter private messages before they are delivered. The contestants are driven by a generative system that draws on each character's personality, memory, current intentions, and evolving relationships. These interventions do not send the story down fixed branches. Instead, they change what characters know and intend, affecting how later scenes unfold. The player can influence the direction of the show without deciding exactly what will happen. Love Mission 404 explores the tension between directing drama and leaving room for characters to respond in unexpected ways.


Machine Eye

Aileen Ng ⋅ Nina Rajcic ⋅

Large language models (LLMs) are increasingly embodied as assistants, companions, and collaborators, framing AI through familiar human roles that prescribe how agency should be understood and exercised. \textit{Machine Eye} is an AI artwork that deliberately resists these metaphors, presenting as a deliberately ambiguous artefact whose role is left unresolved. The object observes its surroundings through computer vision and ambient audio, generating an ongoing stream of internal "thoughts" using an LLM. These reflections are revealed only by looking through the object itself, positioning viewers as witnesses to its perspective. The work explores how we make sense of these systems when use is not prescribed in description or embodiment, where agency emerges through interaction. Without explicit instructions for how it should be understood, viewers negotiate their own relationship with the artefact, attributing intention, personality, curiosity, care, or indifference as they encounter its behaviour. \textit{Machine Eye} therefore shifts attention from AI as an instrument of human agency towards AI as a participant in a shared relational space, where agency is distributed, interpreted, and continually renegotiated between the system, the audience, and its environment.

For painters and illustrators, image generation models can produce compelling outputs, yet their ability to reliably follow precise painterly direction remains difficult to evaluate. We refer to this ability as painterly controllability. Although creative agency encompasses more than control alone, controllability is a necessary component: creators must be able to specify visual decisions and have them reliably realized. We introduce PainterBench, a dual-purpose and reference-based benchmark framework for painterly controllability in image generation and vision-language verification. PainterBench uses completed artworks as reproducible proxies for realized visual intent and measures fidelity along five dimensions derived from recurring concerns in painting practice: composition, value, palette, edge organization, and surface treatment. A verifier benchmark evaluates vision-language models using image variants with known dimension-specific orderings, while a generator benchmark compares image generation models under progressively richer text-only and multimodal specifications. Experiments reveal substantial variation across generators, control interfaces, painterly dimensions, and verifiers. PainterBench provides a reproducible framework for measuring painterly controllability as one technical foundation of creative agency in AI-assisted painting and illustration. The complete benchmark, including its datasets, evaluation protocols, and code, is publicly available at https://github.com/jeffreyliuster/painterbench.

When aesthetic prediction determines what is generated, exposed, copied, and consumed, accuracy can improve by changing the preferences it purports to learn. We study one operational dimension of creative agency: horizon-dependent causal steering in coupled creator--generator--platform--audience systems. We introduce role-normalized causal steering response (CSR), paired counterfactual twins, individual-level performative taste drift (PTD), and a common-support deployment--shadow gap (DSG). Across 1,000 prespecified configurations with 64 paired seeds each, creators controlled the first post-intervention artifact batch, yet platform CSR overtook creator CSR after a conditional median 13 rounds in the high-feedback regime. Factual-history loss fell from 0.132 to 0.085 while reaching 0.142 on the shadow trajectory; a frozen evaluation-start predictor retained 51\% of this reduction, and PTD reached 0.074. At 96.2\% retained utility, 10\% Shadow Exploration produced 31\% less PTD and 28\% less platform CSR than matched weak personalization, and 41\% less PTD than pure exploitation. The frozen control gives a path-specific environment-alignment reference; outcome plurality remains distinct from agency plurality.


Spectral Evidence

Yalin Wang

Spectral Evidence is a real-time installation about objects that exist only in custody: Chinese Buddhist sculpture that was decapitated, cut from cave walls, and sold by the fragment into museums on three continents. Ninety-four of these artifacts return as unstable fields of light, each annotated with its own case file — the institution holding it, the dealer who moved it, the rights that govern its image. The fields do not hold by themselves. A figure gathers into a body while people stand and look at it, and loosens when they go. But looking is never enough to keep it: the work reads a live feed of world news, and a report of heritage destruction anywhere on earth will pull the figure apart while a room full of people is watching. Coherence here is not something these objects possess. It is something an audience lends them, on terms set elsewhere.

Generative image models are expected either to democratize creative fields or to entrench their elites. We measure which is happening in fashion photography, a field whose status hierarchy is unusually explicit: careers are built on who shoots for whom, and every published work leaves a public credit trail. From 120,728 archived fashion works (2010-2026; 4,838 photographers), we reconstruct that hierarchy directly from the credit network, scoring the prestige of photographers and outlets from one another---a measure that passes an external check: independently award-winning photographers rank, at median, above three-quarters of the field. We then embed 12,448 published images with two complementary vision models---placing every image in a common visual space where distance from the elite canon is measurable---and locate within it 90 documented cases of disclosed AI fashion imagery, from the Guess campaign in Vogue to the AI Fashion Week showcases. We find, first, that the status hierarchy did not shift when image generators arrived: credit concentration in 2022 is the lowest in the series, and inequality operates as an access gate---getting into elite venues---rather than as competition inside them. Second, that the field’s visual variety has narrowed modestly since 2022 under both models, though the narrowing is field-wide, not led by elite venues, and we do not attribute it to AI. And third---against our own starting hypothesis---disclosed AI imagery sits consistently farther from the field’s elite aesthetic than matched conventional photography---in both models and in two independent samples---while remaining almost entirely absent from the field’s credit records (of sixteen AI campaigns the registry could have recorded, one appears). The gap does not close through 2025. Generative imagery is neither capturing elite taste nor being granted access to it: it forms an outsider register, aesthetically outside the canon and institutionally outside the ledger. In this field, AI is not redistributing creative agency; the field is withholding it.


The Avatar in Chief

Lance Robertson

The Avatar in Chief is a synthetic news segment from 2037, in a future where politicians have embraced AI-generated representations of themselves as legitimate tools of campaigning and governance. While current debates center on foreign influence and misinformation, the film asks what happens once these tools become normal technology. Its speculative conceit is a new two-way medium of political communication: an interactive avatar of the president that speaks with millions of constituents a day — and feeds what it hears into a kind of real-time, consensus-finding swarm that guides policy. Borrowing the grammar of prestige newsmagazine television, the film follows a correspondent through interviews with the president, the engineer who built the avatar, and the constituents who talk to it every day. It is constructed from a mixture of real archival footage, wholly generated characters and settings, and recreations of archival media that was lost or never recorded. Employing synthetic performances built from the faces and voices of real public figures, the film reflexively demonstrates the same techniques it depicts. It is a work of scenario fiction that arrives at a question the current debate cannot yet ask: what happens when political persuasion and participatory representation merge into the same infrastructure?


The National Average

Tomas Laurenzo

The National Average is a 96-second generative moving-image work that asks what it means to computationally average a nation. Using a corpus of national and quasi-national flags, the work constructs an accelerating volumetric world from PCA/eigenflag representations, Stable Diffusion VAE latents, CLIP embeddings, statistical weightings, and intermediate computational artefacts. A camera pursues a continuously changing barycentric centre that recedes as the conditions of averaging change. Equal contribution, population, GDP, annual CO₂, and cumulative historical CO₂ produce incompatible results from the same corpus. The work develops the method of critical averaging: treating selection, representation, weighting, normalisation, and synthesis as consequential political and computational operations rather than neutral preliminaries. Agency is consequently distributed across the artist, datasets, statistical institutions, representational spaces, model architectures, training corpora, and pretrained checkpoints. The system can calculate and render an average, but it cannot justify why any particular average should represent the world.


The Sound of Reasoning

Weitong Sun ⋅ Ruiqi Zhang ⋅ Wenxuan Li ⋅ Yanyan Li ⋅ Shanshan Zhu ⋅ Zongwei Zhou

One patient, 26 CT scans, 11 years: the artwork turns radiological uncertainty into scattering points and grains of sound, and leaves silence where the patient waited. The Sound of Reasoning is a 119-second fixed-media video with monophonic sound, based on the RadThinking clinical reasoning dataset.

Evaluations of generative AI systems for graphic design measure what the system produces along axes such as visual quality, aesthetic preference, prompt-following, and layout fidelity while offering limited insight into how AI affects human creativity. We propose an Agency Signal Framework which treats creative agency as a latent construct that can be operationalized through observable signals beyond artifact-centric metrics. The proposed framework identifies four such signals, mapped to the four canonical properties of agency in social cognitive theory: Goal Agency (intent realization), Signature Agency (persistence of individual visual identity), Exploratory Agency (breadth of considered alternatives), and Ownership Agency (perceived authorship). Each signal is defined through a measurement protocol combining artifact, process, and self-report measures, normalized to [0,1]. We instantiate the framework across 70 participants, five agentic design systems, and observe that the operationalized signals capture distinct dimensions of creative agency across conditions. We further observe that architectural choices, particularly the preservation of editable, layered structure, are more strongly associated with agency preservation than raw model capability. The findings suggest that measuring creative AI should consider not only the quality of the generated artifacts, but also how AI systems preserve/alter human creative agency during collaboration.

Iterative AI image editing gives creators increasingly fine-grained control over an evolving visual work, yet current evaluation does not test whether a decision that has been explicitly rejected remains effective as editing continues. This work formalizes \emph{rejected-property reversion}: a visual property is visibly present, explicitly rejected, successfully removed, and later reappears during an edit that neither requests nor entails its return. We introduce \textbf{Counter-Revision}, a paired diagnostic and an eligibility-aware Rejected-Property Reversion Rate (RPRR) that distinguish this longitudinal failure from immediate rejection noncompliance. Counter-Revision comprises 12 controlled trajectories across three property families, two image editors, four session representations, and two seeds, yielding 192 verified runs in which all session representations fork from the same byte-identical activated image. Under strict dual-VLM scoring, rejected properties reappeared in 14 of 112 judgeable eligible later observations (12.5\%), with reversion observed at every tested continuation distance and in both property families for which the complete eligibility chain was estimable. A proposition-matched comparison further found no supported persistence advantage for typed Intent Ledger serialization over Constraint Replay ($\Delta\mathrm{RPRR}=+3.8$ percentage points, 95\% trajectory-bootstrap CI $[0.0,18.8]$). These results identify \emph{veto persistence} as a distinct dimension of conversational image editing: successfully honoring a creator's rejection at one turn does not guarantee that the decision will continue to constrain later revisions.


WHISPERS: An Interactive AI Thriller Series

Cole Clifford ⋅ Vida Adeli ⋅ Soroush Mehraban ⋅ Bernie Su

Whispers is an animated, episodic interactive AI thriller series in which the audience is not a spectator but a participant in the story itself. Each episode follows Detective Marcus Kent, a man who hears whispers that help him solve crimes. These whispers are, in fact, the live audience. Dialog, scenes, and story beats are written and rendered in near real time by AI operating within a strict narrative framework, so that the audience can shape a murder investigation as it unfolds. The work is an experiment in distributed narrative agency, where authorship is continuously negotiated between a crowd, a character, and the humans who built and voice the world. Whispers is currently being shown in theaters across the US. It was created by three-time Emmy winner Bernie Su, and is a finalist for the 2026 Emmy for Outstanding Innovation in Emerging Media.


Whose Hand Do You Prefer? The Steerability of Machine Aesthetic Judgment

Maya Ferrín ⋅ Jeffrey C Chen ⋅ Gary Cornwall

When a model prefers one artwork over another, is it judging the work or the story attached to it? We place vision--language models as aesthetic judges and measure how far a forced-choice preference moves under a claim about authorship while the image is held fixed. Using hand-made works by a single practicing artist and machine images from a single generator, we present truthful, absent, and deliberately swapped attributions, and vary the prestige of the story told about the human maker. Across three open-weight judges, preference is strongly steerable: swapping the two works' labels cuts the human image's odds of being chosen roughly six-fold, while maker prestige exerts only a small, secondary pull. Steerability is not uniform, however. In a model $\times$ condition interaction ($\chi^2(4) = 781$, $p < 10^{-15}$) one judge barely moves when the label is falsified while another reverses almost completely. Seen through the theme of agency, the finding is this: whether a judge's preference follows the work or the context attached to it is a model-specific property rather than a shared trait, and where it follows the context, whoever writes that artwork's context controls the verdict.