AI Governance Should Prioritize Control and Knowledge Boundaries Over Limiting Intelligence
Abstract
The rapid proliferation of autonomously acting AI agents makes the governance of agentic systems urgent. We propose a competence circuit, summarized as IKC + KC + C, that decomposes an agent's real-world impact into three pathways: intelligence-mediated discovery, procedural know-how, and direct control. Using the canonical fast-takeoff debate as a stress test, we map catastrophic takeoff to a conjunction of six necessary conditions, showing that most rely on rapid knowledge acquisition or control attainment rather than intelligence alone. We operationalize epistemic intelligence as hypothesis-selection efficiency and validate this framework via stochastic simulation on NK fitness landscapes across six scenarios spanning the conjunctive chain. An analytic bound supported by the simulations shows that intelligence-driven speedup is capped by the ratio of competing intelligence levels regardless of task difficulty. The simulations also expose a fast-takeoff dilemma: a large intelligence-driven speedup multiplier materializes only when the hypothesis space is noisy, which is precisely the regime where each test incurs irreducible time cost, so a large multiplier and a short absolute timescale cannot both obtain. Simulations also indicate that institutional chokepoints disproportionately impact superintelligent agents, leaving them less unproductive search time to absorb mandatory authorization delays. We argue that the dominant safety levers are restricting control and enforcing knowledge boundaries rather than limiting intelligence alone, and this applies to agentic systems at any capability level. Purely digital domains weaken these bottlenecks, strengthening the case for architectures that intentionally "put physics in the loop."