Graph-Attentive Meta-Agents for Supervising Serverless Worker-Agent Pools
Abstract
Event-driven serverless platforms execute millions of concurrent workflows expressed as directed acyclic graphs (DAGs). The pools of workers that execute them are increasingly managed not by hand-tuned controllers but by a meta-agent: a higher-order policy that supervises, throttles, and reassigns the worker agents beneath it. This makes the meta-agent’s own generalisation a reliability property of the platform. We study one concrete failure of generalisation: because tenants continuously edit workflow topology, a meta-agent whose state encoder assumes a fixed input dimension breaks the moment a DAG changes shape. We present MAGE, a graph-attentive meta-agent that encodes each tenant DAG with a multi-head graph attention network and mean-attention pooling, producing a fixed-width embedding from an arbitrary topology, and trains regional meta-agents with centralised-critic MAPPO to set concurrency limits and cross-pool offload ratios. On a simulated serverless engine, MAGE lowers p95 flow latency by 38% against the strongest hand-tuned baseline and by 27% against an MLP-based MARL orchestrator, and cuts peak worker-thread saturation by 47% under injected webhook degradation (from 92% to 49%). Unlike a fixed-width MLP orchestrator, it keeps operating on workflows up to 4× larger than any seen in training without retraining, with a 13.9% latency gap to a policy retrained at that scale. Against other variable-size encoders (DeepSets, a Decima-style GNN) the margins are smaller, and we say which of them exceed seed noise. Because a supervising agent that mis-throttles can starve traffic platform-wide, we pair the learned policy with an explicit oversight layer: a clamped action envelope, per-tenant fairness floors, shadow-mode staging, and single-command rollback. We report where that envelope binds, and argue this reporting should be standard for deployed meta-agents.