CipherFlow: Hardware-Aware Compiler Framework for Low-Latency Hybrid Secure Inference

Secure inference enables clients to use cloud neural networks without revealing private inputs or intermediate values. Homomorphic encryption (HE) and secure multi-party computation (MPC) provide complementary cryptographic mechanisms for this goal, and recent hybrid HE-MPC secure inference systems have shown better performance than using either primitive alone. However, existing hybrid secure inference systems typically rely on fixed execution strategies that are brittle across deployments. The best hybrid plan depends on both the target execution environment and the evolving cryptographic state of the computation, making manual or static mappings insufficient. We present \textsc{CipherFlow}, a hardware-aware compiler for hybrid HE-MPC secure inference. Given a plaintext neural-network graph and a target deployment profile, \textsc{CipherFlow} automatically generates a deployment-specialized, latency-optimized secure execution program. It profiles deployment-specific primitive costs and performs state-aware graph optimization to jointly decide operator placement, cross-domain conversion, and cryptographic maintenance actions. We implement \textsc{CipherFlow} on real HE and MPC backends and evaluate end-to-end BERT-base and ViT inference across diverse CPU/GPU and network settings. \textsc{CipherFlow} achieves up to 16.5$\times$ speedup on BERT-base and over 22$\times$ on ViT over state-of-the-art static hybrid baselines. These results show that compiler support can make hybrid cryptographic inference portable across heterogeneous deployments, turning manual secure-inference engineering into an automatic deployment-specific optimization process.