Fractal-G: Topology-Aware Heterogeneous Graphs for Medical Image Segmentation

Medical image segmentation requires the precise alignment of macroscopic semantics and microscopic geometry, yet existing representation paradigms struggle to balance massive low-frequency semantic context with ultra-sparse, high-frequency boundary details. To address this fundamental information asymmetry, we introduce Fractal-G, a plug-and-play multi-scale fusion module that presents a continuous Grid-Graph-Grid feature reconstruction paradigm. Within this framework, an Uncertainty-Aware Fission Router dynamically translates regular image grids into a non-uniform heterogeneous graph, adaptively allocating dense microscopic nodes to complex boundaries while retaining sparse macroscopic nodes in homogeneous background regions. To ensure unbiased feature aggregation across these varying physical scales, an Area-Aware Continuous Neighborhood Graph explicitly incorporates physical node sizes into the topological message passing. Finally, a Topology-Aware Implicit Renderer projects the unstructured graph back into dense, artifact-free continuous feature fields. Extensive experiments on diverse medical image segmentation tasks (including skin lesions and polyps) demonstrate that Fractal-G easily integrable with mainstream backbones, consistently achieving state-of-the-art geometric fidelity and cross-resolution robustness with only 12\% GFLOPs increase (U-Net based). Our code is publicly available at https://anonymous.4open.science/r/Fractal-G/.