eICU-KG: A Leakage-Audited Clinical Knowledge Graph Benchmark for Long-Tail Medication-Concept Ranking
Yosef Shammout ⋅ Lihui Liu
Abstract
Medication concepts in ICU records are highly imbalanced, so an overall score can hide poor retrieval on rare concepts. We construct eICU-KG from eICU-CRD after mapping drug names to a fixed vocabulary and splitting records by patient. The benchmark hides one link between a medication event and its normalized drug concept, then asks each method to rank the correct concept from the same fixed set of $K{=}147$ candidates. An explicit leakage audit checks patient disjointness; the data-processing pipeline is deterministic. We define HARD as the test queries whose correct concept falls outside the ten most frequent concepts in the training data. DXPOP-Signature, a non-neural frequency prior based on the complete diagnosis signature recorded for a visit, places the correct concept in the top 10 for approximately 61.7% of the full test set but only 2.4% of HARD. We compare five baseline families: frequency priors, diagnosis-only classifiers, matrix factorization, knowledge-graph embeddings, and a relational graph convolutional network (RGCN). As a limited case study, constrained selective reranking on HARD (CSR-HARD) uses a learned gate to select 7% of HARD queries for RGCN-based reranking. HARD membership depends on the held-out target. CSR-HARD is therefore an evaluation-side analysis rather than a deployable routing policy. Across three gate initializations, CSR-HARD increases the mean number of HARD top-10 hits from 46 to 58.7 among 1,886 queries. This is 12.7 additional hits, or 27.5%, without losing any existing top-10 hits or changing any head-query rankings. A separately implemented additive diagnosis-frequency prior reaches 56 HARD hits. CSR-HARD was not rebuilt with this prior, so this is a cross-method comparison rather than a reranking gain against the additive prior. With 10,000 replicates, the patient bootstrap gives a seed-averaged $\Delta$Hits@10 of 0.0067 and a 95% percentile interval of $[0.0004,0.0136]$.
Chat is not available.
Successful Page Load