Adversarial Databases Improve Success in Retrieval-based Large Language Models

Retrieval-Augmented Generation (RAG) is utilized to enhance large language model (LLM) performance by leveraging external knowledge databases. While it is generally believed that adversarial databases should negatively impact RAG's effectiveness, we tested this assumption for the first time in the context of the medical subspecialty field of Nephrology. We used several open-source LLMs, including Llama 3, Phi-3, Mixtral 8x7b, Zephyr$\beta$, and Gemma 7B Instruct in a zero-shot RAG pipeline, incorporating both relevant databases (nephSAP and UpToDate) and adversarial databases (Bible and Random Words). Suprisingly, our results show that adversarial Bible and Random Words databases significantly improved Nephrology multiple choice question (MCQ) test-taking ability of specific LLMs. Utilizing DistilBERT's attention outputs, we provide evidence that adversarial databases can potentially affect LLM performance through changes in attention. Our findings highlight the need for further research into the mechanism(s) and generality of the effect of adversarial databases on LLM performance that we have discovered.