Improving Vision-LLMs with Human Cognitive Signals
Angela Lopez Cardona · Sergi Abadal · Ioannis Arapakis
Abstract
We propose to collect eye-tracking during preference annotation to build a Vision–Large Language Model (VLLM) alignment dataset and to train a saliency predictor of human attention. We will compare real versus synthetic gaze supervision and evaluate attention editing together with Direct Preference Optimization (DPO) to reduce hallucinations and improve human alignment.
Successful Page Load