Pulsar Candidate Classification with Multimodal Large Language Models

Discovering pulsars is of great scientific value in the field of astronomy. Driven by the huge volume of data from radio telescope survey projects, machine learning (ML) methods have been proposed and widely adopted for this problem. However, existing ML methods rely on a single data modality, either visual or numerical, leading to suboptimal performance. In this paper, we first explore the usage of multimodal large language models (MLLMs) for pulsar candidate classification. Specifically, we propose a novel method called StarWhisper-Pulsar that fine-tunes pre-trained MLLMs using labeled data in visual, textual, and numerical modalities to decide if a candidate is a real pulsar or a non-pulsar noise. We show that StarWhisper-Pulsar outperforms state-of-the-art ML methods for pulsar candidate classification in a few training epochs. These results validate the potential of MLLMs in data-driven astronomical research, paving the way for their broader scientific applications.