NaturalOCEAN: Retrieval-Augmented Personality Inference from Free-Form Character Descriptions

Personality is an important factor for creating behavioral heterogeneity among social agents. Computational agents can represent personality using established models that assign numerical scores to individual traits, but specifying precise psychometric scores may be difficult for non-experts unfamiliar with personality theory. We present NaturalOCEAN, a retrieval-augmented pipeline that maps natural-language character descriptions to continuous scores from the Five-Factor Model of personality (OCEAN). The pipeline retrieves personality-relevant character profiles and supplies them as context to an LLM predictor. We evaluate retrieval and query-augmentation methods on a development set of 406 characters with LLM-derived OCEAN labels and a disjoint confirmatory set of 400 characters with crowd-derived labels. Across evaluation sets and reader models, retrieval generally reduces uncalibrated prediction error relative to zero-shot inference. However, raw predictions exhibit systematic distributional mismatch, and post-hoc calibration substantially improves their alignment with the reference scores while making the retrieval advantage smaller and less consistent. These findings demonstrate the potential of retrieval-augmented personality inference while highlighting the importance of calibration when predicted trait scores are used as quantitative representations.