LLMs and Personalities: Inconsistencies Across Scales

We investigated the application of human psychometric assessments to LLMs by administering the Big Five Inventory (BFI) to models of different sizes. Our results reveal large variability due to question order shuffling, challenging the notion of a stable ``personality.'' Notably, models prompted to take up the assistant persona led to more predictable scaling, with larger models exhibiting more socially desirable and less variable traits. In contrast, non-conventional personas displayed unpredictable behaviors.