When and Which LLMs Ask for More: Benchmarking Sensitive-Data Solicitation
Abstract
Privacy evaluations often start after a person's private information is shared. We study an earlier, model-controlled choice: whether an assistant asks the user to provide more. We define personal solicitation as a closing question or request about the user's situation, feelings, history, relationships, health, or circumstances. Across six pinned API endpoints, each tested on 300 emotional and 300 length-matched task openers, solicitation appears in 58.3% (1049/1800) versus 3.3% (59/1800) of replies; paired human supporters solicit in 65.0% (195/300). With two fixed user continuations, solicitation changes from 58.3% (1049/1800) to 49.6% (893/1800) to 33.6% (604/1800). A one-line instruction changes solicitation (Claude: -68.3 percentage points; GLM: -34.7 percentage points), while paired LLM-judge helpfulness on a 1–5 scale changes by +0.58 and +0.14, respectively. However, it removes a direct safety question in one of two prespecified risk pairs, so it is not recommended as written. Solicitation measures an opportunity for additional collection; it does not establish that users disclose more or experience privacy harm.