Learning Distributions from Multiple Data Providers

Motivated by learning from heterogeneous and overlapping data providers, we study a stylized model of distribution learning from restricted conditional samples. The goal is to learn an unknown distribution $p$ on a finite domain $[n]$. The learner is given a fixed family of queryable sets $\mathcal S \subseteq 2^{[n]}$, and each query to $S \in \mathcal S$ returns an independent sample from the conditional distribution $p(\cdot \mid S)$. Learnability is governed by the \emph{co-occurrence graph} associated with $\mathcal S$: two domain elements are connected if they appear together in some queryable set. Pointwise consistency is achievable when this graph is connected on the target support. PAC learning requires more: it is possible when the co-occurrence graph is complete. The sample complexity of PAC learning ranges from nearly linear to quadratic. Every query family with complete co-occurrence graph admits sample complexity $\widetilde O(n^2/\epsilon^2)$, and this bound is tight in the worst case. On the other hand, if every subset is queryable, the optimal worst-case complexity improves to $\Theta(n/\epsilon^2)$. More generally, we identify \emph{hierarchical comparability} as a sufficient structural condition on $\mathcal S$ under which the optimal complexity is nearly linear, $\widetilde \Theta(n/\epsilon^2)$, with pairwise query families as a canonical example. Finally, the full range of polynomial rates between linear and quadratic is attainable: for every $\alpha \in (1,2)$, there exists a query family with optimal PAC rate $\widetilde \Theta(n^\alpha/\epsilon^2)$.