We consider the \textit{Bayesian algorithm execution} framework, where the goal is to select points for evaluating an expensive function to best infer a property of interest. By making the key observation that the property of interest for many tasks is a target set of points defined in terms of the function, we derive a simple yet effective and scalable posterior sampling algorithm, termed PS-BAX.Our framework addresses a broad range of problems, including many optimization variants and level-set estimation. Experiments across a diverse set of tasks show that PS-BAX achieves competitive performance against standard baselines, while being significantly faster, simpler to implement, and easily parallelizable. In addition, we show that PS-BAX is asymptotically consistent under mild regularity conditions. Consequently, our work broadens the application scope of posterior sampling and provides a simple, strong baseline for future exploration in this area.