Abstract: Diffusion-based purification has shown impressive robustness as an emerging adversarial defense. However, concerns have been raised regarding whether this robustness results from potentially insufficient evaluation. Our research reveals that EoT-based $N+1$ attacks encounter limitations due to the gradient dilemma phenomenon, which leads to an underestimation of the threat posed by resubmit attacks. To address this, we propose a sufficient and efficient attack named DiffHammer. We advocate the use of $N$-time evaluations to accurately quantify risk in practice and to enhance attack efficiency. We further bolster DiffHammer with an EM-based attack that bypasses the gradient dilemma by identifying and attacking vulnerable purification clusters. Our comprehensive experiments validate that DiffHammer can discover more at-least-once adversarial samples with a $2\times$ speedup. By mitigating the impact of the gradient dilemma on evaluation, the reliability of diffusion-based purification is called into question.