As more and more AI systems are deployed in the real world, it becomes imperative to study these systems with real humans to avoid unexpected negative consequences during deployment. Yet, this can be challenging for researchers with more experience designing algorithms and less experience running human participant experiments, or deploying systems in the real world. In this tutorial, we will discuss the state of the human-AI collaboration field, emphasizing (i) incorporating humans into AI systems, including multi-agent, machine learning, and reinforcement learning systems, (ii) investigating when to rely on human vs. AI strengths, and (iii) designing human-AI studies to evaluate algorithms with real humans.