Predicting the evolution of chaotic systems is notoriously hard. These systems are ubiquitous in astronomy, the most well known being the gravitational N-body problem. There has been increasing efforts to develop machine learning (ML) methods to predict the evolution of such systems, with the goal of speeding up simulations. In these setting, Active Learning (AL) is often used to improve the performance of models. Here we use the Sitnikov three-body problem, the simplest case of N-body problem that is capable of chaotic behavior, to illustrate that AL may fail to improve training performance in these conditions, likely due to the fractal nature of the decision boundary. This is an important result for astronomers planning to optimize large sets of N-body simulations via AL in concrete applications, such as e.g. the simulation of star clusters with the goal of constraining gravitational wave emission and for applications in other fields involving chaotic systems.