How do brain circuits learn to guide our behaviour? While significant strides have been made in understanding learning in artificial neural networks, applying this knowledge to biological networks remains challenging. For instance, while backpropagation is known to perform accurate credit assignment of error, how a similarly powerful process can be realized within the constraints of biological circuits remains largely unclear. One of the major challenges is that the brain's extensive recurrent connectivity requires the propagation of error through both space and time, a problem that is notoriously difficult to solve in vanilla recurrent neural networks. Moreover, the extensive feedback connections in the brain are known to influence and control forward network activity, but the interaction between the feedback-driven activity changes with weight-based learning is less well understood. Inspired by previous work on motor control models, this work investigates the mechanistic properties of networks pre-trained with feedback control on a stereotypical motor task. We show that the feedback control of the ongoing recurrent network dynamics approximates the optimal first-order gradient w.r.t. the network activities, allowing for rapid, ongoing movement correction. Moreover, we show that the network adaptation to task perturbation using a local, biologically plausible learning rule is both more accurate and more efficient with feedback control during learning, due to the decoupling of the network dynamics and injection of adaptive, second-order gradient into the network dynamics. Thus, our results suggest that feedback control may guide credit assignment in biological circuits, enabling both rapid and efficient learning in the brain.