High-energy neutrinos (above a few \SI{e16}{eV}) can be detected cost effective with a spare array of radio detector stations installed in polar ice sheets. The technology has been explored successfully in pilot-arrays. A large radio detector is currently being constructed in Greenland with the potential to measure the first cosmogenic neutrino, and an order-of-magnitude more sensitive detector is being planned with IceCube-Gen2. We present the first end-to-end reconstruction of the neutrino energy from radio detector data. NuRadioMC was used to create a large data set of 40 million events of expected radio signals that are generated via the Askaryan effect following a neutrino interaction in the ice for a broad range of neutrino energies between \SI{100}{PeV} and \SI{10}{EeV}. We simulated the voltage traces that would be measured by the five antennas of a shallow detector station in the presence of noise. We designed and trained a deep neural network to determine the shower energy directly from the simulated experimental data and achieve a resolution better than a factor of two (STD < 0.3 in log10(E)) which is below the irreducible uncertainty from inelasticity fluctuations. We present the model architecture and study the dependence of the resolution on event parameters. This method will enable Askaryan detectors to measure the neutrino energy.