In August 2017, the Gulf of Mexico's hypoxic zone was declared to be the largest ever measured. It has been estimated that a 60% decrease in watershed nitrogen (N) loading may be necessary to adequately reduce eutrophication in the Gulf. However, to date there has been no rigorous assessment of the effect of N legacies on achieving water quality goals. In this study, we show that even if agricultural N use became 100% efficient, it would take decades to meet target N loads due to legacy N within the Mississippi River basin. Our results suggest that both long-term commitment and large-scale changes in agricultural management practices will be necessary to decrease Mississippi N loads and to meet current goals for reducing the size of the Gulf hypoxic zone.