Technically he could quit and pay back the Navy, but they've been paying him so poorly that doing so would plunge him into debt.
Furthermore, it is not obvious to me that the government has a use for all the the graduates of all the majors anyway; maybe the State of Tennessee could use a few hundred developers to work on sites and services that are provided/run by the state, but could they find work for all the CS graduates that universities in Tennessee can pump out? Society will value those peoples' skills, so their degrees are not worthless in any meaningful respect, but governments only need so many people in that profession. Maybe you could still hire them and have them do busywork, but that would just be an unnecessary waste of resources that would leave a bad gap in those people's resumes.
In aggregate, maybe. In every case? Certainly not. Spending six figures for someone to get a Ph.d. in Esperanto doesn't seem beneficial to society to me.