It's invested domestically in real estate, art, gold, or other assets, or held offshore, either in the form of freely convertible currency or other assets, which is why 98.96% of the cash was returned.
I strongly doubt a significant number of criminals openly advertised their possession of black money by returning it themselves.
Modi's ploy was an unmitigated failure.