I would ordinarily agree, but the Epstein case is exceptional for two reasons.
First - "paying his due" in this case meant a plea deal widely considered to be very lenient. On top of that, the plea deal not only resulted in no useful testimony from Epstein against other criminals, it had an extraordinary provision granting immunity to his co-conspirators! The accusation that the deal was made in bad faith specifically so the government could protect Epstein's conspirators (many of whom were rumored to be prominent politicians, including foreign leaders of US allies) was plausible enough that the US Secretary of Labor actually resigned in July over his involvement in making the deal.
Second - while this dips into conspiratorial territory, there is widespread confusion as to where exactly Epstein's money came from and whether it had anything to do with the sex trafficking. None of the typical players who would be familiar with the deals made by a New York hedge fund had ever worked with him, and he didn't appear to have either a real staff or any real record of investments that such a fund would make. There is a very plausible theory that the money in Epstein's fund came from Epstein getting other billionaires to participate in his sexual abuse ring and then blackmailing them. Needless to say, this makes the idea of taking his money even more problematic.
Much of of this wouldn't have been known to Ito years ago when he first took the money, but it's all out in the open now, and the way the Media Lab has been handling it doesn't reflect well on them.