I require UTF-8. If it isn't currently UTF-8, it's someone else's problem to transform it to UTF-8 first. If they haven't, and I get non-UTF-8 input, I'm fine bailing on that with a "malformed input - please correct" error.
Then it's no longer "someone else's problem".
The article doesn't say that you should. It clearly states that for many cases, the input format is known or explicitly stated in the input headers.
The article talks about cases where the input files are in an unknown input format. Even then, it states: "Perhaps there is a case to be made that csvbase's auto-detection should be a more explicit user interface action than a pre-selected combo-box."
But for the case where the requirements call for heuristics, the article then talks about how that can be done.
> at that point I'm writing an adapter to convert into UTF-8 JIT in front of my service.
And at that point you might need the advice in the article, right?
Right, and at that point you're probably gonna need these statistics-based heuristics to write that adapter. Unless you know specifically what other encoding each bit of input is. If you do, then, again, you are not the target audience for this article.
Sounds like academia. "You" can't pay your bills anyway.