I'm sorry, quite right - you did. Using the computer locale.

The trouble is that people transmit data from one computer to another and so from one locale to another. And sadly, they do not always set the character encoding header correctly, if they even know what that is.

I mean take csvbase's case. It has to accept csv files from anyone. And christ preserve me, they aren't going to label them as "Win-1252" or "UTF-16" or whatever.

There is no alternative but statistical detection. And there is good evidence that this solution is fairly satisfactory, because millions of computers are using it right now! csvbase uploads run into more problems with trailing commas than with character detection getting it wrong at this point, that is your "Schelling Point" I'm afraid!