To me the most complicated thing about formatting and parsing floating point numbers is usually if it uses decimal comma or decimal dot. It never seems to be what you expect if using a mix of different language operating systems.
You're probably going to do something stupid like showing the user a comma-separated list of numbers at some point, which will be needlessly hard to parse for a human when your numbers use a comma as a decimal separator. You (or someone else, or your users if they are technical) will probably at some point make something which tries to parse some output, and that will break if you switch between points and commas arbitrarily. Your users will want to copy a number your software prints and paste it into a calculator or REPL or something, and that probably doesn't work with comma as a decimal separator.
Half-assed "localization" from people who don't know anything about how other countries work is just needlessly annoying to be subjected to.
That's at least my perspective as a Norwegian who experiences a lot of _bad_ localization even though I know English fairly well and configure all my computing devices to use English. The perspective of someone from a country where English is less well known might be different.
<rant>
Examples of horrible localization from clueless American companies or organizations include:
* A lot of software will use your IP address to determine your language. That's annoying when I'm in Norway and want my computers to use English, but it's horrible when abroad. No, Google, I don't want French text just because I'm staying in France for a bit.
* Software will translate error messages, but not provide an error code. All information about error messages online is in English on stackoverflow or whatever. If Debian prints an error message in Norwegian, there's absolutely no information about the error anywhere on the web.
* There was a trend for a while where websites would tick the "localization" checkbox by adding a Google Translate widget, so English websites would automatically translate themselves into completely broken Norwegian automatically. That would've been useless if I didn't know English, and it's even worse considering I already know the source language just as well as Norwegian. Luckily, most websites seem to have stopped doing that.
</rant>
Resist the temptation to use commas or dots as thousand-separators. Seeing a number with a dot as a decimal separator instead of a comma will be fine for most people (even if proper localisation would mean using a comma), but if you throw in commas that mean something else you WILL confuse people. And I imagine the inverse is also true.
I personally don't like it though, and tend to prefer either the swiss system (ie, 3'800'000.0), or maritime system (ie 3_800_000.0) if separators must be used