Hacker News Reader

Comment by comex | Hacker News Reader

_Or_ set your application to use UTF-8 for the "A" APIs. Apparently this is supported as of a Windows 10 update from 2019. [1]

[1] https://learn.microsoft.com/en-us/windows/apps/design/global...

View on HN

It should have been supported approximately 20 years earlier than that. I was coding against Win32 looong before 2019 and wondering for years why they wouldn't let you.

An explanation I heard ~10 years prior is that doing so exposed bugs in CRT and nobody wanted to fix them.

Not sure what that has to do with CRT, given that it isn't part of Win32.

CRT in a form of msvcrt.dll file had a de-facto presence in Windows since the end of 1990s. Later on, since 2018 or so, CRT availability was formalized in Windows API in form of ucrtbase.dll module.

Which doesn't change the fact that Win32 doesn't depend on it.

It is extremely hard to create an application that doesn't depend on CRT on Windows. CRT provides tables for handlers of SEH exceptions and provides the default event handlers. Win32 headers have hard dependencies of the handler tables CRT provides. So you need to go quite a bit out of your way to hack deep Win32 headers. Loading DLLs etc also may call CRT functions.

You can read Mingw64 source to see how many hacks they had to do to make it work.

That's the "vcruntime" not the "ucrt". There has been a distinction since the ucrt was made an official part of the OS.

It's very easy to make a win32 program without the ucrt filesytems APIs so long as you don't mind being platform-specific (or making your own cross-platform wrappers).

I have been developing for Microsoft platforms since MS-DOS 3.3, Win16 and Win32 development without any function from standard C library has been a thing for decades, for those doing C development without caring about portability, like demoscene competitions and gaming.

Using C++ is another matter.

msvcrt was never for applications to use: https://devblogs.microsoft.com/oldnewthing/20140411-00/?p=12...

The bundled one with Windows wasn't. However the same "feature" exists in redistributed versions of msvcrt.

It's still an important piece of the app compatibility story.

> An explanation I heard ~10 years prior is that doing so exposed bugs in CRT and nobody wanted to fix them.

What I've heard is that the issue is not with the CRT, but with applications using fixed-size byte buffers. IIRC, converting from UTF-16 to any of the traditional Windows code pages requires at most two bytes for each UTF-16 code unit, while the UTF-8 "code page" can need three bytes. That would lead to buffer overflows in these legacy applications if the "ANSI" code page was changed to UTF-8.

Does that mean that in this UTF-8 mode, GetCommandLineA would, when the full-width double quote occurs in the command line, return the UTF-8 bytes for that double quote, rather than steamrolling it to an ASCII double quote with the WorstFit mapping?

Yes, I wanted to suggest the same. I modified some old tools I wrote 15 years ago to do that a while ago. Not because I was aware of any vulnerability, but because a few places still used char* and I figured this would basically make it never fail with any weird filenames regardless of the code page.

So now it seems even if you think your app is fully Unicode, still do this just in case? :)

It sounds like something Cygwin ought to do across their ecosystem.

> I figured this would basically make it never fail with any weird filenames regardless of the code page.

Windows filenames are not guaranteed to be valid UTF-16 so A functions with UTF-8 code page can still fail to access some files. If you want 100% compatibility you need to realize that Windows is a WTF-16 system and make your own compatibility wrappers for the W functions under that assumption.

UTF-8 ACP might fix these exploits but it doesn't fix the root issue that your application encoding can't represent the whole internal system encoding (WTF-16, NOT UTF-16 despite what it claims).

Reply on news.ycombinator.com