No need to go to the kernel API to create filenames that are invalid UTF-16. The Win32 API will happily let you do it.
Just made two files named ä.txt which happily sat next to each other, one being NFC and other NFD.
So yeah, don't mess with the normalization of filenames.
I don't use Windows, so I can't check. Linux literally allows any arbitrary byte except for 0x00 and 0x2F ('/' in ASCII/UTF-8). It's a problem for programming languages that want to only use valid Unicode strings, like Python. Rust has a separate type "OsString" to handle that, with either lossy conversion to "String" or a conversion method that can fail. Python uses the custom use Unicode range to represent invalid byte sequences in filenames. It's all a mess. JavaScript doesn't give a damn about the validity of their UTF-16 strings.
(Note that Rust's OsString is different from it's CString type. Well, I guess under Unix they're the same, but under Windows OsString is UTF-16 (or "WTF-16", because it isn't actually valid UTF-16 in all cases).)
I tried using U+13161 EGYPTIAN HIEROGLYPH G029[1], which resulted in a string of length 2 as expected.
Using both chars (code units) and just the first char (code unit) worked equally fine. In Windows Explorer the first one shows the stork as expected, while the second shows that "invalid character" rectangle.
So yeah, treating filenames as nearly-opaque byte sequences is probably the best approach.
Äh, nu går vi och gör något annat.
Win32/NTFS treats them as two different filenames, as it doesn't normalize them before storing/comparing.