Hacker News Reader

Comment by makecheck | Hacker News Reader

If an API has a problem, you fix the API. If necessary, you release a free library to back-port the improved API to older OS versions as well. You come up with a fixed version, make it as convenient as possible, call it the “new standard”, officially deprecate the alternatives, and then write a blog post. Except the blog post would only need two lines of sample code showing how easy it is to work around the problem now.

Not this. Frankly, few developers will even know about the need for careful coding such as this, and even fewer will actually do it because it will muck up each and every program with dozens of lines of extra stuff to work around a deficient part of the PLATFORM.

View on HN

I'm absolutely amazed that Windows doesn't offer a process spawning API that takes an array of strings as arguments[0], if only because that's exactly how a C program expects them anyway.

[0]: https://linux.die.net/man/3/exec

These APIs do exist, they just don't work that way:

> These functions appear to be precisely what we need: they take an arbitrary number of distinct command line arguments and promise to launch a subprocess. Unfortunately and counter-intuitively, these functions do not quote or process these arguments: instead, they’re all concatenated into a single string, with arguments separated spaces

Or in other words, those APIs don't exist, and mentioning a function that has the same C-type but does something totally different is trivia at best.

... which is exactly my point. They apparently exist, and would even appear to work correctly, until some scrutiny is applied.

> They apparently exist[...]until some scrutiny is applied.

But that's the opposite of "does exist"?

When you get to the Windows kernel, the command line is a single PWSTR. Full stop.

Any API or C program main() running on Windows that suggests anything else is fiction - the C runtime parsing what the kernel gave it to a string array on one side, or concatenating into a single string on the other.

Well, it's actually a UNICODE_STRING. ;-) The limit on the length of the command line comes from the range of the Length field of the UNICODE_STRING structure. (NT uses Pascal-style strings internally.)

NT's native process creation functionality is powerful, but baroque: see [1]. There's a ton of stuff that processes can be passed in addition to the command-line. One trick that's not well-known is that CreateProcess allows parent processes to pass an opaque binary blob to subprocesses via the lpReserved2 member of the STARTUPINFO structure. Cygwin uses this blob to pass information about file descriptors, ttys, and other POSIX context; this information block bootstraps Cygwin's fork implementation. The Microsoft C runtime uses it for a vaguely similar purpose: it's how file descriptor inheritance works when neither NT nor Win32 know anything about file descriptors (which are private to libc).

[1] http://www.rohitab.com/discuss/topic/40191-ntcreateuserproce...

[2] https://msdn.microsoft.com/en-us/library/windows/desktop/ms6...

I was a dev in Windows from 2008-2011 so I am not sure you are aware you are replying to someone who is already a big fan of the NT native API (and not so much a fan of the crude hack that is Windows CRT file descriptors, like most things in the MS CRT...) I did mean to type PWSTR in full awareness that I'm using it as a figure of speech for UNICODE_STRING.

Ah, I had no idea. It's sometimes hard to distinguish a lie-to-children from ignorance.

The problem is that parsing the command line isn't done by the operating system at all, but delegated to each individual console application. If you made a new CreateProcess API that took an array of command line arguments, the operating system would need to serialize the array into a single string that could be passed to legacy commands. Unfortunately, there's no way to tell what weird parsing is buried inside the commands, so there will always be gaps in what you can express in the serialized command line.

For example, suppose the console app thinks that apostrophes should be treated as quotes. I pass "'x", "x'" into the new API, and that gets serialized to "'x x'" (two strings to most applications), but this particular app interprets that as one string that says "x x". The OS can't even escape the apostrophes to avoid this, because it doesn't know what language the console application speaks.

PowerShell had to deal with this problem because native command lines need to be rehydrated from its AST before they can be passed to CreateProcess, and (IIRC) they ultimately had to add an operator that means "everything after this point in the command line should be passed to the command verbatim" to cover all of the corner cases from this.

The terrible thing is that there already is an API here that everyone else on UNIX is using successfully - spawn a process with an argv of null-terminated strings and they turn up in the argv of the spawned process.

Microsoft have just chosen not to make it work like that because that would involve admitting they were wrong.

Exactly. While they can't fix the old functions without potentially breaking software that relied on the old behavior, they can introduce a new set of process-launching functions that do the right thing: handle arguments exactly as written, no splitting, no quoting, no metacharacter interpretation.

Then, for new OSes, write a compatibility layer to implement the old functions on top of the new ones, and for old OSes, write a compatibility library to implement the new functions by quoting and passing to the old ones.

Mark the old functions as "deprecated, do not use in new code", and point to the new ones.

This is something that is important. The steps to go about.

1. Fix old solution

2. Implement alternative that is feature complete and document

3. Write comparability layers

If people followed this life as a software developer would be easier on bloodpressure. I remember fondly working with many APIs thst were deprecated but had no alternative and the devs admitted it.

They even copied that broken system into PowerShell "functions" (proper functions are expected to only return explicit results, these return every output they gather, falling behind structured programming).

I've never understood the Win32 platform team's resistance to adding an ArgvToCommandLineW function to mirror the longstanding CommandLineToArgvW[1] function.

[1] https://msdn.microsoft.com/en-us/library/windows/desktop/bb7...

I don't understand your footnote.

Fixed; thanks.

That is a bit more difficult than usual in this case, because it's not just one API, it's two working in unison: the one used by the calling process to pass arguments, and the one used by the called process to receive them.

At present, they're both "everything in one big string"; if an alternative "array of strings" API were added at both ends, you'd need to come up with shims for when the caller passes a string & the callee expects an array, and vice-versa. It's not immediately obvious how you'd do that in a way that works reliably in all cases, especially considering the callee currently has total freedom to parse its command-line however it likes.

Reply on news.ycombinator.com