Microsoft_craziness.h (2018)
gist.github.com
gist.github.com
If you compile a C++ program with Visual Studio, it gets dynamic dependencies on some visual studio DLLs. DLLs that are not necessarily installed by default on the Windows of the target user you're compiling for.
So the question is: MS created both Windows and Visual Studio. So why can't Visual Studio compile to something that works by default on the Windows of the target audience without requiring them to install those additional DLLs? Doesn't the win32 API provide enough, and can't Visual Studio statically link in whatever extras it needs, or those extras be added to the Windows API that is present by default if they're that necessary?
Also, given that Windows comes with pre-installed programs that were written in C++, how did those programs get compiled to not require those Visual Studio dependencies, are MS not using Visual Studio themselves internally?
It can but you have to change some options (`/MT` iirc). And if you want to be really minimalist you only need to statically link the runtime (if you avoid using the C++ standard library).
For similar reasons, you can't have static libraries that are used by higher-level dynamic libraries (e.g. gRPC dynamically linked with protobuf statically linked). In the end, the easiest solution is usually to pick static linking or dynamic linking and use it for everything, although it's possible not to if you're really careful.
Having said that, statically linking everything, including the Visual Studio runtime, works really well and I'd recommend it.
visual studio DLLs.
I assume you mean the C runtime etc, which isn't tied to VS but to 'build tools' i.e. compiler/linker. VS just uses those. It can also use other non-MS compiler/linker.
So why can't Visual Studio compile to something that works by default
It largely can, see next answer.
on the Windows of the target audience without requiring them to install those additional DLLs?
I think (not 100% sure though) because there were historically different versions of those. And as said possibly from different compiler vendors. See also https://stackoverflow.com/questions/16167305/why-does-my-app...
But: things changed somewhat since then though, there's now a 'universal C runtime' which gets installed with Windows 10 and which covers versions of the runtime used per default by the built tools which ship with VS2015 and beyond. Which given the history is quite a piece of work they pulled off.
how did those programs get compiled to not require those Visual Studio dependencies,
Static linking.
First-party toolkit stuff like this should be built into the OS.
But when you install them manually, they are serviced for security updates through Windows Update, so that you don't have to do that manually.
C and C++ runtime libraries aren't an OS API on non-UNIX platforms, each compiler vendor ships their own libraries.
Windows 10 now has C universal runtime library as part of the OS, but that is because nowadays almost everyone only cares about Visual C++ compatibility.
You can have your binaries without extra dependencies, code against Win32 in C without any dependency on ISO C or ISO C++ calls, that is how demoscene competitions work.
Why?
https://github.com/microsoft/STL/issues/1814
https://devblogs.microsoft.com/cppblog/msvc-cpp20-and-the-st...
Since Windows 10 it ships with Universal C Runtime, which is used by VS2015+. System software uses these runtimes too. For Win7+ it was added as system update, though MS later relented and said it was fine to copy-paste all the api-ms* libraries from the SDK into your application directory so it also works on outdated installations.
In the past it was different though, as system software was compiled against MSVCRT.DLL, but you wasn't supposed to use it and instead use version specific DLLs shipped in "Visual C++ Redistributables". Though going back to Windows 9x there was a time when MSVCRT.DLL was the library that was supposed to be used by applications. See http://bytepointer.com/resources/old_new_thing/20140411_092_...
Games still require vc_redist.x86.exe or vc_redist.x64.exe to be installed, even on Windows 10. Ancient games also require ancient versions of these to be installed.
I have always found it very strange that these wasnt just included in Windows.
- The C runtime library (now `ucrt.lib` AKA the universal C runtime) - The compiler runtime (now `vcruntime.lib`)
The former has a stable API, but the latter doesn't - it changes from one compiler version to the next, as it is used to support various C++ language features.
When these two were combined as `msvcrt.lib`, there was no way for windows to ship it with the OS, because it changed with each VC++ version - they could have shipped historical versions, but it wouldn't really help developers who can't guarantee that the the OS version used by their users is newer than their version of VC++.
Now that these two are split, microsoft can ship the stable universal C runtime with the OS, and programs that only rely on the C runtime do not need a redistributable at all. (This makes things easier for other languages like Rust, or things compiled from GCC, which don't need the VC++ runtime).
Programs that use the VC++ runtime still need to ship the corresponding `vcruntime`, although you still have plenty of freedom in how you do that (eg. bundle DLLs, statically link, etc.) and the compiler runtime itself is smaller than the whole C runtime library.
> Also, given that Windows comes with pre-installed programs that were written in C++, how did those programs get compiled to not require those Visual Studio dependencies, are MS not using Visual Studio themselves internally?
They do, iirc Windows pretty much always came with "unspecified" versions of the MSVCRT "for internal use only". Also, iirc, at some point Windows took some steps to remove those from the default search paths or something.
Anyway, my experience with VC++ and the packages is that you get better control when shipping the dependencies separately. You can resolve the dependencies during installation but it is just more complex.
I feel like VS' GUI is somewhat decent
Along with that, the states of the UI components in VS are sometimes not correct. For instance, the find feature. It changes constantly to Search within open files or. Something nobody ever want to use.
The go forward, go back, feature breaks all the time. I do not even understand how to fail to implement this. Or how to break it. It is basically stack.
...
Do you think VS designers should accommodate your particular needs and desire of a particular shortcut over the needs of all other users who participate in their beta and user feedback and instrumentation data?
You can either:
* Statically link, so you don't need these DLLs at all
* Just copy the DLLs along with your app in the same folder
* Ask users to install the VC Redist package
Option #2 is typically the best one and is pretty easy to do. macOS typically does #2 implicitly (i.e. libswift.framework et al), but ships all of those binaries in a "bundle" that is hidden from users.
I think the license precludes distributing the DLL's with your application. You are supposed to use the VC Redist package.
Your app's manifest is more important than you realise and I wonder if the OP is aware of this? Its SxS not SxSW. :-)
This might be useful as well https://docs.microsoft.com/en-us/windows/win32/dlls/dynamic-...
IIRC Windows has been UTF-16 for ages which this link suggests has been since 2000. https://en.wikipedia.org/wiki/Unicode_in_Microsoft_Windows
I always treat UTF-8 as internet territory like web pages because its reportedly easier to recover from corruption which occurs more often online than with the components inside a computer.
The C++ library yes. However IIRC Windows now ships a conforming somehow-modern C standard library (ucrt) and modern MSVCs link to it. It is probably also included in current redistributables for the still supported (by MSVC) old versions of Windows that did not have it. Also, the forever binary incompatible C++ standard lib had its limit regardless of the platform (more because of 3rd party binary libs) and MS has switched to a backward binary compatible C++ redistributable for a few years now (they may rebreak in the future).
So on some points (but not all, and it is likely to never be all) the situation is converging.
> to avoid the classic nonsense that every Linux user sees when they build a binary and bring it to an older OS
The thing with building a binary-only program for "Linux" is that "Linux" is not a single OS, and you don't really have cross-distro binary compat except by luck, and for the different versions of a same distro they concentrate on rebuilding everything from source, plus you have a unified namespace of symbols in ELFs so you would not be able to load 2 different C++ libs anyway. This brings drawbacks, but also advantages (for ex you can use C++ libraries of the platform, with an unconstrained C++ API)
So the end result is kind of what you say, except if you want things to work really well you should probably even do:
* a specific native package for each distro (maybe × version) * try to dynamically link to the fewest possible libs of the platform, ideally only glibc (and yes, an old enough one), maybe also libstdc++ if you want to support plugins, and maybe others if they come with a compatibility policy (so on a case by case basis) * or target something else than a native distro (ex: flatpak, appimage), although it is sometimes kind of equivalent to yet other distros -- anyway at this point it starts to resemble to how you would distribute a binary for Windows -- so with those possibilities existing it is just that under a GNU/Linux distro you also have the classic/more-integrated ones.
Anyway, Chrome manages to do binary packages for various version of Debian, and probably other distros; likewise for VS Code and Virtual Box. And building for multiple targets is arguably less a pain today than 10 years ago.
On macOS libSystem.dylib provides the C runtime (i.e. libc). It cannot be statically linked, and AFAIU you cannot ship it with your own project.[1] To support older environments you declare an ABI/API target using MACOSX_DEPLOYMENT_TARGET, -miphoneos-version-min, or similar toolchain option. It's normative for projects targeting an Apple platform to explicitly declare a target several releases in the past, but without an explicit declaration the default is to build a binary targeting the version of the SDK you're using, which would normally be the most recently released.
Bundling your own dynamic libraries is trivial on Linux by simply defining the run-time search path (aka rpath) when building your application. In general, the systems's dynamic linker is entirely agnostic. And this is even easier on Linux than most other platforms as you can use the $ORIGIN macro to define paths relative to the main binary rather than using fixed absolute paths. $ORIGIN was supported several years before the similar capability on macOS, @loader_path and @executable_path.
Bundling dynamic libraries is extremely simple on Linux, it's just not common.
Linking to an older glibc ABI is also possible--glibc rigorously utilizes ELF symbol versioning--but admittedly very difficult. None of the build toolchains provide built-in facilities (i.e. not even anything similar to the musl-gcc wrapper for cross-compiling against musl on glibc-based systems), and for the most part glibc only versions its run-time binary symbols, not its compile-time header API. That means you basically need to download old glibc headers or, as most people tend to do, build on old distributions.
OTOH, on Linux you can at least statically link your own libc on account of the kernel's [relatively] stable ABI.
In practice neither Linux, macOS, nor Windows provide an ideal situation. But at least in principle Linux supports all the common permutations anyone may want, excepting the absence of any singularly blessed system libc. (But to reiterate both musl libc and especially glibc provide strong backwards ABI compatibility, notwithstanding the absence of compile-time knobs for selecting older ABIs.)
[1] Perhaps you can hack things to load a bundled libSystem copy. I have a vague memory of someone pointing that out on HN before. But in any event that's definitely not a supported feature.
Gosh I wish things worked like this by default on Linux (dynamic linker search path by default starts with the directory where the binary is located)
Because msvcrt.dll must be compatible with MSVC6 or something ridiculous like that (the amd64 version probably needs to be ABI compatible with a different version, because MSVC6 didn't have amd64). They want to maintain backwards compatibility with existing software, so they can't change the ABI around. You might think, who cares about old software? Well, last I checked MinGW links against msvcrt.dll as well, so to provide compatibility with them (and probably software built today with the MinGW toolchain) they still need to keep it around and compatible.
This is analogous to running with really old glibc… even that had a soname bump (it's not at 1 anymore). Can't exactly add symbol versioning either, since 1) it needs to support older versions of Windows, 2) people could totally roll their own dynamic loader… PE isn't exactly secret.
For your last paragraph, as I understand it the Windows team basically uses their own fork of the toolchain, stuck bring binary compatible…
Not without a time machine; the problem is that you can target older versions of windows with newer versions of Visual Studio. When Microsoft produces vc_redist_2022.exe they can put it Windows 11, but how do they put it in the 2018 version of Windows 10?
The arrival of the Windows Store raises the possibility of automatically installing dependencies. You can also create an "unpackaged" app that uses the WinRT APIs without being a Windows Store app distributed through the store, but if you do that you're exposed to all the horrific machinery yourself as a developer: https://docs.microsoft.com/en-us/windows/apps/windows-app-sd...
In re static linking: I haven't checked but I suspect that COM is involved somewhere. Does anyone have a good summary of what's actually _in_ the redistributable?
https://docs.microsoft.com/en-us/windows/win32/sbscs/creatin...
Unfortunately, like in every best practices from Microsoft, some teams are in deep need of internal advocacy.
As for the original craziness header, I found it hilariously, but pretty typically, harsh and unfair. Finding the compiler and all is environment is not an easy task in any OS. I've been programming for a looooong time I remember havign to write code that compiled on: IRIX, Solaris, HpUx, IBM unix, Windows and more.
Anyone who ever took a look at configure.sh knows the horrors it holds at bay.
There is no "necessarily" about it. They are not installed by Windows, period. If they exist on the user's machine, it's because other applications have installed them, for themselves. Applications should have these libraries in their own folders, and those libraries are the specific version the applications were built with.
When you ship an application build with the Microsoft toolchain, you ship the required redistributable libraries in your installer.
Using libraries on Windows is a cinch, because Windows does the smart thing: when an executable needs a DLL, it first looks in the same directory as that executable. Thus your installer can just throw your exe's and dll's into one folder and be done with it.
There is a new thing in Windows: the UCRT (universal C run-time library). It's an attempt to provide something similar to a libc in the Unix-like world: a stable library in the OS, so applications don't have to ship one. Or something of that nature.
Windows has a C library for its own utilities and middleware called MSVCRT.DLL. That is off-limits to applications though. This new UCRT is documented for application use.
Such a seemingly simple problem and yet we have two very complicated solutions. Just goes to show how convoluted things can get at Microsoft scale.
Simple ecosystems and simple software usually have the same thing in common: they are either young, or they've recently survived an extinction-level-event. Older systems with many dynamic constraints breed complex webs.
At the risk of digression, I wish some of the "small government" crowd would realize this.
I have VS installed on my system, yet vswhere is not in PATH.
The is true no matter where Visual Studio is installed to.
This is a pattern that bothers me with Microsoft software on Windows. They leave the stuff they needed to use ALSO for installer stuff in an installer directory, not under my path, not even in a place I might know of. You could have said
%APPDATA%\VisualStudio\VC2932.2.1.21113412\1231231\a31\Internal\Installer
and it would have been just as easy for me to find.
The irony is that there is absolutely no guarantee that this path is stable so in 2025 I'm sure there will be a wherevswhere.exe
https://github.com/Microsoft/vswhere/wiki/Installing
> This is a fixed location that will be maintained.
It's not so much a problem of "scale" as simply poor architecture, neglected technical debt and outright incompetance
The reason I've stayed away from MS is that it's always felt inelegant to me. Not only are the systems clunky but the code just looks so ugly to me.
AnExtremelyLongFunctionNameThatWrapsTwiceOnTheScreen(MsI.BillGates.IncrementValueOnlyIfNotInWindows95).
It is not an elegant system by any stretch of the imagination. Same goes for Powershell.
find_windows_kit_root_with_key
find_kit
Windows.find_kit_root(key)
And is even better if your language supports pattern match on the parameters like elixir, so you can "overload" the function with many types of arguments, eg:
Windows.find_kit_root(%User{key: key})
Windows.find_kit_root(%Admin{key: key})
Providing different behaviors to the same function name
https://github.com/ziglang/zig/blob/master/src/windows_sdk.c...
What is the author referring to here?
That’s why on windows Rust’s OsStr is an extension of UTF8 called WTF8, which allows surrogates (then again “traditional” unix paths are just bags of bytes so it’s not like windows is any worse).
So in practice the only way invalid UTF-16 shows up is from malicious programs (or people testing their handling of non-unicode).
I doubt even really old NT programs (pre 2000) using UCS-2 would have included unpaired surrogates because IIRC those code points were never mapped to actual characters so I don't think that's an issue in practice.
Lone surrogates regularly show up in any environment which allows them.
MS's approach is better in the case where we're all good stewards of the filesystem but the edges are sharp when someone isn't.
CMD /c vcvarsall.bat x64 && set
Then dump the VC specific environment variables in a file and you are good.
I implemented this in tundra a very nice and fast build system. It's in Lua... you can look at it here https://github.com/deplinenoise/tundra/blob/master/scripts/t...
The problem this code is solving is equivalent to finding vcvarsall.bat in the first place.
Also, if you want to run that command you need to run it from the developer command prompt. Otherwise it's location isn't in your path but like I said this script is always in the same location and what it does is that it sets up the environment for you and it's configurable via parameters.
I am well aware that you can use the developer command prompt (in which case you don't need to run that command at all, it would be redundant). But it's a ridiculous and extremely annoying requirement to impose on your users.
// The documentation says that if the string for some reason was not stored with zero-termination, we need to manually terminate it. Sigh!
if (value[length]) { value[length+1] = 0; }
wchar_t is also more than one byte, so the buffer is at most half as big as the code appears to expect.
Source seems to take into account the buffer overflow:
DWORD length = required_length + 2; // The +2 is for the maybe optional zero later on. Probably we are over-allocating.It might cause an OOB write though, with a data race on the registry key (if the key's value happens to grow in length by a char or two between the calls, time of check time of use yada yada).
No, because `RegQueryValueExW` will return ERROR_MORE_DATA and the code bails out on error (also leaking the memory).
2. length is set to 12
3. external change causes the value to now be 12
4. second call to ``RegQueryValueExW` succeeds, as 12 <= 12, no ERROR_MORE_DATA here; length stays 12
5. length + 1 and length + 2 are now OOB
A pointer to a variable that specifies the size of the buffer pointed to by the lpData parameter, in bytes. When the function returns, this variable contains the size of the data copied to lpData.
In other words, it's rewritten to be the actual length before it's incremented for null termination.
DWORD required_length;
auto rc = RegQueryValueExW(key, version, NULL, NULL, NULL, &required_length);
if (rc != 0) return NULL;
DWORD length = required_length + 2; // The +2 is for the maybe optional zero later on. Probably we are over-allocating.
wchar_t *value = (wchar_t *)malloc(length);
if (!value) return NULL;
rc = RegQueryValueExW(key, version, NULL, NULL, (LPBYTE)value, &length); // We know that version is zero-terminated...
if (rc != 0) return NULL;
// The documentation says that if the string for some reason was not stored
// with zero-termination, we need to manually terminate it. Sigh!!
if (value[length]) {
value[length+1] = 0;
}
`length` starts at 2 greater than naively needed, then is updated to the real value (again, always 2 less I guess).EDIT: I develop on unix systems, not windows, so I'm rusty here ... looking up RegQueryValueExW() the `length` ("lpcbData") is in bytes so everything goes fine until `value[length]` which is indexing by wchar_t (16-bit on windows I guess, 32-bit on linux systems) so it's double the byte offset intended, that's way out.
Don't poke beyond your end. Don't poke using a value that was returned by a function you don't control. The code shown does both. Such quality, it Jonathan Blow.
> NOTE(Kalinovcic): I have translated the original implementation to C
Another point, this is a build-time tool: some assumptions of good-faith input are reasonable and necessary. If an attacker can modify paths to visual studio components in your registry, you have bigger problems (just running the attacker's code directly regardless of safe string handling).
> // is stupid, and the actual maneuvers we need to go through
> // are just painful. If programming were like this all the time,
> // I would quit.
Yup. Exactly why I refuse to ever work on Windows again, and won't even bother targeting it from other platforms. Total waste of my time and talent.
#include "Microsoft_craziness.h"`
Then in a c/cpp file: #define MICROSOFT_CRAZINESS_IMPLEMENTATION
#include "Microsoft_craziness.h"
The define causes the actual functions to be put in, while without it only the declarations are included.It can't be part of an end-user application, because looking for Visual Studio on an end user's machine is futile.
Maybe if you're writing some kind of extension for Visual Studio?
The comments in the header point to some lack of understanding on the author's part:
// HOW TO USE THIS CODE
//
// The purpose of this file is to find the folders that contain libraries
// you may need to link against, on Windows, if you are linking with any
// compiled C or C++ code. This will be necessary for many non-C++ programming
// language environments that want to provide compatibility.
//
This needs to clarify the exact situation. Because the normal end-user application cannot be looking for Visual Studio materials. It is completely wrong-headed. They either do not exist, or else were installed by some other application, almost certainly in its own folder. (On this side of year 2000, no application installer should be putting libraries into a system folder.)The folders that contain libraries you may need to link against are found by default; you don't have to tell your program where to look for user32.dll or kernel32.dll.
Any non-system library is in your own folder: you ship that library with your program, and it goes into C:\Program Files\YourProgram\bin or whatever.
Any .DLL files which are in the same folder as your .EXE will be found by it automatically; there is no need to write code to look for anything.
// We find the place where the Visual Studio libraries live (for example,
That place is nowhere, on 99.999% of the installed Windows machines on the planet.
Unless you mean some other application's C\Program Files\OtherApplication\bin folder, where you have no business looking or using.
// libvcruntime.lib), where the linker and compiler executables live
// (for example, link.exe), and where the Windows SDK libraries reside
// (kernel32.lib, libucrt.lib).
kernel32.lib is a kind of liking stub; the library per se is kernel32.dll, which is a system component..lib files are not even required for using .dll files; for instance, you can use .dll files via dynamic FFI in various languages, without a .lib file.
Projects built with Visual Studio generally not have to worry where kernel32.lib has been installed; the tooling takes care of that. Programmers don't have to specify that themselves in project files.
It could be a concern if you use some external build system; it would then need some configuration mechanism to point various toolchain-related variables at all the correct directories. A program using "microsoft_craziness.h" could be built which runs on the that build machine and produces a batch file full of environment variable assignments, or whatever. That I can see.
I can sympathize. Years ago (more than two decades) the situation was already pretty crazy. I managed to use some version of GNU make for building Visual C++ code.
The Microsoft CL.EXE compiler knew absolutely nothing without being told. Unlike /path/to/whatever/gcc, it doesn't know where its headers or libraries are, nothing.
In the Makefile, had to specify all the include and library paths in the SDK installation.
It's like trying to use GCC's cc1 directly or something.
Since that time there have been many more releases of Visual Studio and the SDK, all of them moving targets without a doubt. Yikes!