Don't get me started on the wasteland that "My Documents" has become, given 20 years of incompetent developers.
That's one of the reasons why it had a space. If you wanted to stop using the DOS APIs and get off of `PROGRA~1`, you need to handle spaces properly. Having a bunch of the standard paths (see also `My Documents`) include spaces meant that software vendors shouldn't have been able to get away with using Windows filename APIs without supporting filenames with spaces - everything would obviously break almost instantly.
Doing that, Microsoft made sure that there was no reasonable way for an ISV to claim that they either forgot to test that use-case, or weren't going to support it, when every Windows system in existence relied so heavily on paths with spaces.
software that you can specify the install location for, so it installs a update manager at that location but then always installs the actual programs under the C drive
Microsoft decided at some point people hard drive were plenty big so who cares? Unfortunately SSDs came along and for quite awhile space was quite tight on them!