Any sufficiently advanced uninstaller is indistinguishable from malware
devblogs.microsoft.com
devblogs.microsoft.com
https://www.codeproject.com/Articles/17052/Self-Deleting-Exe...
> Whether they follow the licensing terms for that code I do not know.
I'm guessing they didn't ship the binary with a link pointing back to this page?
These's also another codeproject example that uses a bat file, which is fairly similar to the recommendation in the post. I guess that's the better example.
https://www.codeproject.com/Articles/4027/Writing-a-self-des...
> shellcode is the technical term (in security circles) for binary machine code that is typically used in exploits as the payload. Here's a quick and dirty way of generating the shellcode from the obj file generated when you compile your source files. In our case, we are interested in whipping the shellcode up for the remote_thread routine. Here's what you've got to do:
The whole article has the vibes of some questionable DIY blog along the lines of "Your house is infested by vermin? Here is an easy way to get rid of them using a small, homebuilt neutron bomb!"
my question(s): How is the proposed solution better than the original thing? Isn't this a case of using bad heuristics to determine maliciousness?
In the end, he goes a bit further, and sees its non-malicious. So, with a more elaborate rule or heuristic, wouldn't it be clear its not malicious?
I'm only assuming here, but maybe because it won't crash explorer and it's just a few lines of self-documenting code?
I guess they could do this more robustly. I.e pause the entire explorer process, save all its state, remotely allocate new memory to inject their code, remotely create a new thread, run only that thread using the injected code, restore all the process' state and finally start it running where it left off. A script would be easier though.
I would have assumed (naively?) that they could just copy their uninstaller into a temp folder and run it from there, and just rely on the OS to nuke it in due time, but as a consumer I appreciate the thoroughness of an uninstaller that leaves no trace.
Both detouring and remote thread injection are supported on Windows, but fall into the category of gray-hat techniques; there are some legitimate uses but quite a lot of illegitimate uses, and using these techniques correctly (without crashing anything!) can be a real challenge.
It can do that because, I'm guessing, the file isn't open; the run-time isn't executing instructions from that file. The file was read, the content compiled into memory, and closed.
The script is deleting its source code, not itself. What actually deletes the script itself is the garbage collector in the run-time. Once the fso.Deletefile call and the for loop are executed, they are no longer reachable and so they are garbage. If there is a way for the "var fso" and "var path" variables to go out of scope, they become garbage also.
A binary executable is mapped into memory while the process is running it. In Windows, an open file cannot be deleted.
But, even on Windows, a prog.exe could delete the prog.c source code it was compiled from, right? Same thing, sort of.
What if, hypothetically, the system was suspended in the middle of script execution, and the resume function was designed to reload the script from disk?
It just feels like a different hack to me.
Also - trying 20 times and pausing 500 ms seems wasteful. What are the chances that it's going to succeed a subsequent try if it fails the first try? Why not catch the error message and only retry for errors that have a plausible chance of succeeding if you retry?
If Raymond Chen says to rely on it then yes.
DllCanUnloadNow returns an indication that a DLL can be unloaded. A DLL cannot be unloaded if any threads are executing the code. But a DLL can only change to the unloadable state by executing some code, and that code has to return after it has set the indication. Only after it returns is the DLL is actually unloadable! So a delay is needed for that thread to vacate the DLL.
https://groups.google.com/g/microsoft.public.vc.atl/c/AQvHCW... [2001]
The binary .exe program which the script is trying to delete is the one which created the script and launched it. So that means the .exe is still running at that point and cannot yet be deleted. Lauching the script indicates "I'm about to die", not "I'm already dead". The script cannot delete the .exe until the .exe terminates. Without some event to indicate that, you poll.
The script knows it can delete itself, so it tries that only once.
If a handle could be attached to the process, then the script could do a WaitForSingleObject on it; that would be the prim and proper way.
It doesn't seem worth doing; the chances are low that the process cannot terminate within 20 seconds of launching the reaper script.
As a metaphor: you find the instructions to sweep your floor cumbersome so you reprogram your neighbor's Roomba to come clean your floor. Sure, it may well go back to their house and no harm done, but it's hacky, socially unacceptable, and no matter how hard your broom is to use it's not OK.
I think the retry is necessary because if you launch "wscript cleanup.js" from the process that wants to be cleaned up, you then need to wait for the spawning process to finish executing. I agree if it fails after 20 times, you should probably spawn an alert or something letting the user know that uninstall failed. There are also so many random processes that might take a reference on the file like antivirus in Windows so just spamming retry will help wait that stuff out (this problem does not exist on Linux since Linux generally just does garbage collection which has the downside of not keeping specific paths around, just file inodes).
---- Neither code injection nor detouring is officially supported. I can’t tell who did the detouring. Maybe somebody added a detour to the uninstaller, unaware that the uninstaller is going to inject a call to the detour into Explorer. Or maybe the detour was injected by anti-malware software. Or maybe the detour was injected by Windows’ own application compatibility layer. Whatever the reason, the result was a crash in Explorer. ----
I'd think the anti-malware guess here would be correct, and that (DLL) injection was stopped, and thus some crash happened. Thanks for your reply.
The DLL would execute something from its stack so it can be somewhat dynamic (perhaps some path or something is generated before the injection or so - really little to go on here...) and not need to make a heap allocation within Explorer.exe. (this is perhaps a bit too much to assume idk.)
Thanks for your insights!
I was thinking hmm and so it continues that game of whack a mole.
If a new OS could be designed from scratch, there must be a way to prevent this sort of stuff.
Companies that buy AV/EDR products expect them to detect unwanted behaviour while allowing any sort of weird, hacky, abuses of the system that they rely upon for their business.
It's never been entirely clear to me why windows provides such a rich interface for one process to inject and start executing code in the address space of another but IMHO I want to know when this is happening even when it's done by a "legitimate" uninstaller.
AV/EDR products do try to prevent a lot of stuff. They can 'generically' block things like injection etc. by 'hooking all the things' and injecting into everything (yes, yuck :D and still kind of heuristic based i admit!) to make certain sections read only or remove executable mappings etc. (got/plt/stack/...)
Linux or more specifically ELF files also have an easy vector to allow injection by having a dynamic table entry for debugging purposes which can be trivially overwritten for example. "ibc.so" :(). I'm not sure anyone uses that entry validly... especially since there's better/less awkward debugging interfaces than injecting a debugger DLL into something :') at least in x86/64 Linux land. (ELFShell sure was fun tho!)
By writing a custom linker you can easily incercept all dynamic linking done at runtime and provide whatever you want to the program.
gcc -Wl,-dynamic-linker,/path/to/my/linker myprogram.c -o myprogram
And there is no written or unwritten rule about that. In fact, /home is a subdirectory of / which is owned by root.
The Linux Foundation’s File Hierarchy Standard puts user homes in /home, but it’s by no means mandatory.
/home being the *nix home folder directory isn’t written in stone, but plenty of software expects it. Of course you shouldn’t hard code things like that, but that has never stopped anyone from doing it. (Not that we should reward that with de facto standards necessarily.)
I understand the various reasons why a root file system hierarchy isn’t part of the Single UNIX Specification, but it might have been nice.
If you have separate partitions, would you really want user data to go to the system partition? Or a third partition?
Do you find having more places that user programs can write a benefit?
The fact that nobody does that is pretty much a consequence of the difficulty of coordinating multiple projects that do not have a common authority, not because it is a bad idea.
Maybe the reason no one does that is simply that no one shares your preference.
- easier to cleanup/wipe without risking deleting works/personnal files
- backup solution doesn't have to have a town of entries in an ignore/exclude file
- same as above for syncing software
- tier storage separation possibility
- disk space allocation separation depending on data vs volatile stuff
Personally I appreciate knowing where the config/cache for each application is. (Though it does annoy me when programs don't follow this as in your third example)
[0] https://specifications.freedesktop.org/basedir-spec/basedir-...
Now I believe all software should have a manpage, dialog and a cli argument that describes where all the files[1] generated by default go but that is another subject.
[1] cache, config and even default save
Presumably you mean something like using dpkg/apt for a Debian-style system?
I think that only works if a file is actually installed from within the framework. As soon as you've installed a file via npm, flatpak, pip, snap, specialist plug-in, standalone binary, that ancient program you had to install by hand, or one of the other squillions of ways of getting an executable program, you're out of luck and have to figure it out manually.
I suspect the final form for software installation is probably where iOS and Android are going in the EU, where there's a single means of installing software to the device so that everything can be sandboxed properly, but the acquisition/update process can be pointed to a URL/Store that the user has pre-approved.
macOS comes pretty close to what I'd ideally want in an OS with regards to installation - independent packages that are certified/notarised, but I'd like to see the OS allow for user-specified authorities beyond just Apple. That being said, I'm not sure I'd ever use them as it's part of what I'm paying Apple for, I'm really thinking more of Linux there.
A kind of flatpak/snap approach, but that has signing of the package and centralised management of the permissions for the sandbox at an OS level would be ideal in my view. That way it's still free-as-in-speech as the user can specify which notarisation authority to use (or none at all).
I really don't understand why seperate programs are handling removing their mother program in 2023, that's registry spaghetti messy.
I’d need to educate myself a bit more in terms of whether there are third-party authorities beyond Microsoft for the packages.
Found this introductory video for anyone else interested:
https://www.youtube.com/watch?v=phrD081sMWc
Note: I didn’t intend the Surface pun above, but it happened and we can all be glad that it did.
So whilst Microsoft will sign for you if you distribute via their store, otherwise you pay per year for certificates and can distribute outside the store.
There are problems with the system (cost, bugs, usability problems) but it is decentralized.
It's easy to run unsigned binaries/app packages on macOS: right click on the .app, hold down Option, then click Open and confirm the warning.
The closest Apple comes to this capability is achieved via device Supervision and MDM, which might be comfortable for some of us here in this forum but obviously isn’t practical beyond more technical circles.
Baddies keep ruining all the fun for the rest of us.
IMHO the blame rather lies with our politicians who are unwilling to take the steps necessary to cut the baddies off from the Internet. Let's see just how fast India, Pakistan, Turkey and other scammer hotspots clean up their act when the US+EU threaten to cut them off from the Internet and SS7 unless the scam callcenters are closed down for good... the amount of corruption regularly exposed by scambaiters on Youtube is insane. Billions of dollars of damages each year [1] from that bullshit and our politicians don't. fucking. care.
[1] https://www.vibesofindia.com/fraudsters-in-india-cost-americ...
You don't need the whole world. The Western world is enough - no Internet and phone service (both easily enforced by requiring providers to reject ASNs / phone country codes) means a lot of lost business for an affected country.
> Pressuring India is hard enough, try Myanmar, a place that doesn’t get along with the West at all and is already a hotspot for phone scams targeting Chinese speakers.
Honestly, that's China's problem to solve.
> So I’m all for pressuring India to crack down on scammers, but I don’t see how that would reduce the desire to tighten software controls on PCs.
When software vendors don't have to gate more and more features behind more and more obnoxious bullshit simply to whack-a-mole scammers, they won't.
On Unix/Linux land the prevalence of pipe curl to bash type installers is not much different.
I normally keep both types away from my computers.
True, but saying so will likely to earn you downvotes from those committed to this unhygenic practice ...
This is a problem but only if you install software on Linux by manually going to the project page and copy-pasting whatever curl they have there, I think the difference is that mostly you're encouraged to go the package manager route, whereas on windows downloading .exes directly (ala the curl example) is the norm.
Directly sudoing a curl-ed script is like running a binary on Windows with admin permissions and with Defender turned off, which makes it somewhat more scary to me.
On Windows I use Chocolatey when I can, and if I can't (or it looks dodgy anyway) I'll either just not install it or try it in a sandbox. Things that aren't choco-able are generally commercial software obtained from the vendor's download page, we theoretically trust those things somewhat. YMMV.
Most people would just say yes to any prompt they get, those wise enough not to aren't running random curl scripts either.
As for Defender being any kind of protection, I have my doubts.
> it seems to be increasingly the case that package managers just don't have some software - or have a version that's years out of date.
This is entirely distro dependant, some are very up to date and have most things you'd want, especially if you include the likes of AUR in that. But then there's usually a Flatpak or an AppImage that you can use in the odd case that they don't.
Basically how Linux distributions works since the beginning. Tough at the start the installation source was not remote but a CD-ROM things didn't change.
You have a repository of packages (that can be on a local source as a CD or remote source such as an HTTP/FTP server), that have some sort of signature (on Linux usually the pagkage is signed with GPG) with some keys that the user trusts (and the default are installed on the system), and a software that allows to install, uninstall and update the packages.
Android/iOS arrived later, but they didn't invent anything.
The system is great, in theory. In practice adoption has been held back by the fact that it was originally only for UWP apps which almost nobody writes, and also only for the MS Store. These days you can use it for Win32 apps outside the store but then you will hit bugs in Windows. And packages must be signed.
Still, the feature set is pretty great if you can make it work. For example you can get Chrome-style updates where Windows will keep the app fresh in the background even if it's not running. And it will share files on disk between apps if they're the same, avoid downloading them, do delta updates and more. It also tracks all the files your app writes to disk outside of the user's home directory so they can be cleanly uninstalled, without needing any custom uninstaller logic.
One interesting aspect of the format is that because it's a "special" (read: weird) kind of zip, you can make them on non-Windows platforms. Not using any normal zip tool of course, oh no, that would be too easy. You can only extract them using normal zip tools. But if you write your own zip library you can create them.
A couple of years ago I sat down to write a tool that would let anyone ship apps to Win/Mac/Linux in one command from whatever OS they liked, no harder than copying a website to a server. I learned about MSIX and decided to make this package format. It took us a while to work around all the weird bugs in Windows that only show up on some machines and not others for no explicable reason, but it's stable now and it works pretty well. For example you can take some HTML and JS files, write a 5 line config file pointing at those files, run one command and now you have a download page pointing to fully signed (or self signed) self-updating Windows, Mac and Linux Electron app. Or a JVM app. Or a Flutter app. Or any kind of app, really! Also IT departments love it because, well, it's a real package format and not an installer.
Writing more about this tech has been on my todo list for a while, but I have now published something about the delta update scheme it uses which is based on block maps, it's somewhat unusual (a bit Flatpak like):
https://hydraulic.dev/blog/20-deltas-diffed.html
The tool is free to download, and free for open source projects if anyone is wanting to ship stuff to Windows without installers:
Considering the ability to update itself is a requirement of Cyber Resilience Act in EU, I foresee a big uptick in usage (and app stores usage of course).
It allows for a variety of installers to exist with different features for different use cases.
It allows you to install the application in any location you choose.
It allows for portable installations and to run software just copied from other sources.
All the points you list does not need _Special_ installers / uninstaller.
If the only installer available was one provided by the OS how long do you think it would take to make that the only way to install and run software. These things are being done right now on many platforms in the name of safety, security, and to a lesser extent convenience.
The more phone-like a platform is the fewer ways you have to install and run software on it. So far general purpose computers still allow you to install software in other ways than the built-in method (i.e. just unzip and place in a directory), but it's getting increasingly common to require executables be signed, and things are always moving to be more and more locked down.
Now the use of "Special" installers/uninstallers is from the original comment, I would just refer to them as "regular" installers/uninstallers. I do like the ability and freedom to have an ecosystem of these things, as I don't want the one OS method to be the only way to install applications.
There's the non-sequitur. OP never said that this is what should happen. It is strange to leap to this assumption while also wanting to define portable programs and archives as 'installers'.
In the context of Windows, 'special' installers means the programs you run to be able to use a different program that don't appear on other OSes.
The trouble is that MS never paid much attention to tracking and cleaning up after installations or after uninstallers has finished. Often this doesn't matter but when something seriously goes wrong untangling the mess can be almost impossible, it's often easier to reinstall Windows and usually much quicker (that's if one has a simple installation).
Unfortunately, my installations aren't simple so I take snapshots at various stages of the installion—stage-1 raw install with all drivers, stage-2 essential utilities, and so on. By stage-4, I have a basic working system with most of my programs. Come the inevitable Windows stuff-up I reinstall from a backup image, it's much quicker than starting from scratch.
Between those major backups, I use the registry backup utility ERUNT, it not only takes registry snapshots on demand but also automatically backs up the registry on a daily basis. This, I'd venture, is the most important utility I've ever put on a Windows computer, I cannot recall how many times it's gotten me out of trouble.
Just several days ago I had a problem reinstalling an update to a corrupted Java jre/runtime. Nothing I did would make the installer install as the earlier installation was not fully uninstalled, thus log files etc. weren't a help.
In the end I had to delete the program dir and other Java files I could find, same with registry entries. As expected, this didn't work, as I hadn't found everything.
Knowing the previous version number of Java I did a string search across the OS and found another half dozen or so Java files. Retried the install again and it still failed. I then ran ERUNT which replaced the registry with an earlier pre-Java one and the install now worked. This still meant that some programs that were added later, LibreOffice for example, had to be reinstalled to update the registry.
If I hadn't had ERUNT installed I'd have had to go back to reinstalling an earlier partition backup. And if I'd not had those then I'd have been in real trouble.
That's the short version. Fact is, Windows is an unmitigated mess when it comes to installations. Why can't I force an installer to complete even with faults? Why doesn't Windows remember exactly what happens during an installation so it can be easily undone?
_
Edit: if you've never used ERUNT and decide to do so, always ensure you shut Windows down and restart it after installing a backup registry before you do anything else—that's in addition to the mandatory reboot required to install the backup.
You may have multiple registry backups and decide the version you've just loaded wasn't the one you want. Loading another without this additional reboot [refresh] will blue-screen the O/S. You'll then have to install the backup manually and that's very messy.
If it provides a framework for installers/uninstallers, it'll be fighting the inertia of decades of legacy software, programmer habits, and old tutorials.
If it tracks file ownership by program, it might accidentally delete user files. How would it differentiate between a VSCode extension that should be uninstalled, and a binary compiled with VSCode for a user project? A false positive could be catastrophic.
If it restricts what programs can do to accurately track file ownership, you end up with Android. Which is fantastic for security, but is a royal pain in the ass for all parties:
- The app developers have to jump through hoops for the simplest actions, and rewrite most of their code in the new style.
- The operating system has to implement a ton of scaffolding, like permissions-restricted file pickers and ways to hand off "intents" between applications.
- The user is faced with confusing dialogs, and software with seemingly arbitrary limitations.
In the age of shared runtimes, auto-updaters, extension marketplaces, and JIT compilers, managing installed applications is harder than ever.
Edit: the answer above applies only to Windows, because of its baggage. Linux'es are in a much better position, for example, though their solution is still not perfect.
Define a separate directory for program installations, that user processes cannot write to. Only program that can do so is the package manager, which other programs can call to install packages. Uninstall removes everything related to a program from this directory.
> In the age of shared runtimes, auto-updaters, extension marketplaces, and JIT compilers, managing installed applications is harder than ever.
The only reason these make things hard is that windows lacks any facility to deal with them. Solutions going forward: Outright ban having your own auto-updater, to auto-update you register your program and where to update it from with the package manager. Shared runtimes are trivial for package managers to handle, it's just a package that many other ones depend on. Extensions can be handled as packages.
But what would you do with the millions of existing programs, most unmaintained? And what about programs with strong opinions on update schedules, or built-in extension marketplaces?
It's easy to solve this problem if your first step is "replace every program".
By the way that's what we have on Linux, some programs come as a shell script that you run to install them. Most Java IDEs for instance.
(which can't be arsed to provide proper packages -- darn, what did I just write? :-))
The security situation is so bad at this point that you can't trust any Windows benchmarks anymore. The benchmark suite will run on a "bare" Windows system; probably with updates and Windows Defender disabled and many other system services stopped to maximize performance and prevent background services from slowing everything down. The reality though is that on a regular user desktop all these things and a whole lot more will be enabled, resulting in vastly degraded performance compared to the benchmarks. The end user experience sucks.
Now they're forcing ads down your throat and pestering you at every turn to use more Microsoft software (e.g. trying to get you to use Edge). They've also recently included UI changes in "essential" system updates that can't easily be reverted or undone, breaking people's workflows. It's anti-user insanity and it's all because Microsoft can't actually go back to the drawing board with Windows anymore because the alternatives are just too good.
After using a Linux desktop full-time for a while, going back to Windows feels like going from having modern plumbing to pooping in the woods.
(which means that if you don't trust a provider, not only it's not safe to run the program, but it's also unsafe to install it)
packages are cryptographically signed by the packager, by the way on Debian you add the key when you install a new repository. The signature tells you "This package has been built by X and has not been tempered in the meantime", not "X and this package are not malicious, I promise".
> you have the option to abort the install of an untrusted package before it does something malicious
How do you do this in practice?
If I run apt install p or or dpkg -i p.deb, the thing is installed. APT asks you for confirmation if it has to install additional dependencies but that's it.
I don't have no guaranty such like for any package, I can install it without worrying something bad won't happen during its installation.
Of course you should not install untrusted packages, but still. The same could not be said if the package format didn't have anything to specify arbitrary install scripts.
Isn't it same for windows right now? `.msi` and `.exe` can execute arbitrary code right?
If you must install proprietary software on your Linux system, either package it yourself or use something like Flatpak or Snap (or even AppImage).
Hopefully in the future vendors will increasingly move to providing well-sandboxed Flatpak packages by default.
What about /usr/local/bin? Isn't that specifically for putting non package manager binaries into?
Of course, a decade later almost nobody uses those because they botched the rollout by limiting AppX to the Microsoft Store and an entirely new poorly documented and very restrictive set of windows APIs and app frameworks. They've made huge progress on all of those problems with MSIX to the point that it's a reasonably good and easy to use choice for most apps with some neat benefits like updates only downloading the changes between versions. Of course if your app pushes the boundaries of the sandbox or capabilities or runs into a bug it becomes a huge pain.
I think Affinity Photo's experience with MSIX is instructive; hundreds of negative results on their forum, eventually they had to back down and provide a non-MSIX installer (and at that point do ya really want to maintain 2 separate Windows installers?)
https://forum.affinity.serif.com/index.php?/topic/170529-ext... https://forum.affinity.serif.com/index.php?/search/&q=msix&t...
I'm going to assume you are talking about rpm and deb packages since they are still currently the dominant installation packages on Linux.
> Define a separate directory for program installations, that user processes cannot write to. Only program that can do so is the package manager, which other programs can call to install packages.
Windows does this. Programs are installed in directories under "C:\Program Files" which is only writable with elevated system rights.
> Uninstall removes everything related to a program from this directory.
rpm and debs don't install all the files needed for a program in a single directory. They are scattershot all over the file system and in many of these directories comingled with files from other programs. Windows comes closer than Linux in this regard since it does create the directory under "C:\Progtam Files" which while unfortunately doesn't always contain all the required files usually contains the vast majority.
There are a ton of VSCode extensions in Nixpkgs: https://search.nixos.org/packages?channel=23.05&from=0&size=...
You can use them in combination with the vscode-with-extensions function to create a VSCode package that bundles in whatever extensions you declare: https://nixos.wiki/wiki/Visual_Studio_Code
In the same way 'Linux' (in the widest sense of the term, i.e. Linux distributions like Debian) handles this. User data is not touched by the (un)installer, configuration files are checked for changes from the package default and left alone unless explicitly purged. Files which do not belong to any package are left alone as well so that binary compiled with VSCode for a user project will not be touched:
warning: while removing directory /splurge/blargle/buzz not empty so not removed
This has worked fine for decades and is one of the areas where those Linux distributions were and are ahead of the competition. It works fine because the package manager has clearly delineated responsibilities and does (or rather should) not go outside of those. Do not expect the package manager to clean up your home directory for you, that is not part of its duty.> In the age of shared runtimes, auto-updaters, extension marketplaces, and JIT compilers, managing installed applications is harder than ever.
Most auto-updaters should be disabled on systems with functioning package management - Thanks Firefox but I'll update my browser through either the package manager as I prefer my executables to be read-only for users.
Some packages - the whole Javascript rats' nest being a good example - move too fast to be usefully packaged by volunteer maintainers so those are relegated to a different area which is not touched by the package manager. Other packages - anything Python fits here - are such a tangled mess of mutually incompatible versioned spaghetti that they are hard to fit inside the idiom of package managers so they get their own treatment - python -m venv ... etc. These are the exceptions to the rule that package management can be made to work well. By keeping those exceptions in areas where the package manager does not go - e.g. your home directory - the two can and do coexist without problems.
The issue is that MSI is very buggy when handling explorer extensions. If you're not careful, when you uninstall it'll prompt you to close explorer.
(I know because I shipped a product that installed via MSI and had an explorer plugin. The installer issues were more complicated than the plugin.)
In this case, the issue is that when explorer loads a plugin, it keeps an open file handle to the dll. This gives the installer two options: Restart explorer.exe, or somehow finish uninstalling when explorer.exe terminates.
The product that I shipped just restarted explorer.exe.
I get unattended installs and uninstalls "for free" when well-behaved applications use Windows Installer. Patching is included, too. Customizing installations is fairly straightforward.
On the developer side it has historically used a very quirky proprietary file format (MSI) with a fairly steep learning curve and a ton of "tribal knowledge" required to make it work for all but the most trivial cases. (Though, to be fair, most installs are the trivial case-- copy some files, throw some stuff into the registry, make some shortcuts.)
Worse, it allows for arbitrary code execution ("Custom Actions"), at which point all bets are off re: unattended installs, removal, etc. Some Windows Installer packages are just "wrapped" EXEs (Google Chrome, for example).
I've packaged a ton of 3rd party software as Windows Installer packages. It's an ugly system with lots of warts and legacy crap, but if you need to load an application on a large number of Windows PCs reliably unattended it's decently fit for purpose.
There is reasonable free and libre tooling to generate MSI packages from plain text source (the WiX toolkit) and it can be used in a CI pipeline.
Yes, people preffer to debug their own code rather than spend shitload of time to understand Wix/MSI.
Microsoft deciding early on to not produce low cost tools for Windows Installer also didn't helped with the adoption.
But there is the modern implementation of AppX Bundles which was later extended to create MSIX which allows app distribution without the windows store. There are still drawbacks to using MSIX usually because you want to touch Windows in ways you can't inside the sandbox.
In my experience, you can pretty easily write the nice sandboxed MSIX manifests by hand, it's not too bad, but general MSIX doing weird MSI things you still want better more expensive tools to build them (and of course Microsoft themselves still don't exactly provide that and will point you to plenty of expensive third party installer studios for options, many of which are the exact same ones people have been overpaying for decades).
[0] The one complaint I'm aware of is that you can't do custom installer UI and "attended" installs with user choices. There's one MSIX UI and it is barebones but acceptable. That's all you get.
The whole story and ideas put into Windows installing programs are a stupid joke. It's designed by project managers who have no idea what they are doing and no imagination and is executed by the cheapest least motivated programmers from South Asia Microsoft can possibly find.
A lot of people who do useful stuff for Windows try to stay away from as much of Microsoft's stuff as possible, and integrate it as little as possible with anything else coming from the system because it's just maddening how horrible everything there is.
"Plain-Text files for configuration? what do you think we are? savages? no, we need a hierarchical registry! every serious program must use some opaque binary format to store stuff!" seem to be the general animus at that time. Nowadays, even if you really hated the idea of a text files for configuration in your home direction, people would probably do something more straight-forward like using a SQLite db.
There were also very practical solutions incorporated to accommodate the constraints of the hardware of the time that come off looking like needless complexity today, too. (There's also, arguably, some laziness in the old binary file formats that were nothing more than dumps of in-memory structures. That's a common story across a ton of old software-- not just MSFT.)
Rob Mensching, who worked at Microsoft on Windows Installer pre-release, has a nice blog post about internals of MSI.[0] He goes into some of the overwrought architecture in MSI, as well as quirks to overcome performance limitations in a world of floppy disk-based distribution and much smaller memory capacities. It's a good read.
[0] https://robmensching.com/blog/posts/2003/11/25/inside-the-ms...
Often, as you mentioned, Windows Installer packages are wrapped by an executable (in WiX this is called a "bundle" because you may also choose to add redistributables like the C++ runtime).
However, what you see in installations like SQL Server, Office and Visual Studio is that the installers are bundles as well - of a large amount of MSIs that need to be installed and uninstalled in a specific order. A single Microsoft Installer package is transactional and can be rolled back on failure, but bundles are not as well defined and left open to the implementation of the bundle. Windows Installer does not reach beyond the single msi package.
It is intentionally obtuse at times (MSIFileHash table uses rearranged MD5 hashes for example), and also many features made sense for the late 90's/Early 2000's era where bandwidth was low and connectivity limited, and lots of stuff was distributed on CD's. The look on people's faces when you explain advertisement to them the first time... How their unrelated app can get stuck in a loop of repair for a piece of unrelated software...
It was deprecated by the newer AppX/MSIx/AppV format which uses sandboxes, binary chunks/streaming and no executable code to install stuff.
For my own desktop computing, I prefer MSI packages because I prefer having control post-install to tinker with things if I feel like it. Also, I have the skillset to modify the installer to my whims if I so choose.
I can offer a little perspective on MSIX, having devoted months of my life to it in a past job.
MSIX is nearly unusable outside the Store. It will work in a tightly controlled environment, but when you try to deploy it to a wide variety of users you will run into 1) unhelpful errors that basically can't be diagnosed, 2) enterprise environments that cannot/will not allow MSIX installs. I get the impression that the MSIX team is uninterested in solving either of those issues.
It's not a coincidence that virtually no first-party teams use MSIX to install their product outside the Store. Office tried for a while and eventually gave up.
Despite all that, there are still a few people at MS who will tell you that MSIX is the future. I don't really understand it and I assume there's a weird political reason for this.
Still, when you finally make it work you get a lot of benefits. MSI is clearly dead end tech which is why so many MSIs are just installer EXEs wrapped in the file format. It doesn't have any clear path to modern essentials like online updates, keeping the OS clean, sandboxing and so on. If you were on the Windows team, what would you say the future was?
For enterprise environments it's actually somewhat the opposite: MSIX packages can be installed without admin privileges due to their declarative nature, and it's very easy for admins to provision MSIXs to Active Directory groups because they don't have to do any repackaging work. Yes, some admins have hacked Windows to stop it working because when MS launched the Store they didn't provide any way for admins to opt out, but these days they have the knobs they need. Also, because they're just zips you can always just unzip them into your home directory to get at the app inside. It won't auto update that way, but as long as EXEs can run from the home dir it can work.
Products like Office and Visual Studio have entire teams devoted to nothing but their installers, which is clearly going too far in the opposite direction. Most products won't want to do that.
I assume the Here in NIH refers to an individual team, not MS as a whole.
Teams is entirely NIH https://github.com/Squirrel/Squirrel.Windows for updates to the Electron app.
I would use winget, but MS made it weirdly hard to run as a script on multiple computers, it installs per user, because... who knows.
So still using chocolatey
Therefore the documentation is poor , like the absolute worst I've ever seen. Opening issues for doc issues never results in anything. Pointing out UX issues is usually shot down. Finally, until this year you needed .net 2 installed to build it, which does not play well with windows docker.
This is supposed to happen using MSI-based installers. It's a windows component.
> Why isn't this handled by Windows itself?
Now, here's where things get tricky.
In the article, the issue is an explorer plugin. MSI is notoriously buggy with installing and uninstalling explorer plugins. If you don't jump through hoops, your installer will have a bug where it prompts the user to close Explorer.exe.
I know because I shipped a product with an explorer plugin. The installer was always a thorn in our side; and the workarounds, ect, that we had to do to install / uninstall / delete our plugin were more complicated than the plugin itself.
On Windows, opening a file locks it. So you can't delete a program that is running, you will get an error. It means of course that an executable can't delete itself without resorting to ugly tricks like the one mentioned in the article. That's also why you get all these annoying "in use" popups.
On Unix, files (more precisely: directory entries) are just reference-counted pointers to the actual data (inode on Linux), removing a file is never a problem: remove the pointer, decrement the reference counter. If the file wasn't in use or referenced from elsewhere, the counter will go to zero and the actual data will be deleted. If the file is in use, for example because it is an executable that is running, the file will disappear from the directory, but the data will only be deleted when it stops being in use, in this case, when the executable process terminates. So you can write your uninstaller in the most straightforward way possible and it will do as expected.
For example, in the bad old days I used detours to virtualize the windows registry so that I could do "fake installs" of COM components inside of a VB6 app, allowing it to run without an install and without administrator permissions. This worked by detouring all the Win32 registry APIs, which was sufficient to intercept registry accesses performed by the COM infrastructure.
Detours preserves the un-instrumented target function (callable through a trampoline) as a subroutine for use by the instrumentation. Our trampoline design enables a large class of innovative extensions to existing binary software.
I’ve created a PowerShell module that wraps this library to make it easier to hook functions on the fly for testing https://github.com/jborean93/PSDetour. For example I used it to capture TLS session data for decryption https://gist.github.com/jborean93/6c1f1b3130f2675f1618da5663... as well as create an strace like functionality for various Win32 APIs (still expanding as I find more use cases) https://github.com/jborean93/PSDetour-Hooks
Yes please. Thank you for this
This capability is intrinsic to how ELF linking works. The main application or even any library can interpose a libc function just by defining and exporting a function with the same name, and that definition will be preferentially linked in both the main application and all subsequently loaded dynamic libraries and modules. Your definition can then use dlsym(RTLD_NEXT, "foo") to obtain a function pointer to the next definition, which would normally be libc itself but may be from another library. A running application could actually have several implementations of a function, all proxying calls onward until the terminal (usually libc) implementation.
Basically, the way ELF linking works by default is that the first definition loaded is the preferred global symbol used to satisfy any symbol dependency with that name. It follows that there's normally a singular global symbol table. Though there are features and extensions that can be explicitly used to get different behaviors.
There's nothing magical about LD_PRELOAD within the context of ELF linking. LD_PRELOAD support in the linker (which is the first bit of code the kernel loads on exec(2)) is very simple; all the linker does is load the specified libraries first, even before the main application, so symbols exported therein become the initial and therefore default definition for satisfying subsequent symbol dependencies, including in the main application binary, and even if the main application binary also defines and exports those symbols.
All of this is basically the exact opposite behavior of how PE linking works on Windows, for better and worse--depending on your disposition and problems at hand.
Also note that all of this is different than so-called "weak" symbols, which is a mechanism for achieving one of the same behaviors--overriding another definition--when statically linking. Otherwise, when statically linking, multiple definitions are either an error or it's difficult (i.e. confusing, especially in complex builds) to control when and where one definition is chosen over another.
[1] Though main application symbols aren't usually exported by default, so you need to explicitly mark a definition for export or build the entire main binary with a compiler flag like `-rdynamic`, which is the main binary analog to the `-shared` flag used for building shared libraries. The Python and Perl interpreters, for example, are built with -rdynamic as the interpreter binary itself exports all the implementation symbols required by binary modules, rather than defining them in a separate shared library against which modules explicitly link themselves against at compile time. (This is also why when building Perl, Python, and similar language modules you have to tell the compile-time linker to ignore unresolved symbols.)
Many OSes have file locks, though they often don't use them as liberally as Windows.
Most Windows apps sit under program files, some sit directly on drive root. But all spray configuration/user data files for themselves all over the damn place, requiring unique uninstallers.
MS, build app install/uninstall into Windows directly...
I think Windows 98 already had this ability. Possibly Windows 95 as well. It's a variant of the language called JScript, which is what was used in old versions of IE too.
WSH btw allowed you to run any language that you had interpreter for - they had to support necessary COM interfaces (and to be truly usable, allow you to call COM objects), and register their interpreter class with ActiveScripting (WSH internal) engine.
Then you could use them not just for desktop automation, but also for scripts inside Internet Explorer (essentially, classic IE used WSH engines to implement scripts, iirc)
I've seen WSH (including HTAs) used with Perl, Python, Tcl, Rexx...so long as you install the interpreter with compatible COM service, you could use it.
By the way, there are also HTAs, which are Microsoft HTML Applications. You can create a simple double-clickable GUI with these using only HTML and JScript.
Apparently XULRunner was first released in 2006, but Thunderbird, which uses (used?) the same technology, was released as early as 2003, and maybe this was existing in the Mozilla Suite even before.
Also, I think one of the Start Menus (might have been XP‽) was kind of HTA-ish? Not sure about that part, though.
> Think of it as they had a fork of Firefox
Yep indeed, you are right.
Notable projects using actual XULRunner included Songbird (a music player) and BlueGriffon, an WYSIWYG HTML editor (a successor of Nvu and KompoZer, themselves succeeding Netscape Composer). Both released after 2006 indeed.
I liked XUL, I strongly believe Mozilla could have dominated the market taken by Electron, had they pushed XULRunner more, and perhaps make it transition to pure HTML, like they did to Firefox's core, because that's what people know and because XUL was a maintenance burden. I think XUL tags made more sense than HTML to build UIs, though, and with XUL, Gecko have had a CSS flex-like mechanism for a long time by the way.
1) You could write event handlers in multiple languages, including C. If you wrote them in C, it spawned gcc and compiled it into a library, and dynamically loaded it... The overall idea of a polyglot runtime like that was fun.
2) #1 is only really weird because this could be done at runtime. One of the demo apps was an editor for the GUI itself, where you could add buttons to the editor, then write that event handler in C, and have it compiled and loaded into the editor itself...
It was a fascinating starting point, though full of heavy duty foot guns, and I'm still sad nobody took it further.
Active Scripting, which powers scripts in both WSH and old-school IE including HTAs, is polyglot and extensible. It’s why Active{Perl,Python,Tcl} are called that—the original headline feature (IIUC) was that they integrated with it. It’s also why you could write VBS in IE: IE just passed the text of the script along with the language attribute to AS and let it sort things out.
Nobody did ever a C interprerer, though, I think—perhaps because you basically have to speak COM from Active Scripting, and while speaking COM from C is certainly possible it’s nobody’s idea of fun. (An ObjC-like preprocessor/superset could definitely be made and I’ve heard that Microsoft had even entertained the idea at the dawn of time, but instead they went with C++, and I haven’t been able to find any traces of that project.)
That’s not to say AS is perfect or even good—the impossibility of caching DISPIDs[1], in particular, seems like a design-sinking goof. And the AS boundary was also why DOM manipulation in IE was so slow.
[1] https://ericlippert.com/2003/09/16/why-do-the-script-engines...
Wondering what would it take to port mshta (with all the ActiveX goodies) to other platforms. Maybe it's a little bit late for that, but sounds like it might be a fun project to me.
Wine Gecko supports ActiveX, supposedly, so if someone implements all the common ActiveX components, that could be a cross-platform method of running HTAs outside of Windows.
That said, I'm afraid the Electron API is the closest thing we have to a cross platform HTML application these days. On Manjaro, several packages are already implemented by installing Electron next to the application specific code, so that would be the closest thing to a modern HTA alternative that I know of.
PWAs work fine if you don't need integration with the system itself other than file prompts, for chat apps for example. They're not really alternatives to HTAs to be honest.
It should be noted that HTAs are a common way to infect computers (because they're executables that aren't usually recognised as such) and they're disabled in many security conscious environments.
Seemingly_random_code(seemingly_random_string)
The seemingly_random_code decompresses/decodes whatever is in the seemingly_random_string and hands over control to it. Interestingly the decoded code is another version of the same with different code and string. This goes on for ~100 layers deep then at the end it just downloads and executes some file from the net.
I understand doing one layer. I guess I could maybe see two layers. But why would it bother with 100 layers? Either the antivirus or reverse-engineering tool can grab the final product or it can't.
This is similar to how compilers will often have recursion limits for things like generics, though in that case it's easier to reject the program if the recursion limit is reached.
In very strict environments or certain systems it might be practical to block all unknown files, but this is uncommon for user systems for example where users are actively using javascript or macro documents etc. (developers, HR, finance etc.) The FP rates are too high and productivity can take a big hit. If all users do 20% less work that's a big loss in revenue (the productivity hit can be much more severe even!). perhaps this impact / loss of revenue ends up being bigger than a malware being executed depending on the rest of the security posture/measures.
technically its possible to identify (nearly?) all malware by tracking p-states/symbolic execution/very clever sanboxing etc.. but this simply takes much too long. Especially if the malware authors are aware of sandboxing techniques and symbolic execution and such things as they can make those processes also take extra long or sometimes even evade them totally with further techniques.
I wish it _was_ possible to do all of the cleverness that malware researchers have invented to detect things, but unfortunately, in practice this cannot happen on like 90+% of environments.
If you run like a DNS server or such things, it's possible to do it as such a system would not be expected (ever?) to have unknown files. (gotta test each update and analyze new versions before deploying to prod). As you can imagine, this is also kind of a bummer process but imho for such 'static' systems its worth it.
Personal promotion, I built this as a better alternative:
https://github.com/technion/open_safety
Note the built in .js parser hasn't basically ever updated, if you're writing for this you're writing like you're targetting IE5.
Huh, neat!
like, really? can't you write that in C? I don't think most Win32 apps use JavaScript for their installers.
The point of the exercise is that, on Windows, you can’t, because Windows won’t let anyone delete executables that are currently in use (try it, you won’t be able to delete one either). Upgrading shared DLLs in the face of this fact is why installers for Windows programs often have to have you reboot the system (and in more civilized times asked you to close other programs before installation to reduce the probability of hitting a locked DLL). It’s also why there’s a registry key[1] containing a list of rename and delete actions to be performed on next reboot (usually accessed via the MOVEFILE_DELAY_UNTIL_REBOOT flag to MoveFileEx).
You can’t (straightforwardly[2]) make a self-deleting batch script, either, because the command interpreter parses a command at a time and so wants the batch file to exist. The Windows Scripting Host, on the other hand, will parse the whole file at once, close it, and then forget about it, so you can write self-deleting WSH scripts.
The workaround used by the uninstaller under discussion is instead for the executable to inject some code into the Windows Explorer (on the assumption that it’s always running and the user has to have access permissions for it) that accomplishes the deletion through return-oriented programming, so that the stack it’s executing from can then disappear into the wind (apparently? I’m not seeing how they plan to clean that up).
On a POSIX system you are explicitly allowed to delete any open file—including an executing one—making it languish in a kind of system-managed limbo (and take up disk space, invisibly) until it’s closed. The tradeoff is then that it’s impossible to ensure you’ve opened the same file as somebody else when all you have is its name. (I think you can at least check for success, provided you also have the device and inode numbers for it.)
[1] https://superuser.com/questions/58479/is-there-a-registry-ke...
[2] https://stackoverflow.com/questions/20329355/how-to-make-a-b...
E.g. `powershell.exe -ExecutionPolicy Unrestricted`
My concern is more than Raymond Chen suggest that using it is still the recommended way. So much malware came through WScript.
Install? Copy the directory to where you like. Uninstall? Delete the directory.
And if you wish you could keep any files used/generated with such an app in the same folder, making it 100% self-contained.
I remember being rather grossed out when I learnt Windows has "a registry" (that was a long time ago). "Why would you have a global registry? Whatever preferences a piece of software has they should live where the exe is".
(and yes, I am aware AmigaOS had an installer and dir structure not that unlike of Unix, with `sys:`, `devs:` and so on)
Also, it suffers from the same problem as Windows here, in that you can't delete a file or directory which is currently open. The executable itself wouldn't be open after launch is complete (with the possible exception of overlay executables, but they were pretty rare) but the directory would be locked for as long as it was the program's current directory. If a subdirectory with app-specific libraries was assigned to Libs: that would also prevent deletion.
And for legacy/backward compatibility reasons, once MS allowed this behavior to go unchecked, there was no way to put the genie back in the bottle and stop it, without giving up backward compatibility. It didn't help that Microsoft software tended to be the "tentacle" kind as well.
So in theory apps could be distributed portable .exes but in practice Windows doesn't any ways of interacting with the rest of the system that are that nice.
I mean can you write a simulation of an uninstaller to create havoc on target's system and still remain "the good guy, the OS is at fault" type of situation when you write a malware?
I remember very distinctly this quote about him:
> The only person in the world who leapt to my defense was, of course, Raymond Chen, who is, by the way, the best programmer in the world, so that has to say something, right?
So in my mind I've made the connection that Raymond Chen = best programmer in the world since then haha.
If you mean in other contexts... I think the point is you're not intended to be able to do this? Outside of uninstallers, running code that only exists in RAM is... the type of thing malware typically wants to do more than anything else.
But in terms of what's physically possible, I suppose there's the command prompt, PowerShell, and scheduled tasks? I'm not sure if all of those can be disabled.
I'd be surprised if this capability is only available from jscript though. (and sad, I don't think jscript has been updated in years)
but honestly, please Windows Dev's.. use MSI's please
it make me love you and i have so much love to give.
Alright cowboy, so where are you storing preferences and settings?
The OS isn't to blame when people give root access left and right.
Actually this is why macOS got SIP.
The shift towards protections from malware happened mostly as a consequence of Windows XP. There are now better controls, like assigning low-trust processes like a browser's renderer a low integrity level to prevent them from doing that. But it's also late enough that it's hard to rock the boat too much without breaking existing applications. Microsoft tried to make a clean break and offer more sandboxed applications with a user-friendly package manager (called the Microsoft Store) but this wasn't well received by app developers: most didn't use it at all, and those that did often opted out of the sandboxing.
From microsoft's documentation [1] > Administrator-to-kernel is not a security boundary.
I recommend the talk https://www.youtube.com/watch?v=Y09nAxZFKzc
[1] https://www.microsoft.com/en-us/msrc/windows-security-servic...
For example, antivirus products do this all the time, as do many video drivers and other system utilities.
Also, Explorer has various plugin interfaces where it'll load third party code and run it in-process since the very first version.
var fso = new ActiveXObject("Scripting.FileSystemObject");
rather than var fs = require('fs') // (or the appropriate ES6 incantation)
?And the former windows version is JScript which is an implementation Javascript just using an different name for trademark reasons, but tied to an old version of the JavaScript standard (which is confusingly called "ecmascript" officially), plus some Windows-specific integrations, like ActiveXObject and COM/OLE support.
(There's also a later .NET version of JScript to add to the confusion).
> The apps that do perform operations "as root" do so by spawning a shell process running as root, usually using `su -c`. That's on a rooted device.
So in a scenario like what we're discussing here (a dev deciding to build an uninstaller their own way), on a rooted device (as per the GP's context of "this coming to Linux and Windows"), the app in question could still run as root, and this permission set from Android would solve nothing?
https://arstechnica.com/civis/threads/i-tried-to-uninstall-s...
There's also a feature gap that Google will probably never close, because why bother if you can't load the volume of data those power users are working with in the first place.
Installed software is considered to have a distributor, ie. a legally responsible entity that can be punished for any shenanigans.
I believe this is an outdated model and that with better tooling we could do what you envision.
Like installer.exe?
Installation-free software doesn't have to be web software.
While there ARE some tools that require a more invasive "installation" process (e.g., VMWare Fusion), the overwhelming majority of Mac software is installed by just dragging the application bundle into /Applications.
(App bundles are just special directories, more or less, so you're moving more than just the file, but it presents as a single thing.)
To remove an app, you just drag that bundle into the trash.
This usually DOES leave behind things like local user data or preference files, but those are inert text files and don't impact perf or machine behavior in the long run. Users typically waste more space on cat pictures.
The tl;dr is that "installers" and "uninstallers" only exist because Windows needs them. I have seen MANY MANY FOLKS come to the Mac and be VERY confused by this. "But you NEED an installation process! This can't work!" Nope. Windows needs an installation process.
I joked, in the 90s, that despite all the monopolistic chicanery from Redmond, their real lasting awful legacy would be the degree to which they lowered people's expecations about how computers worked, and I stand by that.
(Something else not needed on sane systems: "clean up" software. If you don't litter the file system with files when you add a program, you don't need special utilities to run that shit down and delete it later.)
Go to your Applications folder. Right-click any app. Click "Show Package Contents." Take a good look around.
What you describe works for a specific kind of app, the ones that can be easily sandboxed and don't have shared dependencies.
Homebrew and MacPorts are used for Unix software that requires a Unix-style package management.
Notable exceptions: Microsoft office, Google Drive.
D/ls from the store do what I describe above. The overwhelming majority of Mac apps going back to the pre-MacOS / OSX days work exactly as I described above.
A: So, people are resorting to injecting code in Explorer to delete in-use files in such numbers that it shows up in our top-100 crash report reasons
B: Well, maybe we should add a public API to Windows to support this incredibly common functionality that apparently has been missing so far?
A: Nah, let's just write a mildly condescending blog post that recommends using an unreliable workaround that is pretty much guaranteed to trigger any client-side intrusion detection software, that will set them straight
B: Right on!
(Meanwhile, somewhere, a third-party developer is gearing up to ship a kernel-mode driver to directly manipulate file system structures from their uninstaller, since their old solution kept crashing: can't wait to read the postmortem once the crash dumps from that one hit the Microsoft servers!)
A: There are a dozen ways to do this correctly. Which right way should we pick?
B: I don’t know, I found this code on the internet. Should I just use it?
A: Sure, if it’s on the internet it must be the right way.
I... don't think so? The particular problem here, is that an uninstaller executable needs to delete itself from disk after doing its main job.
Other than using MoveFileEx with a NULL destination file and the MOVEFILE_DELAY_UNTIL_REBOOT flag, then suggesting/forcing a reboot, I can't think of a straightforward solution.
And that solution instantly lights up your JIRA with 2 tickets:
#1: We MUST not suggest/force a reboot! Users hate it!
#2: CRITICAL BUG: uninstaller.exe still present after uninstalling product
So, then you try things like 'create a Task Scheduler job to delete the file', which then adds:
#3: PRIO 1: uninstaller.exe crashes if Windows Task Scheduler disabled
#4: SHOWSTOPPER: uninstaller.exe still not always deleted after uninstall. Why is this so hard?
Et cetera, ad absurdum. This then escalates to code injection (as described in the linked article), and (you heard it here first) kernel-mode drivers. So, if you're aware of a reliable solution, feel free to share in a comment here, for the betterment of the world!
And what's the problem with this?
> When our product’s uninstaller sees an undeletable file (possibly a DLL loaded in another process), it uses the MOVEFILE_DELAY_UNTIL_REBOOT flag to mark it for deletion, and warns the user “Please reboot as soon as possible to remove the remaining files.”
> However, some user uninstall our product, just to be able to reinstall it later, to the same location. And of course they ignored the warning. Once they reinstalled it, everything works, until a reboot.
That solution instantly lights up your JIRA with 2 tickets: #1: We MUST not suggest/force a reboot! Users hate it! #2: CRITICAL BUG: uninstaller.exe still present after uninstalling product
I'm sure it would be hard to add such a thing to Windows while minimizing compatibility problems but I'm equally sure that it's possible and that it's worth the effort.
Like, for instance, the time that they decided that LaunchAdvancedAssociationUI(), the previous officially recommended way to show UI to allow the user to associate file types with a program, just wouldn't work anymore in Windows 10. Instead of opening up the Default Programs UI in Settings, it just displays a dialog telling the _user_ to go there -- which is even modal so they can't even refer to it while doing so. No compatibility shim or grandfathering for old programs, they just broke all applications that used this like they originally said good programs should do for Windows 8.
Or the case of Dark Mode in Windows, which for some reason they've dragged their heels on implementing barely any Win32 support at all for -- even just simply a call to query whether it is enabled. The current silly recommendation is to obtain the foreground color through WinRT and do a dot product on it to compute luma and determine if it is a dark or light color: https://learn.microsoft.com/en-us/windows/apps/desktop/moder...
Or the fact that the official way of reporting bugs on the Windows APIs is the Feedback Hub, which is completely unsuitable for task.
I don't have sympathy for the Windows team anymore. Their lack of developer support is partially responsible for all of the hacks that applications have to do to ship.
Programmers would hesitate to use it, because they can't be sure the user of the uninstaller will be on a fully updated windows install.
The JScript workaround might seem like a bit of a hack, but it has an advantage of working on every version of windows all the way back to Windows 98.
Linux distros were pioneers in this aspect and "stores" of the modern world are just a clothed versions of package managers. of course, a power user should should have the right to change software sources (repositories). It should be possible to add decentralized package managers like it is possible with flatpack.
Installers and uninstallers are a relic of a time were most software people used were proprietary running under a proprietary environment. Many windows users are still locked in that proprietary closed world. If "Any sufficiently advanced uninstaller is indistinguishable from malware", I'd say that most (all?) microsoft software comes with extremely advanced uninstallers.
Package manager tracks every single file installed by a package, preventing overwrites by other packages.
Strict permissions prohibit software from littering all over the place.
What are you taking about?
Do you want package manager to remove yourb personal data of it was created by an application your decided to remove?
I sure have lots of junk in ~/.config I don't use anymore and probably never will.
Yes, but the modifications that are possible are limited by what the API of the package manager provides. Like Windows Installer does. If that doesn't work you create custom install uninstall logic, which has the potential to be less robust/reliable than what the package manager provides.
Regardless, installation on Linux is often just dropping some files somewhere and perhaps a modprobe.
In Windows you have so many things you can do: filesystem, registry, COM registration, GAC, file associations, etc.
registry -> dconf
com registration -> dbus objects
gac -> shared libs
file associations -> dconf settings
windows scheduler entries -> systemd user sessionAnd the GAC isn't really comparable, perhaps WinSxS is comparable, but the GAC requires a specific API and isn't a matter of just dropping a file in /usr/lib.
What is a package manager, if not an (un)installer shared between projects?
Sharing between projects is not how I’d frame this. Using a shared/standard OS facility is closer. This matters quite a bit when it comes to establishing trust.
Back when I was installing KDE on top of another distro, I'd also have the Discover front-end sitting next to Gnome's Software Center (or whatever it was). That should be somewhat expected and is more an indication of the bloat that major window managers have embraced.
The front-ends are decent about showing what source a piece of software came from, which is nice, but the last time I used them they didn't stop one from installing the same piece of software from two different sources.
All to say, back in the day it was so convenient to have yum or aptitude just take care of things installing software, resolving conflicts, etc. Now we're in a bit of a mire.
So having a package format or package repository doesn't necessarily prevent packages from doing arbitrary or stupid things (there is no difference between a .deb file or an .msi file in that regard). The app store ore repository might add a level of human vetting, but technically there is no difference.
I would argue that it's better to have most of the vetting be technical: ensure that the package/installer format doesn't allow making a mess. Sandbox everything, prevent writes in the wrong places etc. More modern formats like flakpak/msix/etc are of course better in this regard.
And the key thing about them is that they prevent a mess on the technical level so that the human vetting (and thus repository) isn't really required.
https://en.wikipedia.org/wiki/InstallShield
became a defacto standard endorsed by Microsoft and everybody else tries to look like it.
The real Windows "quirk" that this article skirts around is that Windows won't let you delete a file which is open and of course if you are running an executable that counts as an open file. This of course can be a huge hassle if you need to delete something and can't find the process which is holding the file open. It's particularly annoying for things like software builds where you really want a script that automatically and reliably clears a lot of stuff away so you can make a fresh start.
Now, POSIX has the opposite "quirk" that you can "delete" an open file and it completes right away because all you did was delete the link from the directory to the file. The file still exists because it has a link from the process that has it open, it really gets deleted when that link goes away.
That can get you in just as much or more trouble than the way Windows does it can, for instance if some series of events caused your system log to fill the disk, you can "rm" the log and the disk is still full. To really free up the space you need to "rm" and then restart the log daemon.
As for Linux it's got the problem of a proliferation of package managers in the sense of things like flatpak and snap and generally the (dumpster fire I think) of Docker images (like the place where they thought Docker would help stabilize their Python installations but somehow our data scientists kept finding strangely broken Python images that were real nightmare fuel for me; e.g. default character encodings that I didn't think anybody actually used)
Windows has realized the ability to install software in C:/Program Files/ and D:/Program Files/ and often in a user's home directory (if the installer supports it.) 20 years ago I thought rpm's sucked and thought "any software worth installing is worth building from source" where I could do
make-install --prefix=/usr|/usr/local|$HOME
that is I didn't have to beg for my sysadmin to do something like apt-get install nethack
I am impressed with the comparative speed of installing from a package manager, but I still can't install a local copy of software with one, and I should be able to.Pacman is ok, but I recently destroyed the Ruby installation on a computer while trying to use Vagrant and other Ruby packages. I still haven't figured out how to fix it. Lesson learned, stick to AUR and pacman repo.
For per-project dependencies, Poetry is pretty good, although there are other options too.
For Ruby, I think Bundler is the venv/Poetry counterpart. Not sure what you could use for installing global tools though.
Windows does have installer/uninstaller infrastructure called MSI (https://learn.microsoft.com/en-us/windows/win32/msi/windows-...), but ultimately it's up to developers to choose to use it.
I saw Linus from LTT brick his installation (in Pop_OS! I think?) but that was a clear user error.
A user error that can happen to any user who isn't Linux savvy and just wants to paly games, and not learn how a package manager works, and that it can uninstall your desktop environment if you aren't proficient with Linux, is no user error but OS error.
How many MacOS or Windows users expect that going through the installation steps of Steam, it can uninstall your desktop environment? For that user demographic, that is a clear OS issue.
So, he opened a console (like any user?), then used apt-get ... which had a WARNING ... "This should NOT be done unless you know exactly what you are doing!". He then had to type "Yes, do as I say!" in order to "do something potentially harmful" ... and then, what a surprise it did something harmful!
That's a user error that will only happen to people who are cocky, people who are idiots, or people trying to firm up their long held stance that 'Linux isn't for gaming'.
I saw the LTT video the following day to release, loaded a VM up, installed Pop_OS (my first time) and installed Steam, no issues. Very simple. All button clicking.
>going through the installation steps of Steam //
That's mis-characterisation, he went through the Steam install steps (click install by the Steam icon), the OS told him there was an error and refused to do it. End.
Then he went sudo-ing and ignoring warnings. "You can delete System32 this OS isn't ready for users!".
You'll tell me now installs never fail on Windows, presumably, despite having experienced them myself.
It's 100% an OS UX error you're trying to spin into an user error.
mundane gripe: Uninstalling an apt package is too complicated and it is beyond me why it isn't a single command in 2023. I have to skim stackoverflow answers every time I need to do it (multiple times a year).
I believe MSI files were originally supposed to be almost atomic in use (software is either installed or uninstalled), providing transactional operations for installing software and limited UI options. I remember (but can't find) an old quote from someone on the team that designed the file format, basically saying something among the lines of "giving developers the option to execute random commands from MSI installers was the worst decision we've made".
Any open file
Any file that was opened without specifying FILE_SHARE_DELETE in the call to CreateFile[1] (the Win32 equivalent of open(2)). Unfortunately, most language runtimes that wrap CreateFile tend not to pass that flag.
[1] https://learn.microsoft.com/en-us/windows/win32/api/fileapi/...
[0]: https://github.com/golang/go/issues/32088#issuecomment-53759...
[1]: https://bugs.openjdk.org/browse/JDK-6607535
So to me it's just not there...
> jstarks commented on Jun 18, 2019:
> [I]n the most recent version of Windows, we updated DeleteFile (on NTFS) to perform a "POSIX" delete, where the file is removed from the namespace immediately instead of waiting for all open handles to the file to be closed.
[1] https://github.com/golang/go/issues/32088#issuecomment-50285...
While full Unix-like behaviour is only available on Windows 10 for the past five or so years, you can still have the old win32 behaviour on older systems (delete once the last file handle is closed).
AFAIK, that's exactly what they do. And it can cause problems, see for instance https://learn.microsoft.com/en-us/previous-versions/technet-...
Then at least you have one attack vector less per application.
Yes, also on servers I run arch... let's me sleep better :)
winget: https://learn.microsoft.com/en-us/windows/package-manager/wi...
If browsers werent updating itself we would have huge security mess
Edit: since there still seems to be confusion I'll try to be clearer. On NTFS you can delete an open file. This is a solved problem. The DeleteFile API even does this by default now. The thing that prevents deletion is an OS lock. This lock only prevents deletion. Renames are still allowed.
In that case, the file exists only for the process which has already opened it.
The High-Performance File System, a.k.a. HPFS, has been included in OS/2 1.2, which was launched in November 1989.
HPFS was a very innovative file system (its chief architect was Gordon Letwin). It has introduced the Extended File Attributes. It had directories implemented with B+ trees, like IBM VSAM (introduced at some time between 1973 and 1978). It had long file names and Access-Control Lists, like the Multics FS (1965). It used cylinder groups like the Berkeley FFS (1983). It used file extents, like the SGI EFS (1988).
Among the UNIX file systems, the SGI XFS (1993, almost simultaneous with NTFS) has been the first which has added all the innovations introduced by HPFS.
That's not exactly rocket science. Anyone can do that, there's tools for that.
(It's weird that this comment was upvoted twice, now down voted twice)
This is the quote I'm referring to: https://en.m.wikipedia.org/wiki/Clarke%27s_three_laws
It's the third of his 3 laws
Imagine having a package system, instead of that crap.
Imagine a file explorer that doesn't simply prompt for admin rights when you need them, instead silently failing.
Imagine using a file explorer you don't like.
So something like "/var/lib/postgresql/data" in the container becomes something like "/app/my-postgres-service-12/var/lib/postgresql/data" on the host.
I have just one directory to backup, I can also move it to different nodes entirely and OS upgrades don't break anything either due to the running software being decoupled from the OS somewhat.
But my Linux and Windows desktops? It's absolute Wild West over there and I just reinstall the entire OS every few years - they're beyond saving.
UNIX resolved this decades ago: the filename is just a link to a structure on disk. You can delete a file even if it's used, since the file is there but you only delete the reference (filename pointing to the inode) that it has. Programs that have the file open can continue using it. When the file is no longer open by any programs the reference count is checked: if it's zero it means that the file can be safely deleted.
Not a difficult thing to implement, since they did it in the 70s... still Widnows doesn't get it.
In any case, I am not sure the Unix way is the best API design here. Sure, it does help in these scenarios, but now you can have hidden files lying around in the filesystem being used that I can't easily tell are even there.
Under Linux, I have occasionally had the challenge of figuring out what process is holding open a large file that has been 'deleted'. This starts with recognizing that you're getting disparate values from different tools you might use to look at disk utilization. It's confusing as hell the first time you run into it without some established context as to why that might happen.
On deletion of a file that is open by another process, Linux does not warn me about that. Windows will give me a clear error message that the file is open by another process. Linux chooses to let the problem sit because it might just take care of itself once the process ends, but that decision kinda sucks when your disk is full and you're scrambling to get some space back. Windows forces me to deal with the conflict right now. That adds work in some cases while avoiding the confusion and risk factors of multiple perspectives about a volume's utilization.
It's not "Widnows (sic) doesn't get it". Windows made different choices. As another comment expressed, Windows has all it needs to do this at the FS layer, but chooses to use OS locks on top of that.
I know RPMs and DEBs follow a fairly strict patter, in that they keep track of everything installed so it can easily be cleaned up. Why can't Microsoft launch its own set of standards that achieve the same goal?
OneNote also hijacks your filesystem, giving Microsoft full control over it via the internet.
Not to mention, many of the windows defaults are impossible to uninstall, and if you figure out how to, it will reinstall.
Seriously, Windows has gone full malware/bloatware.
I didn't want to, but I switched to Linux. The strange part, despite linux being 'rough', I'm saving a ton of time. Multiple times a week Windows would automatically reboot, causing me to reopen everything(which is slow), figure out what to do with autosaved files, and I might have lost some data. On linux, I don't waste time on these reboots, and can spend those 5-10 minutes solving the 2 issues I ran into. Now those issues are solved and now Linux just works.
1. Clean up whatever you can in the uninstaller. If you're a decent human being this includes the registry, appdata, temp, and a toggle to delete wherever your app spews its config.
The only thing left should be the uninstaller .exe and the directories leading up to it, since Windows doesn't allow you to unlink a exe/dll that's loaded in memory. Your uninstaller should be static linked so it doesn't need to clean up its own .dlls.
2. Figure out how to delete the uninstaller. This can either be some stack smashing butt clenching fuckery, like the guys from the article did or you can be a normal fucking person and just atexit() a batch script in that does
SLEEP 5
RMDIR program_dir /whatever /flags /needed
The SLEEP is to ensure the uninstaller is closedYou can put the batch script in %TEMP% and hope something will clean it up, or if you're feeling extra nice you can set up an action to get it cleaned up at next boot.
After that you're also equipped to write self-deleting malware, which is a far more applicable skill.
The solution really isn't that hard, it might just feel a bit "dirty" (using a scripting language to do some work). But all that "stack smashing butt clenching fuckery" surely is much worse and less stable, so I'm not sure why one would even go that way...
If you can set an action to get something deleted at next boot why not use that to delete your uninstaller itself, rather than adding an extra layer of indirection?
edit: https://learn.microsoft.com/en-us/windows/win32/api/winbase/...
It's the MOVEFILE_DELAY_UNTIL_REBOOT flag. Honestly maybe you can call MoveFileA on the entire directory tree you want to delete starting from the .exe and spare yourself the script, I don't see an obvious reason why it wouldn't work.
There are cases for allowing memory protections to be changed, and because of that, it's possible and people use it. (code-generation / JIT etc.)
the stack being executable used to be a big problem, for example in 1995 with things like stack overflows running wild, but I think those have been largely mitigated by having things like stack canaries/checks stop stack overflows from being done (stack smashing detected!). those canaries were introduced around 1999 or so, and yes there are ways around it (rops/jops etc.), but not through the simple overflows from 1995.
I am all for not allowing it, but admittedly, that's likely some impractical utopian stance on the matter regarding the current use of operating-systems.
Perhaps in 2995 :)
I used to bundle an INF file in a CAB archive (converted to self-extract executable) to distribute software. Using only Microsoft's own tools.
It doesn’t rely on it. It’s just something that’s possible.
Where do I look in the OS for the manifest of all installed files from an installer?
Thanks, I'm mostly a Linux user and I've sorely missed a `dpkg -L` on MS Windows for ages.
Now they aren't Apple, in telling developers "use this or get lost".
I never digged into the question, but why is it used, what benefits did it provide over the UNIX unlink behaviour?
How do you defragment/move files that are unreachable on the file system? How do you shrink volumes when you can't move files that need to be moved?
Edit: Actually, hmm... as I type this, I suddenly recall you can also open a file by its ID on NTFS. And you can enumerate its streams as well. So presumably this could work on NTFS if you loop through all file IDs? Though then that would make these files still accessible, not truly unreachable.
Edit: Actually, hmm... see edit above.
In fact, I think it's not just a (slow!) hack, but a buggy one too. Every time you open a an object that doesn't belong to you, you extend its lifetime. I think that can break stuff. Like imagine you open a socket in some server, then the server closes it. Then that server (or another one) starts up again and tries to bind to the same port. But you're still holding it open, so now it can't do that, and it errors out.
https://unix.stackexchange.com/questions/68523/find-and-remo...
This is how it works on UNIX. Generally better then apps randomly failing because a file(name) is held open somewhere by something.
Well that's what I was getting at, it would suck to not be able to move around file blocks just because a process is using the file. That "sooner or later" might well be "until the next reboot". The current strategy makes it possible to live-shrink and live-defragment volumes on Windows - ironically, saving you a reboot in those cases compared to Linux.
But actually, maybe not - see the edit in my original comment.
I face builds failing because I have a terminal open in a build output directory or a textfile in an editor open is far more often, and annoys me more. (or being unable to replace a running service binary of a service being developed/tested, needing to stop the service, replace it, and start again. Or failing log rotations because a logfile is open in a Notepad. Or...)
Also see my link for a solution on unix, where you can indeed fix this problem, or simply kill the process holding the file. I didn't need to defrag my computer in the last 20 years, neither on Linux, nor on Windows, but hey, it makes me happy that my daily work is hindered for this hypothetical possibility. (which could and is solved in other OSs with appropriate APIs for the job)
Also the original post is about Windows installers... don't get me started on the topic (or windows services), please.
> Also see my link for a solution on unix, where you can indeed fix this problem
Looping through every FD of every process just to find ones that reside in your volume of interest is... a hack. From the user's perspective, sure, it might work when you don't have something better. From the vendor's perspective, it's not the kind of solution you design for & tell people to use.
In fact, I think that "solution" is buggy. Every time you open a an object that doesn't belong to you, you extend its lifetime. I think that can break stuff. Like imagine you open a socket in some server, then the server closes it. Then that server (or another one) starts up again and tries to bind to the same port. But you're still holding it open, so now it can't do that, and it errors out.
> or simply kill the process holding the file.
That process might be a long-running process you want to keep running, or a system process. At that point you might as well not support live volume shrinking or defrags, and just tell people to reboot.
> Also the original post is about Windows installers... don't get me started on the topic (or windows services), please.
This seems pretty irrelevant to the point? It's not like they would design the kernel to say "we'll let you do this if you promise you're an installer".
> I face builds failing because I have a terminal open in a build output directory or a textfile in an editor open is far more often [...]
Yes, I agree it's frustrating. But have you considered the UX issues here? The user has C:\blah\foo.txt open, and you delete C:\blah\. The user saves foo.txt happily, then reopens it and... their data is gone? You: "Yeah, because I deleted it." User: "Wait but I was still using it??!"
And for the live volume shrinking: the kernel can solve this problem, it there is a need for this, there is no need for this invariant for this feature, as it is not only possible to do it via the same APIs offered for ordinary basic file manipulation gruntwork. On unix basically a filename is disassociated from the inode, but afaik the inode holding the blocklist still exists, will be cleaned up later, thus it can be updated if its blocks are moved under the high level filesystem APIs.
You just made a strawman you are sticking to.
You're not the typical customer of Windows.
> Grandma would not have more problems than unplugging the pendrive where the file was opened from, and trying to save it, for example
Actually she would, because in that case writing to the same file handle would error, not happily write into the ether.
Also, you have one tech-savvy grandma. I don't think mine even knows what a "pendrive" is (though she's seen one), let alone try to open a file on one, let alone try to save her files on it, let alone use pen drives on any regular basis.
> You just made a strawman you are sticking to.
The only strawman I see here is your grandma using pen drives to save files.
What I'm pointing at are real issues for some people or in some situations. Some of them you might be able to solve differently at a higher investment/cost, or with hacks. Some of them (like the UX issue) are just trade-offs that don't automatically make sense for every other user just because they make sense for you. Right now Windows supports some things Linux doesn't, and vice-versa. Could they be doing something better? Perhaps with more effort maybe they could both support a common superset of what they support, but it's not without costs.
And with it, less data corruption issues.
Skype for Business stop working, I run "Repair" on the installation, progress bar zips to completion. Skype for Business is now gone. Oh, and so is the rest of Microsoft Office.
So not quite what I'm looking for :D
Sure Nvidia is a headache on Linux, but once you solve that, its all fun and games.
Doxillion Document Converter by NCH software.
It has been 4+ years, and I am still manually cleaning up stuff it left behind on my machine.
Do not use anything by this company, unless you intend to reimage after.
"Any installer that is distinguishable from malware is insufficiently advanced."