A command-line installer for Windows | Hacker News Reader

A command-line installer for Windows

scoop.sh

120 pointsanuragsoni96 comments

I struggle with all the alternate 'installation managers' for Windows. Microsoft has already done this with their MSI system (Although even they have basically abandoned it in favour of the APPX stuff). MSI was a good format, and could be installed silently with the right switches (/q/b), support alternate installation options via MST files, and even patches via MSP files. Even the specific DLL dependencies ('DLL Hell') got fixed via manifest files/attributes. Microsoft published the MSI specification early on, and encouraged everyone to support it.

However, the core problem with Windows software is there isn't a primary software repository for all approved and tested software, like there is for *nix platforms. Ideally that's what needs to be fixed first before yet more client-end installers get created.

To fix this, based on the current landscape, all third-party client-end installers need to support all existing third-party repositories. Even better, everyone agrees on a standard JSON format (or whatever) for their repository manifest and all third-party installers understand that. Then just like Linux et al. all you have to do is add the new repository URL to your installers config, and all the packages advertised within are immediately available.

It seems really simple to fix - but people have to co-operate.

I think the real problem is Microsoft doesn't care. It's probably a higher priority than Windows Backup, but only just.

"the core problem with Windows software is there isn't a primary software repository for all approved and tested software"

scoop uses portable binaries provided from the original source. For many people that's good enough.

Maybe chocolatery is more to your taste.

MSI was the format where you needed to keep the original installer around merely to uninstall it and they eventually hacked-in a stripped down version of MSI installers because well, someone realized how ridiculous that was?

Good riddance.

Not really, that was badly written installers in InstallShield and friends.

Windows installer made a copy of the msi, just in case it was needed later and put it into C:\Windows\Installer. It kept it even for packages, that were uninstalled or superseded.

I never remember having seen such folder.

Having been doing Windows development since Windows 3.0.

I have been hit by such errors, which on my case were caused by .exe installers.

It is hidden+system, owned by builtin\Administrators and competes with WinSxS, who will take most of your diskspace.

I stand corrected. All those years without noticing it. :\

On my Win10 system, that directory is ~600mb, ~300 files and 17 directories. Not advocating for this system, I just don't think it's that much of a problem.

I have a Win8.1 system, where it is 40 GB. That crossed the annoyance line some time ago.

Well, there is the Microsoft store, but it's clear that moderation is difficult to keep bad or worthless apps out.

Beyond that, there probably isn't a huge demand for a command-line installer or other package manager. Most consumers wouldn't know how to install via the command line, and most enterprise environments will have images with necessary software preinstalled or push them out through SCCM or other such management interface.

All of these attempts at alternate installers really only seem to appeal to developers, but in the scheme of things, there are other more pressing concerns even for them.

How would I add another repository to the microsoft store in order to have nightly builds of a specific app or third party apps available?

Is there an apt-like command line interface for the microsoft store?

This is technically possible (kinda how enterprise apps are deployed) but is not exposed to consumers fwik.

I have been using scoop for about 1.5 years and it’s wonderful. I contribute to it as well. It’s simple, robust, reliable. I might prefer it over what you’re suggesting for Windows. I just can’t imagine Microsoft getting that right.

I agree, but for what it’s worth, scoop manifests handle all of these pretty easily. I have contributed a handful of auto-updating manifests, including numerous installation managers, and I check in regularly but none have required modifications so far.

Scoop is, I believe, the best solution we can hope for.

agreed - it is the best one that I have found thus far.

I have a different fix: don't use installers at all and release applications as portable self-contained directories. You don't need fancy support for silent installs, you just push the directory and you're done. No need for a middleman repo either.

Second that. Every single Windows proggie pollutes so many different directories including system ones as well as registry entries - so it became impossible to clean OS and leaving it cluttered with leftovers all over the places.

Windows uninstallers and cleaners became a whole industry by itself.

For software targeting more advanced users and with limited dependencies, I agree. But if your software has large shared runtime requirements, is expected to be used by simple users expecting an icon, has non-trivial installation checks, etc, an installer is the way to go.

I've been using chocolatey for a while, it seems to be the closest thing to something like this.

Many (most?) chocolatey packages are essentially two lines of plaintext configuration: the URL of an MSI, and the command line arguments used to run it silently. Chocolatey does add some value on top - metadata, tracking what you've installed - but all that is entirely separate from the MSI database that Windows already uses to do the same sort of stuff. The biggest value of the main chocolatey package repo, in my opinion, is that it's effectively a curated list of direct download links for MSIs.

EDIT: I forgot that it can also verify checksums, there is obviously value to that.

I did not know this... I might actually have a proper look at Chocolatey now. Thanks!

Microsoft has been pushing all the good parts of MSI into the APPX stuff which is now called MSIX. It has had support for silent installs, options, patches upgraded or added over the last couple Windows releases, with I think a last few pieces scheduled for the next Windows release.

The MSIX format is supposed to support everything MSI currently does soon, and has preview support for even Windows 7.

> However, the core problem with Windows software is there isn't a primary software repository for all approved and tested software, like there is for -nix platforms. Ideally that's what needs to be fixed first before yet more client-end installers get created.

This is certainly a goal of the Microsoft Store, to be a central/primary software repository for Windows software. Sure, it has the "ugliness" of also including all the fun monetization things that -nix can often ignore because FLOSS doesn't necessarily have to worry about paying the bills in quite the same way. But it seems silly to me to turn down the usefulness of a "primary software repository" simply because it also doubles as a retail store for proprietary/closed-source software.

Reading the headline made of think of installing Windows via the command-line (say via imagex). Which is actually very useful in certain tricky situations when you mess up an install.

Once I didn't have enough space on a flash drive to install Windows via an ISO...but I did have a liveboot ubuntu system and 2 spare drives. Interestingly enough imagex and WIM tools were ported to linux, and it was a surprisingly easy experience.

Probably a nitpick but wimlib isn't a port. It's a separate implementation based on reverse engineering the Windows tools and format.

I did that too! The world certainly has changed now that installing Windows is easier to do from Linux!

I'm also looking towards using the winre partition to store wins and some kind of iterative snapshots for PITR recovery.

Any suggestion is welcome!

There was a tool called Unattended [1], built to install Windows from Linux via PXE booting. I've used it a couple of times, but it left off at Windows XP (and Server 2003), which is a bit too far beyond for me now. Anyway, I think Windows 7, and definitely Windows 10 are easy to boot from USB, which is almost as good as a network boot.

[1] http://unattended.sourceforge.net/advanced.php

Windows has always been easier to install than Linux lol. Also wmi's have existed forever. Most Windows specific knowledge is found closed with enterprise sysadmins and not shared out in the open- who should be blamed is your choice. I blame ms

Just a note, ImageX is superceded by DISM, and both of them utterly fail at keeping extended attributes (despite the command line flags). So what you restore won't be the same thing as what you backed up. (I don't know of any tool that handles all of EAs, reparse points, etc. correctly.)

Do you have any examples of failures? I’d like to know what specifically is not working.

Extended attributes? That is the specific example. Create a file with EAs and see if you can back up and restore it with the same EAs.

Where are the NTFS EA used, if that's what you're writing about? I can't find anything except:

https://flatcap.org/linux-ntfs/ntfs/attributes/ea.html

"Used to implement the HPFS extended attribute under NTFS."

Is your complaint is that the tools that are used to make the image of the installation files of the Windows don't include that? I assume that $EA entries don't exist in the installation files of Windows, for which the DSIM tool is used (and it's surely not an universal backup tool).

And how do you read or write these? And what do you use them for? I was able to find only ZwSetEaFile and ZwQueryEaFile which are documented for kernel-mode drivers? Are there any plain user-level API's? https://attack.mitre.org/techniques/T1096/ I also find: "it may (be) worth noting that Windows does not contain any api that can be used to remove extended attributes."

WSL uses them for one (so good luck backing up WSL installations) https://blogs.msdn.microsoft.com/wsl/2016/06/15/wsl-file-sys...

Other stuff does too, e.g. https://docs.microsoft.com/en-us/windows-hardware/drivers/if...

Those are user-level APIs as well, though usually called via NtQueryEaFile and NtSetEaFile.

And (part of) my complaint is that DISM's /EA flag is broken.

you can use windows build-in powershell feature "Install-Package" with package provider "chocolatey" just like this on a fresh windows install:

  Set-ExecutionPolicy unrestricted
  Get-PackageProvider -name chocolatey
  Install-Package chromium
  Install-Package 7zip
  ...

That sounds very neat. Haven't used Windows in ages but likely have to in the future, this might make it more manageable.

Is there any drawbacks to this considering this installation method isn't listed on the installation page on their website?[0]

The closes I found to this installation method was:

"When you have Visual Studio 2010+ and the NuGet extension installed (pre-installed on any newer versions of Visual Studio), you can simply type the following three commands and you will have Chocolatey installed on your machine."

  Install-Package chocolatey
  Initialize-Chocolatey
  Uninstall-Package chocolatey

[0] https://chocolatey.org/install#installing-chocolatey

I discovered Chocolatey during a Windows stint at work ~3 years back and found their package quality to be very poor. I'd go as far as saying packages had a 50% chance of leaving you with a usable installation of software.

We ended up changing provisioning scripts to download .msi installers directly and run them with appropriate switches.

I also remember their website having a lot of "maintainers required" notices - I got the impression that the situation was a little desperate.

Has package quality improved? Is Chocolatey a genuinely reliable way to install software, now?

I've been using it since about 2 years and never had any problems installing anything. I guess I'm mostly using popular well-maintained packages though, so YMMV if you are installing some more obscure software.

I'm not sure about the maintainer situation. Some packages get updates quite regularly while other packages are always a few versions behind the latest release.

No. it has only gotten worse. Scoop is the way to go.

I feel like Chocolatey had some good moments early on, but I realized that I stopped using it because I started to feel like I needed to audit every PowerShell installation script it tries to run (and that's the majority of how it operates; the majority of "packages" are just PowerShell scripts that in turn download an MSI from a well known location, checksum it [maybe], and run it), because of all the poorly maintained packages and some increasingly worrisome PowerShell scripts in some of them. Particularly with things like Software Development tools, you could also never tell if a package was "just" a minimal install of the software or some particular developer's preferred kitchen sink, and the package dependency model alone wasn't enough to tell you because it might just be buried in the PowerShell installer directly rather than in the declarative dependencies.

As someone else notes, weirdly Chocolatey's packages seemed to get worse and grow staler after the Kickstarter than before. I guess in hindsight it's about where I stopped feeling an interest in volunteering to help packages, but I'm not sure if it was subconsciously about "what was the Kickstarter money for?" or not. I feel it was more the "kitchen sink" problem that hurt my interest in Chocolatey most directly, because those maintainers would happily sit on the most common name for an application (say, "python") with their kitchen sink, wouldn't take suggestions to better their packages and then you just ended up having to wade through search listings for "slim-python" or "real-python" or "just-python" or "brians-lite-python" or "python-dave13". It stopped feeling like a helpful community and more "every developer for themselves" at that point.

Warning: chocolatey is bad news. I can’t say if it was ever a decent pkg manager or why it went down the tubes if it was, but maybe it was the offort to monetize it as an already-super-bloated system.

Just run with scoop. It’s very tasteful, and even makes powershell look attractive.

I will second this! I have tried a bunch of them looking for a viable apt-get type solution on my Windows dev machine. On the Mac I can use homebrew, but Chocolatey was nowhere close to homebrew... Scoop is the viable alternative.

I've been exclusively using Linux/*BSD (Slackware/Gentoo/Arch) for personal use since upgrading from my K6-2 500 to an AMD Barton 2600+ because finding, installing and managing "trustworthy" software is easier.

Linux has hardware compatibility challenges and software with bugs, but so does Windows. I don't, however, have to search through monstrosities like Download.com. I don't have to figure out what software may have a new release that is important to me. Critical dependencies are handled automatically. Every distribution may handle these in slightly different ways but these basics hold true.

Every time I've had to use Windows I've spent an enormous amount of time searching for and installing things. Once installed I then discover the software has changed to a limited free trial or some other nonsense that ends in another search, install and disappointment loop.

It's tools that have shown up in the last few years like scoop that have made dealing with Windows at least somewhat tolerable.

Curious how this compares with Chocolatey

I was curious too, turns out it's addressed in the wiki:

https://github.com/lukesampson/scoop/wiki/Chocolatey-Compari...

Short version:

- Installs to ~/scoop/ by default.

- No UAC popups, doesn't require admin rights.

- Doesn't pollute your path.

- Doesn't use NuGet.

- Simpler than packaging.

- Simpler app repository.

- Can't always install a specific version of a program.

- Focuses on developer tools.

Can I install MS office using scoop? Or other proprietary software that usually install globally and need admin permissions?

Choco can do that (not that I am affiliated). Also, I like my "path being polluted". The least these programs can do is symlink to a single binary directory so I can run them from anywhere.

Simpler repository is just a euphemism for- we have little.

As an end user I have no business whether it does it doesn't use nuget.

I'm not trying to diss - I respect and appreciate not players in this space but its important to say it like it is. Also, given how "open" choco ecosystem is- I'd love to see non-official tools on top of that.

I gave up on Choco because it got too hard. I could never tell which package I should install, I was never sure if I should be installing from an elevated prompt or not, and then they added a bunch of premium plans to choose between with a confusing feature matrix.

I really want Chocolately to suceed, but so many packages are outdated, and there are a lot of duplicates, so you often don't know which one you should get. I find the web UI very clunky too.

Have you ever tried to uninstall Chocolatey? When I tried doing that for the first time, it was a terror so I never looked back.

Most things seem to want an elevated prompt to install, I wish the failure modes were better.

It also seems pretty slow, having to download stuff to even tell if it can install a package.

I have to partially agree... I usually create .cmd scripts for windows that have what is needed, with enough checks to rarely have interference, there are some things that don't come out right sometimes though.

The question for me is, can I get all the VS build tools, python 2.7.x and 3.x, git for windows and node.js all installed and where it will work from a command prompt with scoop? Setting up a new developer is the primary use for the scripts I have currently. I have one for windows (chocolatey) and one for mac (homebrew), I wouldn't mind if the one for windows was cleaned up a lot.

>The least these programs could do is symlink fo a single binary directory so i can run them from anywhere.

Yes, and scoop does exactly that! Everything is added to the path, and if you do like how it’s added to the path, you can easily modify the JSON manifest and submit a PR. The community is thoughtful and considerate but generally against bloat.

Scoop also adds shortcuts to the Start Menu.

There are some proposals on the table (and have been for a while, without much support) to do more customizations like context menu modifications, but some applications (like VS Code) at least give some instructions for adding that yourself when you run the install.

Thats a major positive score from me.

I really like that the manifest files are just single JSON files.

Chocolatey packages are pretty complex [1], and even worse is the automatic system [2] used for building the main packages [3], which tries to keep everything updated. There's a very steep learning curve if you want to contribute and fix packages.

Contrast to the simple manifests in the scoop main bucket [4] and it's looking much easier. There's some variables there for auto-updating that aren't immediately intuitive, but it doesn't look nearly as complex as chocolatey.

Between the extras [5] and other community buckets [6] this is looking pretty compelling -- can't believe I've never heard of it before.

[1] https://chocolatey.org/docs/create-packages

[2] https://chocolatey.org/docs/automatic-packages

[3] https://github.com/chocolatey/chocolatey-coreteampackages/tr...

[4] https://github.com/lukesampson/scoop/tree/master/bucket

[5] https://github.com/lukesampson/scoop-extras/tree/master/buck...

[6] https://github.com/lukesampson/scoop/blob/master/buckets.jso...

For me the big reason I switched is the first two. Installing locally to a single path makes for a lot less messy filesystem, and I don’t have to go admin to use it.

Also frankly, Chocolatey’s repos have become a mess, full of broken and untrustworthy software, and I just do not trust giving it admin permissions anymore. I don’t know what it is about windows software distributors that seems to lead all of them down this road, but I hope scoop can hold out...

As to windows software requiring admin to install. A lot of it comes down to that for the most part, software that installs globally should be limited to admin users. That doesn't mean that a lot of windows software can't install locally.

I do kind of wish that scoop used ~/AppData/Local/scoop instead of ~/scoop (from other comments). Looking into it more, it's interesting. I just didn't get a lot of insight from the home page in the post link. But I do agree with most of the troublesome issues with Chocolatey mentioned in this and other comments.

You can change where it installs by setting an env var.

sweet... may check it out to see if I can get what I need for my baseline scripts that currently use Chocolatey.

I’ve been converting as much as I can from Chocolatey to scoop, mostly because the install experience (no admin/uac).

I had a great experience, especially after adding the “extras” bucket. The last time I tried (a year or more ago) I found a lot of apps missing. Seems that has improved quite a bit.

Linux dependency managers are just that: they manage dependencies. Most Linux programs expect particular libraries to be installed system-wide. Chocolatey was modeled after this concept, with packages that need other packages, volunteer package maintainers, a hosted package repository, etc. All this comes with a fair amount of complexity.

Most Windows applications, however, are fully self contained. They ship their dependencies right inside the installer. The key insight behind scoop is that on Windows you don't need a dependency manager. You just use need something that can download and extract installers without all the next, next, ok nonsense and without needing admin rights.

So that's scoop. It downloads an installer from the program's website, extracts the files to a directory under your homedir, and sets up a few PATHs and symlinks.

It has no repository that needs to be hosted and maintained: after all, installers come straight from the source. And the definitions of which program can download which version from where and what should happen next are tiny JSON files managed on scoop's GitHub repo. Want to tell scoop about a new version of your favorite program? Create a PR.

Chocolatey is a great effort that has taught many Windows users that yes, we can have that apt-get goodness too. But I hope that Scoop takes over as the de facto install-software tool. It's so much simpler with very few downsides. Its design is spectacularly good.

Scoop has support for dependencies, and it is really easy to fit into PowerShell scripts :)

True, but its dependency support is much simpler than that of competing tools. I don't think it has fancy version resolution stuff and I don't it needs it.

It's usually dependencies on other programs, eg scoop itself depends on git (to pull install instructions from GitHub). That's different than having a system wide shared libffmpeg.

From what you say, it seems to be like Arch Pacman as opposed to Debian Apt (chocolatey): upstream is largely untouched and the package managers is basically just orchestrating. Is that correct?

I guess - I have no experience with Pacman.

Scoop doesn't even (currently) have the concept of package maintainers. Anyone can make a PR with bumped versions or other fixes (though I understand that as Scoop's popularity increases, some cracks in that model are starting to show).

The important metric here would be package activity. If you don't have the package or its out of date, all other benefits don't really matter. Granted, in the Windowsing world, package count is a lot lower, as it's all about big applications and not minor libraries and utilities as in Unix (or worse). So it's easier to catch up.

The best thing for a Windows packager would be some kind of automatic change detecting site powering it -- so if there's a new release for WinWidget appearing on the Download section of Frobnicorp. Inc, the recipe for the previous release is tested and updated.

Chocolatey repository is a mire of abandonment and nuget is utterly unreliable so the first thing that comes to mind is "favourably"

>>Scoop reads the README for you<<

I'd rather just have a clear and concise README. Better yet, if vendors distributed their applications as single bundled EXE files like back in the day.

Don't take this the wrong way, I haven't used scoop and I have nothing against it. Just wish there weren't a need for it.

Scoop together with a large collection of portable software is a godsend in a corporate environment, where you don't have admin privileges on your work machine and getting up-to-date software approved and installed by your IT department is a royal pain in the ass. I can confidently say that were it not for Scoop, I'd be in order of magnitude less productive at my work due to the lack of necessary software. Scoop is one of those things that makes a potentially unbearable work environment a productive and even enjoyable one. Yes, Scoop is not perfect. Sometimes it breaks and I have to manually fix it (it likes to uninstall outdated packages before downloading and verifying the checksum of the updated ones). But the time spent on fixing it is easily recovered by boost of productivity that it provides.

I realise I can find packages by scouring the GitHub repos[0][1], but I'd also appreciate a web search UI, especially as the number of available packages grows. This could also let me easily discover what version(s) are available etc.

I can't seen to find it, but does such a UI already exist?

[0] https://github.com/lukesampson/scoop/tree/master/bucket

[1] https://github.com/lukesampson/scoop-extras/tree/master/buck...

https://repology.org/

It's a common web interface on top of many package repositories, including this one.

Thanks, I've never come across this before, and it looks really useful for finding Linux packages, even if it's not exactly the most user-friendly UI!

My main machine broke two days ago and I had to buy and setup a new one quickly to resume working. I used scoop to install all applications I needed to work. It was easy and all the apps stay organized in their folder inside my home folder. I find scoop very refreshing and easy to use compared to other windows alternatives. I really recommend it to people, specially developers.

This looks pretty good. At the moment I'm using MSYS2 on my work machine (have to use Windows).

Any thoughts on this installation method?

    iex (new-object net.webclient).downloadstring('https://get.scoop.sh')

I'm not sure IT Security here would be too impressed.

You can download/run the powershell script separately.

`curl | sh`, take 2:) I hope nobody would be impressed, unless there's some safety features that aren't obvious here.

FYI powershell comes with curl now (and ssh).

I'm aware of openssh on NT being a thing now, but last I saw I thought powershell's "curl" was just an alias to their equivalent, which means that ex. options don't actually match for anything non-trivial. (IIRC, they did this with lots of commands; `ls` gets mapped to a local equivalent, but `ls -lahr *.foo` would not do what you think it should)

It's the real curl[0]. I actually like the aliases and they've even added ctrl-shift paste in the options. Maybe they should be the default.

The windows filesystem still makes no sense but that's an NT issue (dropbox cant sync con.sh???). SSH has a surprisingly native feel but graphical apps (vim, screen) still have rendering problems, and the ssh-agent daemon isn't set up right by default.

Until they make Windows LTSC unusable, it's not _too_ horrible right now. But for a serious programming project there's still no question for me (but fix your damnable sound drivers Ubuntu).

[0] https://blogs.msdn.microsoft.com/commandline/2018/01/18/tar-...

When you do it from a non-admin powershell, it's precisely as secure as downloading and running an installer.

It's easy to add new bucket entries by adding json files. Also if I understand it correctly, their auto update feature allows many apps to be self maintained i.e. you don't have to update the json when a new version comes out.

Why handle windows only? This was posted recently on HN : http://gofi.sh - Fish - A systems package manager like homebrew but cross-platform

Why in the world does it symlink stuff into C:\ProgramData? Normal Windows installers don't do that.

I guess it is for cross user install support but that can be achieved in other ways easily

Scoop is about the only thing that works on my stupidly locked work laptop. Unfortunately, McAfee AV just started deleting scoop scripts randomly. God I hate windows.

Changing the Execution Policy triggers a UAC prompt...

Not if you define the scope for execution policy.

At my place of work, IT has taken the scorched earth route and blocked Powershell altogether.

How does it compare to chocolatey ?

I wouldn't put so much trust in this unknown package manager, Chcolatey or Homebrew on Windows would be far better choices.

You don't know what you're talking about.

Ha one important reason why windows won was because you never had to touch the keyboard ever again.

A lot of the people that thing was meant for are now using smartphones and tablets. PCs will be for power users and office workers.

MinGW ships with pacman, I can’t see why one would need this.

Because not everybody needs all the stuff that comes with MinGW? Why install all that stuff if they don't intend to use it, just to get a package manager designed for another platform?

Apart from the fact that the MinGW port of pacman is designed for the MinGW environment only, and therefore doesn't necessarily integrate all that well with the rest of the Windows system, the packages it provides are mainly ports of software from other systems (chiefly Linux and BSD) rather than native Windows software.

Doesn't seem all that relevant.

> Looking for familiar Unix tools?

Is the leading line in the motivational paragraph for this tool, and its example usage is

> scoop install curl

Your comment doesn’t seem all that relevant.

Most Windows devs don't care about MinGW.

Reply on news.ycombinator.com