Hacker News Reader

Comment by tomswartz07 | Hacker News Reader

Perhaps I missed it, but the article doesn't mention this in any way: Windows ABSOLUTELY needs a way to go from 'Fresh Install' to 'Fully upgraded' with minimal interaction.

Linux can do this: run `apt-get dist-upgrade` in Ubuntu, and you get all of the most recent updates.

However, in Windows, you install 3 updates, restart, install 5 updates, restart, install 110 updates, restart....

View on HN

One of the core issues with breaking this pattern is that you're installing a new feature on this reboot, then installing all of its patches on the next reboot.

  So you install 186 updates on a fresh install. This includes .NET 4.5 (or 4.5.1 or 4.5.2), and requires a reboot.

  Now you're done right? You just installed 168 updates, how many updates can there be? On your next reboot, you get another 60 updates, of which 20 are .NET 4.5 updates and security updates. Another is the OS SP1, which needed a reboot before install.

  Now you're done right? You just installed 228 updates, how many updates can there be? On your next reboot, you have another 168 updates, most of which are patches for OS SP1. 

  Now you're done right? You just installed 396 updates, how many updates can there be? On your next reboot, you have another 12 updates, including...

I've seen this chain across more than 6 reboots. I once literally spent an entire day just patching a server.

Yep, this is exactly what I'm talking about.

I don't see why they cant 'look ahead' to say; Oh- this update X depends on update Y, but it's going to be installed, so we could just add it in the queue anyway.

Or rather, slipstream future updates to current downloads. E.g., if you're installing component X that has patches A, B, C, then perhaps you should get customers to download component X that is already in a patched state.

What I always wonder, while twiddling my thumbs during the interminable upgrade-reboot-upgrade cycle, is how many of these these updates will be overwritten by a later update. Am I wasting my time downloading multiple versions of the same file?

These upgrade-shortcuts could introduce a significant increase in testing complexity if you want ensure that you can go from any system version to the current version.

On the other hand, with the current system people upgrade from Windows 1.0 to 8 (e.g. https://www.youtube.com/watch?v=8WP7AkJo3OE).

Unfortunately the update reboot cycle is incredibly reliable, which is why it has persisted. Can't beat "millions of machines have taken these steps".

Thoroughly agree that a more simplified method would be awesome though.

That's the kind of functionality I expect from Windows Deployment Services.

that was decent work. Good job - really accurate.

I'm going thru the process as we speak. :)

I use the powershell script from the packer winrm project.

it's ugly as sin and requires many many reboots (and is slow as fuck) but at least it gets them all installed.

This is also compounded by how long those updates take because they somehow managed to make writing files to disk a CPU-bound process at some point in the mid-2000s. If you have 50MB of updates to install on any other operating system it will take a few seconds but on Windows XP+ it will run at a few hundred KB/s while using 100% of a CPU which can run SHA-512 at well over 1Gbps.

You can skip maybe half of this by installing the Service Pack manually right away. It still sucks, but not quite as much.

No idea why Windows insists on installing individual updates before the Service Pack that includes them all anyways...

An addendum to this is that Windows needs to restart way too often compared to Linux. When I apply my daily yum updates for CentOS, only kernel updates, which are very infrequent, require a reboot.

Every time I do Windows Updates, I need to reboot. This is highly annoying and IMO makes Windows Server YYYY completely unsuitable for servers.

Windows Server needs to restart at MOST once a month ("patch tuesday"). If I look at a 2012 Core server, it hasn't even restarted nearly that often in the past year. In fact 9 times in 12 months.

Plus I can schedule this restart at any hour, on any day, I wish. And do. For example I restart mine on Thursday morning at 1 am.

If that really makes Windows Server "unsuitable for servers" then your downtime requirements must be so high that any single server wouldn't be able to meet them (i.e. even with Linux you'd need two or more servers to provide that level of uptime).

Once a month is way too often. Even if your downtime requirements are "so high", once a month is ridiculous.

For the sake of this example, let's assume you have customers that depend on your server being up, for the product/service they're paying you for.

Tell the customers that you plan on rebooting once a month, and see how they feel about it.

A server is just a name. Just because a host machine needs to go down doesn't mean that the "server" has to go down.

In that scenario I'd never have just a single server, be it Linux, Windows, BSD, or anything else.

Being in a situation where you're completely unable to restart a machine without losing money, is being in a situation where you're living on a knife edge regardless. A single hardware failure and you're in deep trouble.

I've been in exactly that situation (the entire company rested on a single Ubuntu server), it was really unpleasant and management didn't do a thing about it until we started getting sector errors which the RAID controller's error handling was struggling to cope with (after that incident and the downtime, management shelled out for four physical boxes, a SAN, VMWare licenses, and a setup with 15 minute rollbacks and dynamic migrations if a physical server died).

his point remains. being forced to run 2 servers so your server doesn't go down once/month is a bit ridiculous.

No, it isn't untenable, yes you can work around it, but that's a problem that shouldn't need to be worked around.

Agreed. Up to a certain point, a single server is going to be considerably more cost effective than multiple machines, as well as requiring less devOps time. (Especially nowadays when you can easily lease a server with, say, 16 cores and 48GB of RAM.) I can recall having to restart our production server exactly once in the past 12 months for a kernel update. Every month would be an irritation for sure.

Is it normal to update server software like PostgreSQL or Redis on Linux without stopping the running process during the update?

Maybe Linux doesn't need to restart the OS as often, but you're still going to have down time, right?

You can update the software without pulling it down. In order to use the update you need to restart, but there's a large difference between restarting a service and restarting an OS. It isn't really analogous.

It takes 10 seconds to restart Windows these days, so it's almost analagous.

Do you have any benchmarks to support that claim? I have Windows 8 and 10 systems which take minutes to reboot on an SSD which beg to differ.

You can see it when you do a video search for: Windows fast boot. Here are a couple:

https://www.youtube.com/watch?v=p_vUZR2RtRs

https://www.youtube.com/watch?v=xkKoggifsE4

(And you don't need to log in for your background services to start.)

Neither of those support your original claim. The author is focused on one specific feature, which is nice but excludes the entire shutdown process and booting up until the Windows logo appears. Anyone who's run a server knows that just the BIOS initialization is often much longer than 10 seconds. And, of course, many services take a noticeable amount of time to startup so time-to-available is longer than time-to-login-screen.

Worse, the conversation was actually about the time needed to install updates. Now that Windows logs out before installing updates, you have a fairly sizable time delay – tens of minutes after a large update even on an SSD – while the service is unavailable but before the system starts rebooting.

How does Fast Boot work? Well, it's actually not booting at all – it's logging out all active users and hibernating:

http://blogs.msdn.com/b/b8/archive/2011/09/08/delivering-fas...

That's a really nice bit of work in normal usage but it means that you have to do the standard cold boot process when you're getting a system update. That means that a sysadmin would need to review each update to know whether it would trigger a fast or slow boot, assuming that a patch for e.g. SQL Server wouldn't always trigger a cold boot to be conservative.

The alternative is what everyone's been saying, namely that you have to assume that a reboot is slow and have n > 1 servers if uptime matters.

I understand all of that and I agree with you.

However, if people are going to make irrelevant points about "not having downtime with Linux" then I am going to fuck with them.

That is BS. When you install updates it restarts to install them. The time is proportional to the number of updates.

Popular (gaming) services have weekly downtimes to patch servers, and while users are annoyed, they seem to get over it and keep playing just fine.

It's completely insane to have multi-hour downtime every week the way Blizzard does. Between that and their regionally segregated Battle.net (there's no good excuse that you can't play Hearthstone with anyone anywhere), they're basically stuck on server architectures from the 90s.

Windows Server needs to restart at MOST once a month ("patch tuesday").

BUT if Windows patches came out as often as CentOS patches do, I'd been rebooting every other day.

This is the approach taken by LY-Linux ("patch leap year"), although better implemented IMO. In fact, I typically only have to restart my servers once every 4 years (at most).

Look it up, it's awesome. (Please don't.)

This might be the case if you run servers on single machines rather than in clusters. And if you are running servers on single instances, aren't you already giving yourself a SPOF? If you need a specific server to be running at all times it seems like you have a problem.

What I really don't understand is why Windows Update takes so long to apply each update. I wouldn't mind 200 updates and a couple of reboots if it only took 10 minutes to go through them.

In most Linux distributions, an update simply unzips a tarball somewhere and runs some bash scripts. Unless it's a very large package, each update usually only takes a fraction of a second on SSD, and maybe few seconds on HDD.

Windows, on the other hand, often takes minutes installing each update. Going from freshly-installed to fully-updated can easily take several hours, even on an i7 with SSD and a very fast connection (so the download speed is not a factor). It's even worse on an older computer; I've seen a smallish .NET update taking over 10 minutes to finish, and there were several such updates. What the hell is it doing behind the scenes that takes so much time?

>restart

This isn't going to happen. MS is tied deeply into restarting due to how it loads libraries and other issues. Unless they write a new OS from scratch and give up on legacy compatibility, well, don't hold your breath.

I don't even mind the multiple restarts, the part that bothers me is that I have to babysit Windows Update as soon as my machine finishes restarting, tell it to get updates, tell it to install now, and then have it restart again. If Windows Update could collect all the update packages, and I could leave it alone for 20 minutes and come back to a fully updated system, that would be fantastic.

Yeah, this seems like it should be pretty straightforward (I'm sure it is harder than it seems, but still). Set a flag that persists through multiple reboots that says "after Windows loads, if there are updates, apply them and reboot, repeat, otherwise unset the flag".

Fun fact: I helped tweak up a Powershell script that does this exact thing: https://github.com/tomswartz07/packer-windows/blob/master/sc...

I owe you a reasonable beverage of your choosing. Please redeem anytime you're in melbourne.au :)

If only it had a place to register a setting or store a file containing information like "update until you reach the latest version". It'd also need to be able to get through all the updates without crashing even once (otherwise we'd need to check the disks and repair start-up files, and that can't be automated), so maybe this is just impossible.

Heck, I'd be okay with booting into a Windows PE drive, if it meant that after 1 reboot I'd have a fully upgraded OS.

If this day has taught anything: MS is moving to Linux

Even in Linux you've got to restart once, if you update the kernel or somesuch. The problem with Windows is the restarts before the final one, where you need to finish upgrading to version N+1 before the updates taking you to N+2 will even download.

If the updater software could just plan out a patch-graph ("once I do X, then Y will be available, so I'll do Y, and that means Z will be available"), then apply all the updates it had recursively discovered and only restart at the very end, everyone would be happy.

Even though the resulting mechanism wouldn't be very similar at all to Linux's idempotent "just install the latest version of the package, you don't have to take the steps in-between" system, it's the UX that matters.

Actually the 4.0 release has added support for no-reboot kernel patching, IIRC.

A lot of the time, restarts are required to remove old versions of files that are still in use. AFAIK, there is no way to remove files that are 'in-use' (and not locked) without having to make an entry in the registry to schedule a deletion on the next reboot.

I always wondered about that.

Shouldn't a real OS and filesystem be capable of removing something in use?

IRIX always could. So can Linux a whole lot of the time, if not all the time, as can BSD and Mac OS.

Here's something:

Once I upgraded a pile of IRIX boxes to a new version of some CAD software. A few of them had the older version running...

Didn't matter!

The files got cached, and those versions continued running the next day, until shut down. Next launch, the new version popped up.

One could just run a program, then delete the program right out from under the running process, and that process didn't know. IRIX just cached it all, and when the process was done, cleaned up the cache.

> Shouldn't a real OS and filesystem be capable of removing something in use?

It's a choice. Windows specifically uses DLLs as non-dirty swap, i.e. if the memory manager want's to free up pages it may free memory pages used for mapping a least recently used DLL because it will not require it to write any "dirty" memory. A DLL page can never be dirty. Windows knows that it really easy to read in the image again (from the DLL) - so it can just kill the page and reuse it immediately. But then it has to know that it can get the exact same memory page back from the DLL. If the DLL has been deleted or changed that would lead to some strange errors.

This is no different from the situation in Linux, BSD etc, where executables and libraries are simply mapped into the process with mmap() - they're backed by the files and the system is free to discard those clean memory pages as it sees fit.

When you install(1) a new version of a file, the old one is deleted and the new one written in its place. The file handle in existing processes continues to point to the original, and the filesystem will wait until everything's closed it before actually discarding the file data.

Thanks for clarifying. That boils down to linux perfectly hiding this problem from the user, while windows painfully makes the user restart every time a file is in-use and needs to be removed.

Thanks.

Solving this is one of Matt Wrock's goals with Boxstarter. I've been experimenting with it recently and found it quite nice.

The way around this is to update your deployment images from time to time.

It's one reason I liked service packs, new service pack comes out, generate a new install image.

I'm guessing this is the plan with the new system of "builds" we've seen in the Technical Preview. Just install the month's build and then the few updates it needs after that. Fingers crossed, anyway.

Right, and that takes care of the 'big ones', but you still have tons of other minor updates that are released every Patch Tuesday.

I currently have Jenkins build an OS image every two weeks, just so our Deployment Server has an up to date image.

> I currently have Jenkins build an OS image every two weeks, just so our Deployment Server has an up to date image.

Ya, lots of tools have made this easier recently, but thinking about it, doesn't WSUS handle this?

I haven't been a Windows admin for a while, but I thought with WSUS it was install, join and forget while it did it's thing. Maybe it's been too long ...

WSUS just caches updates locally, so you can download once and have the WSUS server distribute copies to your forest.

It's a GPO that points your domains to the WSUS server. And I've only used Octopus to automate WU installs.

> WSUS just caches updates locally

New updates do supersede others though, if they're marked to do so from Microsoft. That should cover at least some of the .Net point release updates and the like.

> released every Patch Tuesday.

It is the second Tuesday of the month, not every Tuesday. They've only broken that pattern a handful of times in the years that they have been doing it when an "emergency" patch was required.

>> released every Patch Tuesday.

>It is the second Tuesday of the month

that's exactly what "every patch tuesday" means (notice the "patch" adjective).

Yeah I think they need to address this but it's going to require a pretty big, fundamental change so it might take a while. It's certainly needed.

Remember the Windows 95 days?

    Your mouse pointer has moved. 
    Please restart Windows for this change to take effect.

I got "You must restart your computer to apply these changes" in Windows 8.1 just by moving my mouse to another USB port a couple of weeks ago. I guess some other non-critical update was queued up.

That's exactly what they're trying to do with the new app model.

I think he's referring to system updates, not just installed software.

My usual workflow for setting up a Windows machine goes like this:

  1. Install Windows
  2. Download updates
  3. Reboot
  4. Download updates
  5. Reboot
  ...
  28. Download updates
  29. Reboot
  30. Download updates
  31. Reboot
  32. Install software

It's not great. OS X has been doing cumulative updates from 10.x.0 to 10.x.current for what, a decade now?

MS is rebuilding a lot of essential utilities in their "modern" app framework (including the start menu), but as far as I know, the lower level system updates still get distributed the same way as before, and frequently have long chains of prerequisites.

Reply on news.ycombinator.com