Hacker News Reader

Comment by allegory | Hacker News Reader

An anecdote.

When the company I was working for at the time's AC failed in the machine room and melted three racks of kit, all 7 of our FreeBSD machines at the time were back online from scratch on hot spare kit in under two hours.

Our Windows guys were still working on restoring the master Active Directory pair and DFS cluster TWO DAYS LATER even with a DR strategy in place.

Quality engineering, documentation and automation ability shits on everything else IMHO and that's where FreeBSD lands every time. Performance isn't a major thing for me.

Linux on the other hand is heading in the direction of windows. I'll probably get flamed by the systemd proponents here but it took me three hours to work out how to debug this command:

   timedatectl set-ntp yes

Which returned:

   Failed to issue message call

View on HN

Linux on the other hand is heading in the direction of windows.

The reasons for this, in part, are the different mentalities in the Linux and BSD communities. The Linux community used to be quite adamant about how "Linux is not Windows" and sticking to their paradigms, but more recently they've been absolutely obsessed with pipe dreams of conquering the desktop, and have been completely undermining their technical workflow in the process. Instead of "Linux is not Windows", it seems companies and communities alike have decided to pander to a crowd that has no respect for computing and Unix culture, who are used to closed systems that hold your hand, and quality is suffering in the process.

The BSD people on the other hand, have stayed true to their heritage, and show no signs of compromise. BSD people also tend to be very familiar with Linux as well, whereas the opposite is much less common.

Reminds me of the old saying that <em>"Linux is for people who hate Windows, and *BSD is for those who love Unix."</em>

> Failed to issue message call

Reminds me of some mad attempts to debug some problem in Unity/GNOME... I gave up pretty fast after stranding on orphaned freedesktop.org pages.. I distinctly remember ranting on reddit about the problems and I've got a few .odt presentations from GNOME developers that not so really explained some concepts.. it's a different world. I can't judge if it's better this way for the Desktop but I've found it impossible to debug without sacrificing enormous time and energy about things I'd never wanted to deal with.

Stark contrast has been xCAT by IBM.. it's used to manage clusters of 10000 machines and it's only small readable Perl and Shellscripts.. totally awkward at the beginning but once you get a grasp it's a flexible power tool.

A friend of mine calls it "desktrap loonix".

The sad part is it's still better than windows.

Despite punching windows in the face a while back, I don't agree that GNOME is anywhere near windows. Not even going back to windows 2000.

>GNOME

Oh dear Eris, don't get me wrong, I don't claim GNOME in particular is a software fit for any purpose.

My point was more like that the base system is still really good and systemd even with its failings is better than dealing with Windows. As for the desktop, there's enough choice that you should be able to find something that's maximally convenient while minimally retarded, with a small amount of customization. That something will most surely not be GNOME.

That's kind of what I mean - it's gotten worse in some ways, but you can still pick your poison and it's not as bad as having to deal with whatever retardation Microsoft come up with.

>'Our Windows guys were still working on restoring the master Active Directory pair and DFS cluster TWO DAYS LATER even with a DR strategy in place.'

I'd be overjoyed if I never had to admin Windows again, but it's a bit unfair to frame that as an OS comparison.

The most basic of direct from TechNet 'best practices' would reduce the impact of a failure of that sort to minutes.

That said, that sort of failure not at all uncommon. Further, I expect such disasters would happen far less often if an existing, poorly implemented Microsoft environment could be remediated into something better as directly as anything unix-like.

I think it's completely fair. Windows is completely non-deterministic due to the insane amount of coupling inside it and the sheer snowball of crap that has stuck to it over the years.

In this case, they had tested DR strategy etc in place. Unfortunately when it came to doing it for real (onto the same backup hardware the DR plan was tested against), TSHTF and Windows threw an obscure COM error loading the AD catalog. That required contacting MS 1st line who didn't know what it was so had to escalate it to the AD guys. One obscure registry key change and an ACL change (related to ESE) and it was back.

All best practices followed, yet complexity, poor design and secret knowledge crept in and shot the whole process.

Stuff like that scares the shit out of me having been on the end of it way too many times now.

Many a night have I spent up at 2AM trying to work out why the hell something odd has gone on inside Windows and taken something out in production. Not once in 20 years has a proper Unix (Solaris/FreeBSD) box woken me up or shafted me for hours.

You're just not trying hard enough. There was a Xen hypervisor bug for years that could, when the stars aligned and the moon was full, jump the clock ahead 50 minutes. Nobody knew what was causing it until during a bug session the right guy was looking at the right bit of code and noticed an obscure problem in some in-line assembly.

Every system has bugs.

This isn't about bugs, it's about system transparency.

Every system has bugs, but as the other reply said, quality engineering and most importantly transparency determine the impact.

Never been a fan of Xen. It doesn't strike me as quality software. Then again no virtualization solution has to me, yet. I'd rather just deploy all the services to the base machine and use MAC to isolate them.

The scenario described would indicate that there were delta's between the testing environment (where the DR strategy was tested), and the production environment. It was probably related to OS updates/changes being applied over time, resulting in a configuration that changes. I've always found it good practice to build and maintain a staging environment that mimics the production environment in all aspects. When configuration changes (aka security patch) are needed, they are tested and validated on the staging environment before they are deployed on the production environment. This gives you an opportunity to test and validate the results in a non-production (aka no rules, no SLA's) environment. Part of this involves validating that procedures such as DR will continue to work as expected on the new configuration, before it gets rolled out to production. From my experience, this methodology minimizes scenarios of unexpected behavior in the production environment (aka downtime). I would recommend this methodology (or anything similar) regardless of the OS/distribution you're using.

That's exactly what was done. The guys we had were shit hot ex Microsoft admins.

The DR environment was the staging environment for the patches. Periodically the production kit would be block copied back to the DR environment and sysprepped.

Every step to prevent differences between the two clustered environments were taken.

"Our Windows guys were still working on restoring the master Active Directory pair "

I was a Solaris admin at a previous employer. We ran annual DR exercises, and saw similar issues with the Windows kit. Even with months to prepare for the exercise, it took the entire exercise, sometimes longer, to bring 'Windows stuff' back to life.

What we (the unix team) did was to write into _our_ DR plan a few paragraphs on how to be up and running without DNS, or Active Directory.

I see DHCP, DNS and AD (in that order), at least in a windows environment as step one of anything.

Solaris was nice and easy to get back apart from one machine I dealt with: the starfire E10k. The SSP failed on one that we had and the entire system uses crypto key exchange so only the original SSP can bring the system up. Fortunately we had an E15k being installed at the time so it was an emergency migration job.

I miss big iron. I had an old maxed out 1000E and a disk tray full of 4.3Gb SCSI disks at home for a bit (until I got the electricity bill!)

"I see DHCP, DNS and AD (in that order), at least in a windows environment as step one of anything."

It gets _messy_ but I think it's worthwhile to write out in biz continuity plan how to function without DHCP, DNS, and AD.

Take my case: we sometimes had the tier 1 and 2 services (the unix-based ones at least) up _days_ before DNS and AD were available. In the real world the business can't wait for DNS and AD to have access to their tier one and two apps: customers are waiting for product.

Note that a lot of work went into getting the Solaris stuff to this point: in particular using ZFS and zones made the process or restoration a breeze. By the time of our fourth annual DR exercise, getting stuff up and running was essentially waiting for it to restore from tape, adjust for things like 'DNS y/n' and done.

  Failed to issue message call

What was that, a dbus configuration problem? Are you quoting it correctly because if you really ran into a failure message that's not googleable that's scary.

No idea. I never solved the problem. It just worked again randomly after a period of time (a couple of hours).

That was enough to scare me away from CentOS 7 and systemd to be honest.

On the whole I don't think systemd is that much more complicated to understand than a collection of bash scripts. It's a domain-specific declarative language for dependency-based execution. At least, once you understand it, it has less gotchas than bash scripting.

It's the rest of the utilities that they're trying to couple to it that are bullshit. Starting with the journal - asking for the last few lines of the journal can take half a minute for some reason. Fortunately you can ignore those parts and just use the smart init system, i.e. you can ignore the cron replacement and run plain cron, you can ignore the nptd replacement and run plain ntpd, etc.

sounds simmilar to

https://wiki.archlinux.org/index.php/Systemd_FAQ#.22Failed_t...

Reply on news.ycombinator.com