Comment by pifflesnort | Hacker News Reader

Comment by pifflesnort | Hacker News Reader

Parent Full thread

> High-overhead based on what, the compressed packages? I'm guessing you're a fan of the OSX .dmg approach... which also supports file compression.

DMG is an archive distribution format. The packaging pseudo-filesystem is going to be sitting between the user/kernel, interdicted through a file, and then through a file system, for all access to all installed software.

Think FUSE.

> The 'unstable dependency graph' is anything but, instead it's quite straightforward to manage. To explain, think of PATH variables in most modern operating systems.

First of all, you just called PATH straight-forward. Try explaining PATH to the average desktop or mobile OS user, and then try explaining why this is better than the App Store or drag-installs.

Second of all, the problem isn't the PATH thing. The problem is a huge dependency graph of non-ABI stable libraries and packages that are comprised of a ton of files splatted over the disk.

If you get rid of the non-ABI-stable libraries, and don't splat your files everywhere, you don't need some crazy pseudofs-based package manager to manage your software.

Problem solved. No package management (in the Linux sense) required. Isn't it a good thing to remove unneeded, reducible complexity?

>The packaging pseudo-filesystem is going to be sitting between the user/kernel, interdicted through a file, and then through a file system, for all access to all installed software.

A pseudo-filesystem is not high-overhead, unless you believe symlinks and pointers are high-overhead.

>First of all, you just called PATH straight-forward. Try explaining PATH to the average desktop or mobile OS user, and then try explaining why this is better than the App Store or drag-installs.

Here's the thing: an 'average user' doesn't care about 'a ton of files splatted over the disk' either. How many Windows users bother even looking in the Windows directory or Program Files directory? The 'a ton of files splatted over the disk' issue is only an issue for power users, who like to feel in complete control of their system.

As for App Stores, they're just like a traditional package manager but with a simpler UI and the option to buy software before downloading it. If you're familiar with Ubuntu, compare Synaptic with the Ubuntu Software Centre? See a difference beyond the cosmetic layer? Now compare the Ubuntu Software Centre with the Apple App Store, can you see a difference beyond the cosmetic layer?

I've got a strong feeling of déjà-vu discussing this, are you a Haiku-OS.org forum member?

> A pseudo-filesystem is not high-overhead, unless you believe symlinks and pointers are high-overhead.

Retrieving data from multiple on-disk compressed package archives and presenting them as a union pseudofs is a heck of a lot more overhead than hitting the file system directly.

> As for App Stores, they're just like a traditional package manager but with a simpler UI and the option to buy software before downloading it.

Actually, they're nothing like a traditional UNIX package manager, because they don't handle dependencies. They're application bundles (ala Mac OS X) with a store front-end.

The OS vendors provide stable API/ABIs and comprehensive base system libraries, which means they don't need a UNIX package manager at all.

It's really depressing to see Haiku come so close to completion, only to have project direction taken over at the very end by Linux developers (literally, that's their day job), who are bringing Linux ideals to a system that was originally built as a successor to the original Mac OS.

>Retrieving data from multiple on-disk compressed package archives and presenting them as a union pseudofs is a heck of a lot more overhead than hitting the file system directly.

It's really not. Imagine a file system where all the "real" file locations are hidden, and the visible file structure just symlinks to these "real" locations. Practically no overhead at all. There will be a minor performance hit with uncompressing files before starting a program, but that's about all.

> Actually, they're nothing like a traditional UNIX package manager, because they don't handle dependencies. They're application bundles (ala Mac OS X) with a store front-end.

My point was in terms of usability. The average user does not care about dependencies, nor do they need to care about them, even when using a package manager.

> The OS vendors provide stable API/ABIs and comprehensive base system libraries, which means they don't need a UNIX package manager at all.

That only works when all applications are written using those standard libraries, and that doesn't always happen, even on OSX. Why punish software developers for wanting to use libraries that better suit their needs?

>a system that was originally built as a successor to the original Mac OS.

The system was not built as a successor to the original Mac OS. The BeOS team were led by ex-Mac people, granted, but when it came to OS design they took ideas from many places, not just MacOS. As I've said before, the idea behind BeOS was to look to the future, not to be tied to the past.

> It's really not. Imagine a file system where all the "real" file locations are hidden, and the visible file structure just symlinks to these "real" locations. Practically no overhead at all. There will be a minor performance hit with uncompressing files before starting a program, but that's about all.

Are we talking about the same thing? It's a:

- pseudofs

- in userspace (IIRC)

- which parses multiple archive files

- and vends their contents as unionfs

If this bought you anything at all useful, then maybe it would be worth all the complexity involved ... but it doesn't.

> That only works when all applications are written using those standard libraries, and that doesn't always happen, even on OSX. Why punish software developers for wanting to use libraries that better suit their needs?

It's not punishing developers. I've shipped software for Mac OS, Mac OS X and iOS for 15+ years, and believe me, being able to ship custom versions of libraries without having to deal with a package manager, library incompatibilities across applications, et, al, is an advantage.

> The system was not built as a successor to the original Mac OS. The BeOS team were led by ex-Mac people, granted ...

Led by ex-Mac people, including the former CEO of Apple. Targeted at the BeBox (PPC) and PPC Macs. Shopped to Apple for purchase. It was intended to be the successor to MacOS that Copland wasn't.

> As I've said before, the idea behind BeOS was to look to the future, not to be tied to the past.

Then maybe the new Haiku leadership should stop blindly copying what they're familiar with from Linux.

> Are we talking about the same thing?

Admittedly I stopped following the development of Haiku and other alt operating systems over a year ago, so I couldn't say for sure whether the file system is in userspace. Doesn't seem like a good idea in the long run, hopefully it's just a temporary measure.

> being able to ship custom versions of libraries without having to deal with a package manager, library incompatibilities across applications, et, al, is an advantage.

But it's not always an advantage. Library duplication has a cost in terms of storage, plus the mechanism for program upgrades is clunkier than package managers allow (essentially either you have to download the full version of each program update or programs are bundled with their own package management mechanism, duplicating effort).

> Led by ex-Mac people, including the former CEO of Apple. Targeted at the BeBox (PPC) and PPC Macs. Shopped to Apple for purchase. It was intended to be the successor to MacOS that Copland wasn't.

Jean-Louis Gassée was a senior member at Apple, but never the CEO. BeOS was targetted at BeBox (their own machine) at first, then when that platform didn't take off it was targetted at PPC Macs, then x86 PCs. The purchase talks happened because the OS was struggling to gain market share.

> Then maybe the new Haiku leadership should stop blindly copying what they're familiar with from Linux.

As I said before, the design is quite different from Linux package managers, so Haiku are doing things their way.

> It's really depressing to see Haiku come so close to completion, only to have project direction taken over at the very end by Linux developers

If you find it so depressing, instead of criticism, offer help. You are not helping. You are only offering your opinion.

> If you find it so depressing, instead of criticism, offer help. You are not helping. You are only offering your opinion.

So you're saying that reasoned constructive criticism is not helping?

You can't offer 'help' to people that don't want it. You'll waste your time and theirs.

All your criticism can be summed up by "it doesn't look like a Mac, therefore it's the wrong approach".

Haiku is not a Mac. It's not Linux either, but, from a computer management perspective, tools like ports and yum solve many problems Macs also have. It's no coincidence each and every developer in my office who use a Mac also has either ports or brew installed. Dragging .app folders into icons does not solve every software management issue.

If you need to keep your machine in a known and repeatable state, dragging icons will not take you far.

What you're really saying is "I disagree with you, but I'm going to take a rhetorical roundabout in which I attempt to disqualify your argument by calling it 'not helpful'"

> All your criticism can be summed up by "it doesn't look like a Mac, therefore it's the wrong approach".

No, my criticism can be summed up like so: If you create unnecessary, reducible complexity, then add even more complexity on top of that to manage it, you're doing it wrong.

> It's no coincidence each and every developer in my office who use a Mac also has either ports or brew installed.

You're right. It's not.

First of all, note what you said: every developer.

Second of all, note what they're installing: UNIX software. UNIX software is a distribution mess, and it requires a UNIX package manager. If you don't create the mess in the first place, you don't need a package manager to manage it.

Let third-party packaging systems deal with the UNIX packaging mess, and let engineers who need that software deal with the package managers needed to manage it.

Net result: Your users (which includes engineers when they don't have their software development hat on) don't have to deal with the package management disaster unless they need to deal with UNIX software.

QED.

> If you create unnecessary, reducible complexity, then add even more complexity on top of that to manage it, you're doing it wrong.

I believe the fact nobody has been able to reduce the "reducible" complexity you point out for the past 40 years is telling. And no, the NeXT .app folder thing does not reduce it - it replaces the shared library "problem" with other problems (security and just plain malfunction) through obsolete duplicate libraries that may or may not get updated by someone other than the packager.

> First of all, note what you said: every developer.

I'm sorry, but I don't think BeOS was ever aimed at someone other than developers. It would be quite surprising if Haiku were aimed at a different public.

Again, I fail to see your problem with package management. My mother uses a Linux machine and the only contact with package managers she has is the auto-update feature - and her machine is working for 4 straight years without a full reinstall, which is more than most other OSs can say about machines that are not actively managed by someone knowledgeable. I manage dozens of Linux machines and I never experienced the package management disasters you describe.

Maybe you should get some first hand experience and, dare I say, read the manual. If you experience a significantly larger number of disasters than your peers, the problem is most likely not the tooling.

So you're saying that Mac OS, Mac OS X, Android, and iOS are all broken? And Linux's software distribution mechanism is working for end users?

> I'm sorry, but I don't think BeOS was ever aimed at someone other than developers. It would be quite surprising if Haiku were aimed at a different public.

The involvement of former Apple employees, targeting Macs, the attempted sale to Apple, the single-user OS, the UI riffing off MacOS, the focus on media, the simple UI -- those we're all flukes? It was actually meant to be a difficult to use Linux clone that focused on shipping packaged UNIX software, despite its particularly weak posix compatibility and complete non-standard directory layout?

I was hacking on BeOS since DR2. I have friends who worked there. It was contemporary with my career, and I was following Apple's and Be's parallel efforts closely. It was, as far as my knowledge and experience goes, a user-focused operating system ala MacOS.

> Maybe you should get some first hand experience and, dare I say, read the manual. If you experience a significantly larger number of disasters than your peers, the problem is most likely not the tooling.

Many years ago, I wrote one one of the package managers in wide use today. Then I spent years shipping software for multiple platforms. I think I understand the trade-offs, and implementing package management only made the underlying failure of the Linux package manager model more clear.

What kind of ridiculous system requires an army of volunteers to maintain a graph of dependencies in lockstep, making it nearly impossible for 3rd party developers to directly release simple binaries that work across their user's systems. Instead we have to add Chrome package repos to our Ubuntu installations that break across major releases.

It's stupid. There are better ways, which are evidenced by the fact that they're working, today, and have been working for some 20 odd years, and are being further defined in the AppStore model.

Maybe you should stop being so selfish by trying to force your inexperience and UNIX sysadmin use case on everyone.

> So you're saying that Mac OS, Mac OS X, Android, and iOS are all broken?

No. I said they solve some problems and don't solve others.

> And Linux's software distribution mechanism is working for end users?

I am yet to meet a non-technical Linux user who has experienced what could be described as "package management disaster". Software management is mostly automated for their use cases.

> Maybe you should stop being so selfish by trying to force your inexperience and UNIX sysadmin use case on everyone.

I solve my problems, you solve yours. Deal with it.

> No. I said they solve some problems and don't solve others.

Yes, they don't solve the problem of distributing messy UNIX software with complex and unstable dependency graphs, written for systems with anemic base libraries, that install a litter of files all over your system.

That's a good thing. In fact, I'd say the world would be a better place if we'd never implemented package managers and instead UNIXes were finally forced to clean up their mess. The real world doesn't work that way, so instead the best we can do is create new systems that don't inherit the mistakes of the old.

> I am yet to meet a non-technical Linux user who has experienced what could be described as "package management disaster". Software management is mostly automated for their use cases.

Use cases like, say, trying a new piece of software that hasn't been packaged?

Getting updates directly from the vendor?

Being able to have their vendor-supplied applications continue to work across major OS releases?

Or how about from the vendor perspective, where we have to deal with mistake-prone volunteers packaging up our software, inconsistent and incompatible libraries and an ever-shifting dependency graph. The only reason anything ever works at all is because there are hundreds/thousands of people whose sole contribution is to keep the entire dependency graph afloat. Just how many human hours do you think are spent on software packaging overhead?

> I solve my problems, you solve yours. Deal with it.

Thanks for elucidating exactly why Linux developed shouldn't be anywhere near the design process for consumer software like an OS.

You're projecting your view of Linux package management onto Haiku, when the situation with Haiku is going to be pretty different.

For starters, let's start with libraries. To use a concrete example, let's say the Cairo vector graphics library. What happens when a program requires Cairo 1.12.14? The library package gets downloaded. What happens when a second program requires Cairo 1.12.13 (i.e. the previous version)? The library gets stored alongside the newer version, so both are available. What happens when a program has forked Cairo, and requires that version? Hopefully the answer is clear.

So in this example we have three versions of Cairo. Seems inefficient right? In some ways it is, but efficiencies can emerge when programs do target the same version of Cairo on our system, as the library will already be ready for new programs to use.

It's also important to realise in what ways having these three libraries is inefficient. The only real issue here is storage, as we'd probably be looking at three times the storage space for the Cairo functionality. However, as inefficient as this is, it's still more efficient (from a storage point of view) than each application bundling its own libraries, as with installing the libraries separately there's at least a chance of the libraries being shared.

There is an alternative, which is to specify library versions that developers should use and bundle these with the system. This has advantages and disadvantages. The advantage is that the functionality available to you is easier to scope out. The disadvantages are that recommended libraries get updated less frequently, so if you wish to use a new version you're either forced to wait or bundle your own library, which brings you back to the inefficiencies you had before.

Taking time to reflect, I see now that your issue isn't really with package managers per se, but rather that you dislike the 'rolling release' approach of many open-source projects. It's the shifting target that annoys you, how this gets managed is secondary. Would you say that's a fair appraisal?

I can report the opposite - I've had to assist a fedora user when yum/rpm left them in a state that prevented a ton of new packages from being installed due to an interaction between the official fedora repos and rpm fusion, the repo everyone tells you to add to fedora to get video drivers and codecs etc. It was actually quite complicated to resolve as packages relying on specific libraries that were being held back had to be identified, removed, external repo removed, libs updated, packages readded, repo readded.

All it took to get him in that state was copying a pasting a few lines thats suggested on every fedora forum, and a few unwise version dependencies in some official packages, and an external repo that falls behind the official one a fair amount.

That is not a description of a well working system. And casuals frequently end up with tons of held back packages with apt because of not using dist-upgrade.

I'm a heavy linux user and happily deploy it all the time, but to suggest that current package management needs no improvement is insanity. It's a highly advanced system of bandaids for decades worth of iterative engineering on a core design that predated shared libraries, CVE's and when update cycles were well longer than a year.

A clean slate design would be an infinite improvement - it's just a crazy difficult problem without it basically being a new platform and requiring modifications to the 3rd party code that you're installing.

> I can report the opposite

For every problem, how many times did package management work flawlessly?

> All it took to get him in that state was copying a pasting a few lines thats suggested on every fedora forum

So, the user tried to somehow bypass the official distro package management. If you open up your microwave oven and electrocute yourself, is that because of a basic flaw of microwave ovens?

> For every problem, how many times did package management work flawlessly?

None. It built a massive teetering edifice of fragile interdependencies that incurs staggering human cost in maintenance, and has almost singlehandedly stood in the way of progress by third-parties that would benefit from working outside that system -- from commercial software vendors to OSS vendors that would prefer to be able to release versions on their own schedule.

If it wasn't for package managers making it possible to maintain fragile dependencies in lockstep, Linux distributions would have been forced to confront simple issues like ABI and API compatibility, third-party distribution, and other basic facets of producing a user-friendly and vendor-friendly OS ... we may have even had a "year of Linux on the desktop" by now.

Instead, we keep throwing humans at the problem of keeping the teetering pile of dependencies operational, largely because of people like you that cargo cult the ideas out of some strange combination of stockholm syndrome and a lack of vision for how things could be better.

Why was anyone was surprised that Google threw away the entire Linux user-space when developing Android?

> So, the user tried to somehow bypass the official distro package management.

... because that's the only way to achieve perfectly reasonable goals as a user.

If I want to install Chrome, guess what it does? It adds an external repository that broke in strange ways when I updated Ubuntu.

> If you open up your microwave oven and electrocute yourself, is that because of a basic flaw of microwave ovens?

Your microwave doesn't refuse to operate unless you only buy food that has been prepackaged by your microwave vendor, that can only be mixed with other food also repackaged by that vendor, and only if all the food has been pre-certified to work together in that exact combination.

For every problem, how many times did package management work flawlessly?

Fedora systems often enter states where manual intervention with the package system is necessary to fix and issue preventing updates, they're usually just much simpler. And upgrading from major release to major release very commonly results in orphans or zombies. At least until the most recent release, the official instructions for resolving live upgrade issues involved 2-3 reboots and manual management of the package database in single user mode.

So, the user tried to somehow bypass the official distro package management. If you open up your microwave oven and electrocute yourself, is that because of a basic flaw of microwave ovens?

No, it's the expected behavior of many fedora users and common among developers. Fedora has a zero tolerance policy on binary blobs and anything potentially ip encumbered. This is their arms length provider of such, and the official packagers will direct you there if you look to get your software included in the main repos but they have concerns about patent coverage.

They rely on the use of that repo so much that they actually first shipped gnome 3 with the open source nvidia driver they shipped with blacklisted because of how bad it was, and instructed nvidia users to install the proprietary drivers to get a working system.

Completely agree. It seems people who have no problem with Linux package management are bringing over the same problems to what should be a non-programmer-friendly system.

> a non-programmer-friendly system.

'Non-programmer-friendly' like how MS-DOS was non-programmer-friendly with its memory models and lack of multitasking, or like how AS/400 was non-programmer-friendly with you choice of RPG or COBOL, or do you actually think that making something friendly for non-programmers automatically makes it non-friendly for programmers?

Sorry, I meant user-friendly, where the user is not a programmer. Actually, making a package system friendly for users will make it friendly for a lot of programmers too - who are also users (and don't feel like wrestling with package systems too).

Reply on news.ycombinator.com