My sixth day with Haiku: Under the hood of resources, icons, and packages
medium.com
medium.com
Also disheartening (though not at all surprising) is the developer response to this: "We optimized for the most usual usecases here. If there is actually a demand for this beyond one user, we can implement". That's my problem with "magic", it only appears to work the way you expect until your usecase falls outside of the expected, then the illusion is destroyed.
The HVIF format sounds like a really good idea. If I ever get around to making my own desktop environment, which is looking more and more likely to be the only way I can escape the dystopia laid out for personal computing's future, I might use it.
You keep saying this. I do not think it means what you think it means.
Package managers add a certain kind of complexity to parts of the system that, without it, would indeed be "simpler." But it is an extremely powerful lever that allows removing far, far more complexity than it adds, especially when it comes to porting software. We were very thoughtful about this tradeoff and tried very hard to get it right.
Let me ask you this, then: You have repeatedly made this claim. How much have you actually used Haiku, both before and then after the package manager? Because the entire point of the package manager was to pull the complexity away from the users and to us (the developers and maintainers), where we can manage it and automate it away, which is what we have done. If you as a user are seeing some issue, we would greatly like to know about it so we can solve it.
> Sure, he recommends some behind-the-scenes magic that can make it a little better, but that's never going to be as good as just being simple in the first place.
If it were truly about simplicity, don't you think Apple would have done it already? Or some other company -- that is, being simpler means spending less time and money; and any company that could save money and turn more or the same profit without losing customers would do it in a heartbeat.
> That's my problem with "magic", it only appears to work the way you expect until your usecase falls outside of the expected, then the illusion is destroyed.
You are using ""magic"" in scare-quotes. Why? You do know this is all code, written by us, and which can be modified by us, or someone else, right? There is not some file called "package_manager.cpp" which was written by someone in a basement 30 years ago that is incomprehensible to human eyes. These are well-thought-out systems that are designed to fit in with the rest of the system in a coherent way.
It is also the case that "we optimized for the most usual usecases" is how 99% of software is written. So why are you singling out package managers, or at least our package manager, as a problem here?
I also note it sounds like you are going a lot from theory (i.e. "I read the package manager works this way") and not practice ("I used the package manager and found X".) Is that right?
At least probono seems more than happy with Haiku and its model:
[15:52:55] <nepugia_q> if you are serious about wanting to make a better linux desktop linux distro sign me up :P (have been trying to bootstrap a distro for some time now, with much less gnu tools than we have now (and putting / on a tmpfs and so))
[16:02:58] <probono> nepugia_q now that i have found Haiku i think we should invest here
I disagree, so do others[0], but there's no need to rehash this argument[1].
> If it were truly about simplicity, don't you think Apple would have done it already?
Apple did. Twice.
> You are using ""magic"" in scare-quotes. Why?
Because I have come to distrust software that pretends work one way when it really is doing something different. In Haiku, this means presenting files to the user as though there are in one place when they're really in another, with the results that trying to manipulate them leads to errors that require you to peel back the abstraction, the "magic", to understand.
> it is also the case that "we optimized for the most usual usecases" is how 99% of software is written.
The most usual usecases for who? Haiku devs I imagine. And you know what? That's perfectly fair since you're the ones doing all the work and I'm just here griping. But that doesn't change the fact that I don't agree with your choices.
> So why are you singling out package managers
Because I'd be hard-pressed to think of something that has brought more complexity for less benefit than package managers. I remember a time when it was common and simple to carry applications around on floppy disks and just copy them to a new computer and they'd work. Package managers, even Haiku's, don't allow this because they "optimize for the 99% usecase", or rather, the 99% of the developers' use cases.
> At least probono seems more than happy with Haiku and its model:
Good for him, but I'm still not convinced.
But it's ok. Really. Calm down. I'm just an old curmudgeonly bastard anyway, you don't need to win me over.
[0] https://www.reddit.com/r/haikuOS/comments/7659fa/where_is_ha...
The comments agreeing with your opinion there have very few upvotes; mine have much more. Not that this has any bearing on the technical merits or demerits of the actual positions; the best solution is not necessarily the most popular one.
I do note that most of the users who complained about the package manager at the time it was merged have since changed their tune after we polished it off; and the users on the linked Reddit thread are probably just wondering why the package manager had to be added at all, since it was the main cause of the release being delayed.
> Apple did. Twice.
Okay. So why are you unsatisfied with macOS, and see the need to build your own system, then?
> In Haiku, this means presenting files to the user as though there are in one place when they're really in another
Huh?
So you are saying you do not like the feature of Windows Explorer that you can look inside .zips? Or the feature of most Linux desktops that you can mount ISOs or other disk image files and modify them without, you know, actually having them on physical partitions? Because the Haiku package manager is no different than either of these in terms of "showing the user something that 'isn't there'."
> with the results that trying to manipulate them leads to errors that require you to peel back the abstraction, the "magic", to understand.
The files are on a volume that is (mounted as) read-only. You will get equivalent errors from trying to modify read-only volumes on Linux, macOS, Windows, or any other OS. So what is really "magical" about this? There is no "abstraction" here, the files really are on a read-only volume.
> The most usual usecases for who? Haiku devs I imagine.
And users. There are only a dozen or two developers; there are hundreds (maybe thousands? nobody has done real counts) of users.
> But that doesn't change the fact that I don't agree with your choices.
Again, you are disagreeing from a theoretical standpoint here, not a practical one, of "I used this system and it is not viable for X, Y, Z reasons", only "this seems like magic to me and I don't like it."
> Because I'd be hard-pressed to think of something that has brought more complexity for less benefit than package managers.
Correction: Linux package managers. Haiku is one of the first major examples of a dependency-solving package manager that does not follow the Linux philosophy on a large number of points.
I really do believe that the major problems you cite about the Linux package managers are design flaws in the Linux philosophy, not package managers themselves, and that Haiku has sidestepped nearly all of them.
> Package managers, even Haiku's, don't allow this because they "optimize for the 99% usecase", or rather, the 99% of the developers' use cases.
This just shows you have not used (or at least read the documentation on) Haiku's package manager, because it does allow this, though the feature is not exposed to the GUI because it is, as noted before, very rarely used (if ever?). You can mount a package file from anywhere, to anywhere, with the `mount` command as with any other disk image. HaikuPorter uses this very functionality to create chroots, in fact.
> or rather, the 99% of the developers' use cases.
Once again you are implying that Haiku is somehow an elitist system that only the developers have any say in. This is just not true, and I don't know why you continue to imply it.
> I'm just an old curmudgeonly bastard anyway, you don't need to win me over.
If you want to be opposed to package managers "just because," then no, I don't. But if you are going to state your opposition to them on technical grounds that Haiku's package manager is not subject to, then yes, I am going to dispute your claims.
Call me when I can put an application on a thumbdrive and take it to another system and run it without having to worry about dependencies or being connected to the internet. I can do this with AppImage, MacOS App Bundles, and quite a lot of Windows programs and I can do it today, not some "maybe we'll implement it" future. Oh yeah, and there's no magic, they are single files or folders and can be manipulated as such without any special magic from the OS. My understanding is that even if Haiku was modified to accommodate this kind of thing, it would require magic handling to do so because dependencies.
Also, you want to talk practicality? In part 4 Probono had to install Haiku to a larger disk because he was outgrowing the one it was installed on. Here's a question: why could't he keep his install where it is and just put new applications on a different partition? Like you can do on a Mac, or on Windows (usually), or with AppImage?
There are practical limitations imposed by hpkg. You think they're reasonable trade-offs, I disagree.
I really don't understand why you seem so intent on convincing me that my opinion is incorrect. This is a real limitation of your system that actually exists, the complexity behind it is real. You're of the opinion that those are not significant, and I am of the opinion that they are. Your constant attempts to convince me otherwise, mostly by telling me I'm in the minority, or my usecase is wrong and invalid, are not and never will be well received. Frankly, they remind me far too much of Linux Desktop evangelists.
> You can mount a package file from anywhere, to anywhere, with the `mount` command as with any other disk image.
Which I can only assume means I must manually mount any dependencies not in a blessed path first before I mount the application that is also not in a blessed path, since the convention is not to package non-base system dependencies with applications.
You can already do all of that up until the "without having..."; and as I said in the article, someone could write a script for that in a matter of an hour; and of course a GUI in ... as long as it took to bikeshed ;) There is no technical limitation here. We have not implemented a GUI for it because, as I said, few users seem to want or need it.
> Oh yeah, and there's no magic, they are single files or folders and can be manipulated as such without any special magic from the OS.
Why do you keep calling this "magic"? Packages in Haiku are file systems. This is in fact how AppImage also works, which you seem to like. Are you complaining about how AppImage is internally implemented?
> My understanding is that even if Haiku was modified to accommodate this kind of thing, it would require magic handling to do so because dependencies.
No, it would not. It would indeed require magic on Linux because the tools that understand dependencies, where to get them, and interface with the user (e.g. apt) are separate from the tools that manage the actually installed packages (e.g. dpkg.) No such distinction exists on Haiku.
In fact there is a screenshot in the linked article where probono dragging a package into the system packages directory, and the system automatically asked him if he wanted to install its dependencies. All that is missing from your usecase is a way to get these from a local folder instead of downloading them as is done right now. It would in no way be some "magic hack" that violates abstractions, good design, etc. like it would on Linux.
> Here's a question: why could't he keep his install where it is and just put new applications on a different partition? Like you can do on a Mac, or on Windows (usually), or with AppImage?
I already explained this: you can do it manually already by using the "mount" command. Support for it does not exist at the GUI level, and very few users requested it, so we (the development team, who are not paid to work on this, and have ~1-2 man-months total per month to spend on Haiku between the entire team) did not implement such a feature that less than 1% of people want. But once again, there is no technical barrier here and it is already possible if you don't mind CLIs.
> There are practical limitations imposed by hpkg. You think they're reasonable trade-offs, I disagree.
So far, the "technical restrictions" you have referred to in this comment are actually incorrect, they are simply "there is no GUI" not "this is not possible with the present design." The one you referred to before ("can't modify packages") is also a restriction of AppImages, I believe? So I don't actually know what technical restrictions you are referring to at this point.
> I really don't understand why you seem so intent on convincing me that my opinion is incorrect.
You are perfectly welcome to dislike packages because you want to use .zips, or some other personal preference or the like. I am, however, intent as I said in the last comment on disagreeing with you when your opinion is "I don't like X because it can't do/does Y", when in fact X can or does not do Y, as is the case with most of your objections to the Haiku package management system thus far.
> You're of the opinion that those are not significant, and I am of the opinion that they are.
And which one of us has spent 5+ years both working inside said system as a developer, and on said system as a user, and in the community of said system with the other users who use said system on a daily basis, some even as a daily diver?
> Your constant attempts to convince me otherwise, mostly by telling me I'm in the minority, or my usecase is wrong and invalid, are not and never will be well received.
No. That is not what my "in the minority" has (largely) meant. It has meant that "we did not implement X because few people seem to want/need X and we have very little time in the first place," i.e. if we had more time, we might have implemented X, and if someone sent us a patch that did X we would likely accept it; not "your usecase is not valid."
> Which I can only assume means I must manually mount any dependencies not in a blessed path first before I mount the application that is also not in a blessed path, since the convention is not to package non-base system dependencies with applications.
Correct. But you can add another hard-coded blessed path to the system with two lines of code and a recompile. We would almost certainly accept a patch that added blessed paths via a configuration file. (Again, this seems to be rarely needed or requested; this does not mean it is not a valid usecase, it means our precious little time will be spent first on more important things. Patches and/or sponsorships wanted.)
It's possible I'm misunderstanding here, but aren't hpkgs mounted in an overlay fashion? That's not how AppImages work. But also, AppImages are designed to deal with the shortcomings of Linux, unfortunately necessitating some complexity. One would expect a fledgling OS like Haiku would have more flexibility.
> So far, the "technical restrictions" you have referred to in this comment are actually incorrect, they are simply "there is no GUI" not "this is not possible with the present design."
If the functionality isn't exposed to the user it may as well not exist. And if it is as straight forward as you claim (which I doubt), why hasn't it been done? You seem to be motivated to shut me the hell up enough that you've probably spent the requisite amount time arguing on the internet about it already.
> And which one of us has spent 5+ years both working inside said system as a developer, and on said system as a user, and in the community of said system with the other users who use said system on a daily basis, some even as a daily diver?
I'm not arguing that this isn't what Haiku users want, I'm arguing that it isn't what I want, because that's the primary determinant for me.
> No. That is not what my "in the minority" has (largely) meant. It has meant that "we did not implement X because few people seem to want/need X and we have very little time in the first place,"
"95% usecase", you say, but you can't even determine the order of magnitude of your current user base (is it hundreds or thousands?), let alone any potential future users. But whatever, lets accept that as true. I don't care. I'm not saying you should do it my way, I'm saying I don't like the way it is done. That's all. Appealing to your rationale for why you don't do it my way isn't going to convince me otherwise.
> Correct. But you can add another hard-coded blessed path to the system with two lines of code and a recompile.
Holy shit. I have to actually recompile? That's some Linux-grade badness there.
> We would almost certainly accept a patch that added blessed paths via a configuration file. (Again, this seems to be rarely needed or requested; this does not mean it is not a valid usecase, it means our precious little time will be spent first on more important things. Patches and/or sponsorships wanted.)
I don't think blessed paths are a good solution in the first place (see: magic). There should be no such thing, files and folders are just files and folders and structure should be entirely up to the user.
But anyway, as I keep saying, it is completely fine that you guys, the people actually doing the work, do things the way you see fit. Yes, I get that you have relatively limited time, as we all do (there's a reason my desktop os ideas aren't actually a project yet). However, and again as I keep saying, that doesn't mean I have to agree with your choices.
They can be mounted either way. The /system/ packagefs mountpoint is a kind of "overlay" in the loose sense, but it does not use the standard "overlayfs" but rather packagefs manages the "overlay" itself for a variety of reasons (both technical and practical.)
But I don't know what difference that really makes vs. AppImages.
> If the functionality isn't exposed to the user it may as well not exist.
Okay, so it "doesn't exist," but it still could be implemented without too much effort.
> And if it is as straight forward as you claim (which I doubt), why hasn't it been done?
Because, like I already said, we have precious little time and far more important things to work on, and very few people seem to have wanted this. And some of the ones that (at one point) did were third-party application developers who could have written a tool to do it, but they didn't either, so it seems in the end they also found it unnecessary.
> You seem to be motivated to shut me the hell up enough that you've probably spent the requisite amount time arguing on the internet about it already.
If all you were saying was "I want X feature but you are never going to do that / it is impossible", I am going to spend my time arguing against the "but". Then once it gets down to "I want X feature", I would to convince you to file a ticket or ask about it on the forums to collect feedback so we could actually consider it and prototype something, and eventually implement it if there is enough demand and we have the time.
But you seem to have no interest in actually using Haiku probably ever, so spending time implementing a feature that one person currently wanting it may never use does not sound like a good investment to me.
> but you can't even determine the order of magnitude of your current user base (is it hundreds or thousands?),
We (intentionally) don't collect metrics of how many actually running systems there are; we can only estimate from the number of downloads of the nightly images and package updates. Which are consistent with "hundreds to low thousands." Hopefully our disdain for pervasive user tracking will not incur a demerit in that we do not know certain things trackers would tell us...
> I'm saying I don't like the way it is done. That's all. Appealing to your rationale for why you don't do it my way isn't going to convince me otherwise.
Again, your disputes are "I don't like the way it is done because X", when in fact X is not true, and that is my dispute.
> Holy shit. I have to actually recompile? That's some Linux-grade badness there.
Uh. You have to recompile code in order to add features. If someone added support to get this from a configuration file, then you wouldn't. Not sure why you are so surprised.
> I don't think blessed paths are a good solution in the first place (see: magic).
Next you are going to tell me that /dev and /proc, being "blessed paths" are "magic". If you are not going to say that, what is the actual difference between that and this, that makes these paths "not magic"?
> and structure should be entirely up to the user.
If you are really saying entirely, without any limits, then you are right, we are going to hit some disagreements. If you do admit some limits (really, any at all), then we may still agree. macOS is an "opinionated" system, and Haiku is too; not nearly as much, that's for sure, but we are not the "wide open west" of Linux.
Right, magic.
> But I don't know what difference that really makes vs. AppImages.
AppImages don't pretend to be somewhere they're not. The mounting just gives the application a way to see its files. hpkgs are mounted as part of an installation step, AppImages are mounted as part of a launch step. It isn't ideal, but like I said it has to work within the limitations of Linux.
> But you seem to have no interest in actually using Haiku probably ever
This is true. I have to say, having my only interaction with the community be arguing with you has notably soured me on it, unfair to the project as that obviously is. But more than that, there's just too much magic in it for all the limitations it still has. In the far future when it has VMs, application sandboxing, and reasonable hardware acceleration it will be easier for me to accept the things I don't like about the design, but that's a long way off. A simpler system would appeal to me a lot more and make it easier to sacrifice those other things.
> a feature that one person currently wanting it
Probono also wants it, he said as much in this very article, and he is an enthusiastic user of Haiku right now.
> We (intentionally) don't collect metrics of how many actually running systems there are;
I'm not saying you should, I'm just saying you're being a bit presumptuous when you claim to speak for 95% of users, whom you've not only never polled but also can't even measure. Not to mention that such claims sound a lot like the "99% of users just want Chrome" excuses of Linux Desktop evangelists. This is why I keep saying that what you really mean is 95% of Haiku developers, the parts of the community you have frequent contact with.
> Again, your disputes are "I don't like the way it is done because X", when in fact X is not true, and that is my dispute.
X is true. hpkgs are not simple and employ "magic", they are not bundled with their dependencies, they cannot (today) work from arbitrary locations without jumping through hoops and recompiling. These are true statements. The only disagreement seems to be how much this matters.
> Uh. You have to recompile code in order to add features. If someone added support to get this from a configuration file, then you wouldn't. Not sure why you are so surprised.
I'm surprised no one thought to make it a configuration option already. It seems like such a simple thing to look at a hardcoded path and say "maybe I shouldn't hardcode this path".
> Next you are going to tell me that /dev and /proc, being "blessed paths" are "magic".
Yes and no, because you can mount devtmpfs and procfs in arbitrary paths, but also yes because too many developers hardcode their location. I'm going to go with "yes", and in an ideal system they wouldn't even exist as part of the "file" hierarchy, per-se. UNIX baggage, etc.
> If you are really saying entirely, without any limits, then you are right, we are going to hit some disagreements.
Ideally, yes. I don't see any reason that any kind of file structure need be imposed on the user. Ideally there is no "system" partition, no "/boot" as I understand it to be called in Haiku, there's just whatever volumes exist and whatever files are on them. The system was booted into ram from a single file, its interfaces are not exposed as a file hierarchy. Applications and documents go wherever the user wants them, on permanent media, removable media, network shares, whatever. There's no such thing as having to "install" anything. The system is a file, the applications are files (or folders). Boot a new version off a thumbdrive and all your applications are available because hey, they're just files and folders out there on volumes.
Because the world isn't perfect, yeah, it is unlikely to be this perfect. For one, the location of the system file will be subject to the whims of the boot mechanism. But really, that's still a far cry from "all applications must be installed on the same volume as the system or otherwise in a list of blessed paths".
overlayfs and iso9660 are not magic. So is mounting two ISOs in an overlayfs on Linux magic? Or on macOS? I guess I really do not understand how or why you quantify things "magic" at all.
> AppImages don't pretend to be somewhere they're not.
The entire concept of "filesystem mounts" means data pretends to be somewhere it is not. Haiku has used the concept in a slightly unorthodox way, but it is fully within the bounds of the concept, and we could do the same on Linux without modifying the kernel.
> I have to say, having my only interaction with the community be arguing with you has notably soured me on it
Once again, I am not trying to change your opinions; I am trying to understand and then correct your misinformed understandings of how this all works under the hood. If your objection to Haiku is "I don't want a package manager", point blank, then I can tell you why people like them and why we do, but obviously there is nothing for me to argue with there.
> there's just too much magic in it for all the limitations it still has
I would really, really appreciate it if you could explain to me your understanding of the word "magic", and how it applies so frequently to technology...
> Probono also wants it, he said as much in this very article, and he is an enthusiastic user of Haiku right now.
So then that is one. He has also only used Haiku for a week now. Maybe he'll change his mind; maybe he won't and the feature will get implemented. But either way, the stuff we are focused on right now impacts the whole userbase (system stability, optimizations, porting more apps), and so we are going to keep that as the focus.
> I'm not saying you should, I'm just saying you're being a bit presumptuous when you claim to speak for 95% of users, whom you've not only never polled but also can't even measure.
I speak for and about the users who have spoken up. Perhaps I'm wrong; but if there are people with strong opinions, I would expect them to express them somewhere in some fashion. I haven't seen anyone do so on this topic for some time.
> Not to mention that such claims sound a lot like the "99% of users just want Chrome" excuses of Linux Desktop evangelists.
But are they right? 99% of people who tell me they want to use Haiku more but don't, say "if it had Firefox or Chrome..."
> This is why I keep saying that what you really mean is 95% of Haiku developers, the parts of the community you have frequent contact with.
The Haiku forums and mailing lists are active enough that concerns and comments very often show up there. We have had more than one major feature or bugfix made because people chimed in there... In fact, one of the features probono noted as missing (word-wise-delete in text editors) was implemented last week as a result of that. We definitely do listen to the community.
> hpkgs are not simple and employ "magic"
What is "magic"?!? All of the major behavior of them is documented at various levels and depends on no "hacks" or undefined behavior.
> they are not bundled with their dependencies
If their dependencies are merely Haiku itself (i.e., as native apps should), they have nothing to be bundled with. If they are non-native apps that use Qt or the like, yes, they are not bundled, because a 100MB Qt package + 5 1MB apps is much smaller than 5x100MB bundled Qt apps, etc. But yes, you are correct here.
> they cannot (today) work from arbitrary locations without jumping through hoops
Yes, there is no GUI yet.
> and recompiling
That is only needed if you want the paths to be "blessed" with the dependency solver. But if you are only using native and not ported apps, you have no need for dependency solving...
> The only disagreement seems to be how much this matters.
And about the nature of "magic", it seems. But indeed we seem to have fewer points of misunderstanding by now..
> I'm surprised no one thought to make it a configuration option already. It seems like such a simple thing to look at a hardcoded path and say "maybe I shouldn't hardcode this path".
Haiku's philosophy is "sane defaults and minimal configuration where necessary." We have a "pseudo-test" that we generally go through before adding new options to the GUI (1. Did you ever change this setting? 2. Did changing it make you happier?)
We do have the find_paths() API for applications to not hard-code paths, but, you know, somewhere there has to be a hard-coded path for find_paths to return.
> UNIX baggage, etc.
Well, I certainly am not the biggest fan of the "everything is a file" philosophy indeed; but I also think if you took a poll among Haiku users and developers about `/dev/` you would not get a majority concensus to hide it more (it already does not appear in the GUI), and most would either be neutral or opposed to removing it (in favor of handles.) So if you are opposed to even `/dev`, then I think your philosophy and Haiku's are much more irreconcilable than I realized.
> Ideally there is no "system" partition, no "/boot" as I understand it to be called in Haiku
You are misinformed. `/boot` just refers to the partition Haiku is currently booted from. So `/boot/home` is where the normal home directory is (unless one has moved it). So there is no "system partition" segmented from the "data partition" as such, no.
Essentially, `/` on Haiku is not existent in any real location; but is rather the root mount point for filesystems. So `/boot` is the boot partition, I might attach a thumb drive and it will appear as `/waddlesplash thumb` or whatever else its name is, etc. In the GUI, navigating to `/` shows the nominal names of the partitions (with /dev/ and other such dirs hidden, usually.)
> The system is a file
But this is what Haiku does; the system is `haiku.hpkg`, and we just also have those files appear in `/system/, too, because to us hiding them completely would be much more "magical" (or more properly, opaque).
> But really, that's still a far cry from "all applications must be installed on the same volume as the system or otherwise in a list of blessed paths".
Which is a decision that is, as I noted already, open to change. There is no technical reason it has not been done.
Magic is illusion. It's things appearing to work in a way that they don't actually work, and as such the illusion collapses when you attempt to do something not accounted for by the magician. Case in point: Haiku applications appear to be files, but if you try to move them then you get told you can't modify a read only file system. Even if you did manage it it sounds like they wouldn't work because you didn't really move the application, because it isn't really there and is part of a different file (hpkg) somewhere else that you should have moved instead. The only way to fix these inconsistencies is to employ even more magic to hide the way things actually work. That's magic.
> If your objection to Haiku is "I don't want a package manager", point blank, then I can tell you why people like them and why we do, but obviously there is nothing for me to argue with there.
I have other criticisms of Haiku, but we seem to be stuck on this one so I haven't brought them up. Not that it matters, because as mentioned I'm just someone on the sidelines pointing out things I personally don't like while you and other Haiku developers are actually doing the work. I have a particular hatred for package managers, it's true, and it's because they only exist to manage dependencies which is something I don't think should have to be managed at all, and they do so by restriction.
> Haiku's philosophy is "sane defaults and minimal configuration where necessary." We have a "pseudo-test" that we generally go through before adding new options to the GUI (1. Did you ever change this setting? 2. Did changing it make you happier?)
Consider that both of these things are highly subjective, and who's choosing them? If it is developers, then, well, it's kind of a feedback loop isn't it? Keep going that way and if you're not careful you become Gnome.
> Well, I certainly am not the biggest fan of the "everything is a file" philosophy indeed; but I also think if you took a poll among Haiku users and developers about `/dev/` you would not get a majority concensus
Why do you insist on making this appeals to the majority all the time? The end of that logic train is "everyone should just use Chromebooks". The fact that you're even bothering to work on Haiku suggests minority opinions should matter doesn't it? Besides, as I keep saying, I don't really care what the "majority" thinks.
> to hide it more (it already does not appear in the GUI), and most would either be neutral or opposed to removing it (in favor of handles.) So if you are opposed to even `/dev`, then I think your philosophy and Haiku's are much more irreconcilable than I realized.
As I said, ideally /dev wouldn't exist. I assume Haiku has it because it wants to be more UNIX compatible. If I made my own system based on the Linux kernel I'd almost certainly have to expose it somewhere too simply because there's no other way to interact with devices in Linux.
> You are misinformed. `/boot` just refers to the partition Haiku is currently booted from. So `/boot/home` is where the normal home directory is (unless one has moved it). So there is no "system partition" segmented from the "data partition" as such, no.
I can see how that could have been misconveyed, but that is what I meant. The "system" partition is the officially blessed one where applications are "installed" into the system.
> But this is what Haiku does; the system is `haiku.hpkg`, and we just also have those files appear in `/system/, too, because to us hiding them completely would be much more "magical" (or more properly, opaque).
Fair enough, but why does it need to have applications installed on the same partition? You keep saying that this isn't a technical limitation, which I guess is true so long as one is somehow able to properly bless the appropriate partitions. Of course, right now that would mean recompiling Haiku if I wanted to run something off a thumb drive.
> Which is a decision that is, as I noted already, open to change. There is no technical reason it has not been done.
If what you're saying is true, then no, it's just a cultural one. That's almost worse. Linux doesn't need to do things the way it does either, but culturally it is impossible to do something else at this point.
It's not an illusion, though. They really are files on a read-only file-system. The message is entirely correct. AppImages would do the same.
> I have other criticisms of Haiku, but we seem to be stuck on this one so I haven't brought them up.
Well, I am interested in those, too, of course. And despite how antagonistic this apparently is to you (sorry), I really am taking notes here.
> Why do you insist on making this appeals to the majority all the time?
Because the general case is the one that should be optimized for first, and other ones come later.
> If it is developers, then, well, it's kind of a feedback loop isn't it? Keep going that way and if you're not careful you become Gnome.
If you are trying to protect software from the developers making it, you are going to have a bad time.
> Besides, as I keep saying, I don't really care what the "majority" thinks.
That's fine if you are just you. Obviously people creating products have to think about it.
> Fair enough, but why does it need to have applications installed on the same partition?
Nothing "needs" that, this is just how it is set up; the current default as it were.
> Of course, right now that would mean recompiling Haiku if I wanted to run something off a thumb drive.
Because there is no GUI, yes. Or you know, like I've said before, the executable bit still exists and you can use it, too.
> Linux doesn't need to do things the way it does either, but culturally it is impossible to do something else at this point.
Yes, indeed...
> If what you're saying is true, then no, it's just a cultural one. That's almost worse.
No, it is not. Linux has created in everyone the fear of the culture above all else, to the point where elaborate technical solutions are crafted to cultural problems. Haiku has, and will continue to, change its "culture" when it becomes clear the time has come for that.
The user is never exposed to the mounted filesystem of an AppImage, which is only mounted while the application is running. Unless I'm mistaken, which is entirely possible, hpkgs don't work like that. The hpkg is mounted and the filesystem is exposed to the user who launches the application through it.
With AppImage, no expectation exists that the mounted fs is a real fs because the user never interacts with it.
Let me put it another way: Suppose Haiku already had all the changes it would need to use hpkgs in arbitrary locations. So I plug in a thumbdrive containing an application hpkg and all its dependency hpkgs. The hpkgs get mounted, then I launch the application, then I close the application. Now I remove the thumbdrive. What happens?
With an AppImage in this scenario, what happens is exactly what you'd expect: nothing. The AppImage fs was mounted at launch, and unmounted at close, and never seen by the user.
When is the last time you browsed through /usr or C:\Program Files\ in a file manager?
The same is true in Haiku. It's there for the power users; normal users should not need to go into it and poke around. The user should interact with it via the Deskbar, desktop symlinks, etc. Not by browsing /system/.
> The hpkgs get mounted, then I launch the application, then I close the application. Now I remove the thumbdrive. What happens?
The filesystem the thumbdrive contained gets unmounted, and in the process all child vnodes and file descriptors are closed, and so the packages on the thumbdrive are consequently unmounted also.
> With an AppImage in this scenario, what happens is exactly what you'd expect: nothing. The AppImage fs was mounted at launch, and unmounted at close, and never seen by the user.
HPKGs would be mounted for as long as possible so that the applications contained could be, say, launched to handle a file. But as I noted above, this should indeed work just fine on Haiku with nearly the same paradigm as AppImages do.
Literally every day.
> It's there for the power users; normal users should not need to go into it and poke around.
See right here is a red flag for me in terms of culture: I do not consider there to be a difference between "power Users" and "users". This distinction only exists as an excuse for why things have to suck. Why does this thing suck? Oh, it's because normal users won't understand X or only want Y, unlike special snowflake power users. I think users should be treated with some respect instead of condescension. Again, one of my big problems with Linux Desktop culture, and maybe you can see why I've previously painted Haiku with that same brush. You're excusing an incongruity in the metaphor presented by the system and how the system actually works with the Linux Desktop Evangelism approved mantra "well normal users don't care".
> The filesystem the thumbdrive contained gets unmounted, and in the process all child vnodes and file descriptors are closed, and so the packages on the thumbdrive are consequently unmounted also.
Well that's good, at least it doesn't hang around as a ghost until the user attempts to interact with it and it throws an error. It still seems odd that one would have to perform the extra step of "installing" (mounting) the hpkg before finding the application and running it, which is an advantage of AppImage: you just run it.
I don't really know what to say here because, well, it's true: they don't. The majority of users of technology are not technically savvy and just want to write documents, talk to their friends, get work done, etc. They don't know how it works and don't really much care, they just need it to work.
> Why does this thing suck? Oh, it's because normal users won't understand X or only want Y, unlike special snowflake power users.
We developers are obviously "power users" ourselves, so we need the system to work for that case; and lots of our users could not write a line of code to save their lives, so we need the system to work for that case, too. (Unlike a lot of Linux desktop developers and supposed evangelists, who you see using macOS instead of Linux and just developing inside VMs, which might be the source of why Linux is so terrible in this regard.)
So why bother making Haiku? All of that strawman's needs are served perfectly well, in fact even better, by a Chromebook.
Maybe this is something you can only really get if you grew up with computers that booted into BASIC.
> We developers are obviously "power users" ourselves, so we need the system to work for that case; and lots of our users could not write a line of code to save their lives, so we need the system to work for that case, too.
Sure, and I'm saying that a system need not put a wall between the two. One side that understands "how it really works" and another that is presented with a facade and expected never to look behind the curtain. A well designed system can be understood in pieces, or layers, giving the user a path to more power and a better understanding of their tools as they seek it at their own pace. The attitude I'm against is the one that says if you don't want to do the "normal user" thing, then the next step is a C compiler.
That is not true in theory or in practice except for a very, very small number of people. Maybe if you use the "Linux on Chromebook", but then it's more than a "Chromebook" is typically understood to be...
> Maybe this is something you can only really get if you grew up with computers that booted into BASIC.
What "this" are you referring to here?
I think 99.999% of people that use computers are more than happy with the fact they no longer boot into BASIC, and would dislike them doing so again, and this includes 99.9% of programmers, I'd say.
> A well designed system can be understood in pieces, or layers, giving the user a path to more power and a better understanding of their tools as they seek it at their own pace.
There isn't a "wall". You can go as deep or as shallow as you like; and there are plenty of middle grounds, too, in Haiku. So we definitely adhere to this part of your philosophy.
> The attitude I'm against is the one that says if you don't want to do the "normal user" thing, then the next step is a C compiler.
And we definitely agree with that, for the first two or three standard deviations. But way out in the long tail? Yeah, you are going to need the compiler.
What do you mean? Your strawman user just needs email, a webbrowser, etc. That's all perfectly well served by a Chromebook.
> I think 99.999% of people that use computers are more than happy with the fact they no longer boot into BASIC, and would dislike them doing so again, and this includes 99.9% of programmers, I'd say.
Yeah, see, this is not-getting-it. It isn't about booting in to BASIC, it's about recognizing that the Computer is a tool to make the user's life better, and part of that role is making sense and guiding the user towards leveraging it better.
> There isn't a "wall". There is an abrupt incongruity in the metaphor for how applications work. They appear to be files in some location (/system/Applications?) but are actually not. The user sees that /system/Applications/MyHaikuApp is a file and thinks "oh, I can just treat that as a file", but then discovers it isn't true. They were lied to. It's an illusion.
> But way out in the long tail? Yeah, you are going to need the compiler.
No disagreement here, but consider your solution to me wanting to use hpkgs from a different partition.
Maybe it is in theory. In practice it turns out not to be; Microsoft Office or LibreOffice do exponentially more and faster than Google Docs, online image editing is really not on par with desktop editors either free or open source, conference calling is still mostly Skype or Zoom that have much better apps than websites, etc. Or maybe they need tighter integration than the web provides. My point is just that I have rarely come across users that really, truly could get by with a web browser, and only a web browser.
I don't know why you claim that my theoretical user is a "strawman." It's most of the people I know that use computers, and survey shows that it's indeed most of the people who use computers in general.
> Yeah, see, this is not-getting-it.
Your comment was extremely vague, and I specifically asked what you meant, so it's not surprising I got it wrong...
> It isn't about booting in to BASIC, it's about recognizing that the Computer is a tool to make the user's life better, and part of that role is making sense and guiding the user towards leveraging it better.
Of course I agree with this. Haiku has some phrases very, very similar to this in our FAQ, user guide, and interface guidelines (https://www.haiku-os.org/docs/HIG/index.xml).
> There is an abrupt incongruity in the metaphor for how applications work. They appear to be files in some location (/system/Applications?) but are actually not. The user sees that /system/Applications/MyHaikuApp is a file and thinks "oh, I can just treat that as a file", but then discovers it isn't true. They were lied to. It's an illusion.
But they are files mounted in that location. You can treat them as a file: copy them to another location, get info about them, search them, etc.
You cannot modify them in that location; but are users really that unfamiliar with the concept of "read-only files / folders"? Can we really not trust them to understand that? We decided that we could, so that's what we did, instead of hiding all this behind more layers of abstractions. We aren't "lying" to them by showing them those files, it's just the same as if they plugged in a volume and chose to mount it read-only.
> No disagreement here, but consider your solution to me wanting to use hpkgs from a different partition.
Yes, and as stated many times above, your use-case is in the minority (i.e., the long tail) ... for now. When we get around to better enterprise support, then it will be less so (i.e. run packages off of network drives), but that is not a current priority at all. But as also already stated, we would be more than happy to accept a patch that added this.
And if I copy that file out of /system/Applications, and put it on a fresh Haiku install somewhere else and try to run it, what happens? Granted it might actually work, but not if it depended on any other hpkgs not in base Haiku or things that were in its hpkg. If applications actually were just files (or even folders), then this would work as expected.
> Yes, and as stated many times above, your use-case is in the minority
And therefore irrelevant, yes, I get it. My point is it shouldn't matter that my usecase is in the minority, because my path towards such simple functionality [0], functionality suggested by the --unfortunately illusory-- metaphor presented to the user, shouldn't require me to break out a compiler. Instead it is complicated, because a complicated system was put in place to manage dependencies, a problem so desperately in need of solving that Electron, which bundles an entire web browser with every application, is popular and widely used.
[0] DOS, the original MacOS, RiscOS, NeXT Step, the new MacOS, etc. all had this working.
But it would also mean the LibreOffice package would be 300-400MB instead of 100MB; etc. etc. We chose the dependency-solving model precisely because we care to save (consecutively) terabytes of data transfer, disk space, etc. in exchange for a moderate amount of added complexity. Re-packaging applications is not so difficult for the users who really need that. Most seem not to.
> And therefore irrelevant, yes, I get it.
Two comments ago, you agreed that use-cases in the long-tail were rightfully not optimized for or prioritized in the default install. Nowhere did I say (and I indeed said the opposite) that this meant they were "irrelevant." Do you now disagree with this?
> My point is it shouldn't matter that my usecase is in the minority, because my path towards such simple functionality [0]
So you are saying that all "simple" usecases, in the "minority" or not, should be implemented? I think you might be surprised at just how many there are. Then how we prioritize all of those?
So I don't see why any use-case, simple or not, should get any priority besides the number of users that want or need it multiplied by how crucial the activity is for them. Especially given how starved for time we are at present.
> a problem so desperately in need of solving that Electron, which bundles an entire web browser with every application, is popular and widely used.
Considering that Qt has facilities to bundle applications together into a single binary, as do most other cross-platform toolkits, I don't think this is the reason most people chose Electron (especially seeing as most Electron apps install quite a lot more than a single binary.)
Conveniently you left off the important part: the thing I'm trying to do is a thing that appears to be possible with the presented metaphor, so therefore should be simple, but since the metaphor is illusion it is not simple.
> Considering that Qt has facilities to bundle applications together into a single binary, as do most other cross-platform toolkits, I don't think this is the reason most people chose Electron
Consider that such things exist because despite every OS having a dependency manager to "solve" the problem, it is still less of a headache to just bundle dependencies!
Again, it is technically possible, we just did not put it in the GUI. So you are correct here.
> so therefore should be simple, but since the metaphor is illusion it is not simple.
I once again dispute your claim that the "metaphor is illusion" for all the aforementioned reasons...
> Consider that such things exist because despite every OS having a dependency manager to "solve" the problem, it is still less of a headache to just bundle dependencies!
Every OS? No, only Linux does. (I don't think the Windows and Mac App Stores count as "dependency managing" because, well, they don't check dependencies at all.) Obviously Haiku does not stop programmers from bundling their dependencies, they are still perfectly free to do so. But developers don't make applications for Haiku themselves for the most part (yet.)
The metaphor is that the application is a file. If you look at /system/Applications/ that's what you see, right? Is the application a file? No, the file is a part of the hpkg, itself part of a dependency chain of hpkgs that are the application.
> Every OS? No, only Linux does.
Windows has a package manager too believe it or not, but it mostly just applies to Microsoft's own junk, .NET stuff, and C runtimes. I wouldn't be surprised if MacOS had one too, but I also wouldn't be surprised if it didn't since it got Application Bundles right the first time when they inherited it from NeXT.
> Obviously Haiku does not stop programmers from bundling their dependencies, they are still perfectly free to do so.
Nothing stops Linux developers from doing it either, but culture forces mean it doesn't happen enough to rely on.
As I said in my previous comment, that really is a file. So how is this an illusion?
The only things that put stuff outside of `apps` are, uh, non-applications (e.g. CLI tools with a command in `bin` and data in, well, `data`, libraries with headers in `develop` for GCC to find, etc.) Which the user should not know or care about.
Applications with binaries in `/system/apps` and data elsewhere are rare. Most ported applications with no headers or the like get put into `/system/apps` along with all of the libraries they ship and the like. (Of course Qt apps will need the Qt package, etc.; but native apps at least can just get copied out of /system/apps and will still work.)
Alright, so that's a result of me misinterpreting what I was seeing. Dependency problem still considered legit.
> The only things that put stuff outside of `apps` are, uh, non-applications (e.g. CLI tools with a command in `bin` and data in, well, `data`, libraries with headers in `develop` for GCC to find, etc.) Which the user should not know or care about.
This is another of those things I disagree with. A commandline environment should be just another application. Like, a "UNIX" or "GNU" environment would just be an application that opens a terminal interface to a self-contained little world with the common expected commandline tools. Of course, without any form of application sandboxing or namespaces in Haiku that isn't really possible. I just really don't like the idea of global state that isn't part of the base system.
It's an especially bad idea with build environments. There's a reason people put build environments in docker containers.
> Of course Qt apps will need the Qt package, etc.;
So here's another thing I don't get. One of the big problems with the Linux Desktop that we can both agree on is how everything is so disjointed and fragmented right? Haiku has a native UI, yet you encourage straight up porting of applications using other toolkits anyway. I get that an OS needs applications because people have shit to do, but why not just make them bundle QT with the application? At least then Haiku users would never have to worry about the dependency at all, and you wouldn't have needed to build a package manager to manage it.
I'm sorry to say that the dependency problem is really the fault of those applications, and we are really doing our best to support them while not consuming excessive disk space or too high of a maintenance burden.
Native applications should ideally depend on Haiku itself and nothing more, and then we would not have this problem, and probably Apple feels the same way about macOS ones. But we can't all live in that world just yet...
> A commandline environment should be just another application. Like, a "UNIX" or "GNU" environment would just be an application that opens a terminal interface to a self-contained little world with the common expected commandline tools.
Okay... but at least in this case, users don't usually (ever?) use GUIs to copy around binaries of CLIs because Linux has specifically trained them not to. So we could invent a new paradigm here, but a lot of this stuff is unfortunately hard-coded into said CLI tools and it may be prohibitively difficult.
> Of course, without any form of application sandboxing or namespaces in Haiku that isn't really possible.
Well, there are chroots and su, but that is probably not what you had in mind.
> I just really don't like the idea of global state that isn't part of the base system.
Well, on Haiku, it is avoidable: never install anything in `/system` besides the base system, and instead install all your packages in ~. This at least is supported already (I don't know if HaikuDepot has an option exposed for it, but pkgman has the -H option which does exactly this.)
> It's an especially bad idea with build environments. There's a reason people put build environments in docker containers.
Indeed. HaikuPorter builds always run in chroots with only the declared dependent packages available to them.
> Haiku has a native UI, yet you encourage straight up porting of applications using other toolkits anyway.
Yes, because the manpower to make third party applications does not exist yet. There are about 2 (literally, two) third-party developers who do not also have commit access to Haiku itself. We would very, very much greatly prefer using Haiku-native applications; the ecosystem just does not exist, and until it does we are tolerating (and this is the word other developers would use, too) ported applications to close the gaps.
> but why not just make them bundle QT with the application?
Because then every Qt application would be 75-100MB larger, rather than having one 50-70MB Qt package and shipping each Qt app on its own? And on a system where a lot of apps are ported Qt ones, this will potentially be a 10x increase in space usage, as most Qt app packages are < 10MB (and the complete image of the base system is itself only 415MB or so, remember, and 60MB of that is the Noto fonts.) Not to mention that when there is a Qt update you then have hundreds of packages to rebuild instead of just one...
> At least then Haiku users would never have to worry about the dependency at all
Haiku users already do not "worry" about the dependency: They push "install" and the dependencies are installed, and that's that.
It took years to make the package manager. You honestly believe it was worth it for this?
> Well, there are chroots and su, but that is probably not what you had in mind.
su definitely isn't. But depending on support for bind mounting, in particular read-only bind mounting, chroots might be.
> we are tolerating [...] ported applications to close the gaps.
Which is fair. Though personally I'd have just put them in a compatibility environment, like WINE but translating from Linux to Haiku. A lot of things wouldn't need to port then, and those that did require some changes would still probably be easier than a straight-up port. Of course, I'd have built on top of the Linux kernel anyway and got that compat for free by just creating a new namespace.
> Because then every Qt application would be 75-100MB larger, rather than having one 50-70MB Qt package and shipping each Qt app on its own? And on a system where a lot of apps are ported Qt ones, this will potentially be a 10x increase in space usage, as most Qt app packages are < 10MB (and the complete image of the base system is itself only 415MB or so, remember, and 60MB of that is the Noto fonts.) Not to mention that when there is a Qt update you then have hundreds of packages to rebuild instead of just one...
Then why not just include QT in the base system? You seem resigned to relying on QT applications, may as well make it official. How is spending years to develop a package manager better?
> They push "install" and the dependencies are installed, and that's that.
Unless, and I realize there are a whole lot of people out there simply unable to grasp the concept of this, they don't have an internet connection.
It took about two years. The rest was largely getting HaikuPorts into shape and making it behave the way people expected. Few (if any?) major changes to the package manager itself occurred after the initial development and then merge. So the main time was actually getting stuff ported for the most part.
> But depending on support for bind mounting, in particular read-only bind mounting, chroots might be.
Bind mounting is supported. It does not appear to support read-only binding, but I only looked at it for about 10 seconds just now.
> Though personally I'd have just put them in a compatibility environment, like WINE but translating from Linux to Haiku.
If we are going to have to tolerate ported apps, we want that to be on our terms, not theirs. Part of that is having them be actually integrated with the system as much as we can do -- i.e. clipboard, native file dialogs, icons, look and feel, etc. Otherwise, people will just feel like they're using a shoddy recreation of Linux (which is what it would be) and not an OS in its own right.
> Of course, I'd have built on top of the Linux kernel anyway and got that compat for free by just creating a new namespace.
One of Haiku's main goals is the holistic integration of the complete system, which would be seriously compromised if we used the Linux kernel. There are a good number of instances where we made changes to the entire stack that would simply have never been possible on the Linux or even a BSD kernel.
> Then why not just include QT in the base system?
Why not just admit defeat and surrender to Windows? They have 90% of the market-share, anyway.
Adding Qt to the base system wouldn't just be a compromise on our principles, it would essentially be a complete abandonment of them.
> How is spending years to develop a package manager better?
For all the reasons I already iterated...
> Unless, and I realize there are a whole lot of people out there simply unable to grasp the concept of this, they don't have an internet connection.
Yes, probono remarked about this, and (quite literally today) there has been discussion about how to better facilitate this. It seems we have more or less reached a concensus on starting with "pkgman export": collect a package and its dependencies inside a .tar, and then support reading these in "pkgman install" (and eventually in the GUI by some mechanism.) So, support for this should be merged before too long.
And I guess this is where our priorities on principles differ. To me, the simplicity of singe-file or single-folder applications is much higher. Let's say Haiku starts to take off in the next few years --and honestly I'd like to see that just so Linux Desktop isn't the only non-proprietary choice--, I'll bet that you'll still be dragging around that QT dependency for the foreseeable future. People will wonder why it isn't just installed by default because of its ubiquity.
That's ok, you'll say, because the package manger takes care of it and so users don't have to think about it. But at that point the distinction between being part of the base system and just being a nigh-mandatory package is pretty moot.
I'm not sure why you're finding it so hard to understand that most developers and members of the community wanted package management.
I understand that a lot of the community, and especially the developers, wanted package management. I don't contend otherwise. I contend that it being a popular idea didn't make it a good one.
I'm a JS developer, but I enjoy learning and using C in my spare time. I'd love to try contributing but it seems like you use C++ so it might be a few more years :D