How we might abolish Cabal Hell, part 1
well-typed.com
well-typed.com
There is also no single cross-platform package manager. Mac has macports and homebrew, windows barely got one, linux has yum, apt and others. As a language designer, you don't want any part in this, you want your code to work. So you make your own.
edit: To clarify, I'm suggesting that we're stuck in a local maximum - now that all these systems already exist and have widespread adoption and tool dependency in their own environments, there isn't much of an impetus to use Nix or something similar even though in the grander scheme of things it'd be way nicer.
With an operating systems package manager, the focus is on shipping working, uber stable code, which is unlikely to break someones system. This (imho) is due to two different factors: 1) Operating systems have to work, no two ways about it, if your OS is broken, everything else is broken. 2) Users of operating systems are not necessarily experts. If their package manager ships experimental code which breaks their particular system, they don't know how to report the bug to the central OS maintainers. I think this is handled somewhat by having different repositorys with different levels of stability, a la arch linux's [core/extra/community/testing/AUR], however at the end of the day, the main repository must avoid having breaking code.
With a programming language package manager, the focus is on having up to date features, and catering towards power users who, if things go wrong, can generally fix them, or at least know how to contact, and how to phrase their requests for assistance. This means that it's generally more acceptable to expect users of a programming language repository to occasionally be served broken code, as defects will be reported more rapidly, and more clearly.
I find it interesting comparing the two package managers that I use most often in my day to day computing experience: Cabal, and pacman (Arch linux). Pacman offers a single version of each library per repository, and that version is as stable and tested as the repository requires. This is in keeping with the spirit of an operating system package manager, as it allows power users to install unstable packages from more experimental repositories, but tries to serve as stable code as possible to general users. Contrast that which cabal, which basically offers no stability guarantees, but allows for much more fine grained control over library versions, sandboxing etc.
TL;DR
In my opinion, OS and Language package managers have goals which are at odds with each other: OS package managers want to maintain stability, Language package managers want to allow for bleeding edge code. The concessions that cabal makes towards stability (versions, sandboxes etc) often cause other problems as well.
- The "puts things on disk" part (posix "install" or a low level tool like dpkg or rpm is most like this)
- The "determine what needs to be installed" part, (aptitude, yum)
I'm all for a per-language or system version of the latter, but once that's done, you should generate packages installed by the former.
Heck, wrap all the commands (cp/mv/install/chmod/chown/etc.) that write stuff to permanent places on disk to actually do "add to a package", give it a basic name/version number, and have the low level tool handle adding/removing it from the system (or multiple systems, or deploy it, etc.). All the dependencies, compatibility, etc. are handled by the higher level system.
This gets you the best of both worlds - system level packages, and the ability to install whatever you need. FPM (https://github.com/jordansissel/fpm) is a pretty good example of this philosophy.
But, instead, we get every punk ass CPAN descendent spraying it's crap all over the filesystem, needing the whole build environment installed, touching the internet in weird ways that don't guaranteed repeatable behavior, etc. sigh
For anyone constrained to GNU/Linux, right?
I don't feel constrained by a free software operating system. Besides that, a bunch of people use Nix on OS X. On Windows I suppose you're stuck with the inferior package managers.
I appreciate the sentiment, but there's a substantial difference between "constrained to" and "constrained by".
Why do people only think of the triad?
And yet all these nifty package managers can't even seem to get things as simple and long solved as permissions straight. Even autoconf sets permissions better than, say, pip, where I have to remember to check my umask before I "sudo pip install something".
I agree we shouldn't have one package manager per language, but I think so far no one package manager has proved sufficiently general and robust to serve all use cases. For example, I think something like Nix would be wonderful for all language communities, but it doesn't run on windows. That immediately takes it out of the running. And as far as I'm aware, all other package mangers would have the same "Hell" problems of cabal because of the GHC optimization mentioned above.
I know Python. Would I prefer to write my package manifests in some bastardized Ruby DSL or Python? Of course I'd prefer Python and the Ruby guys would prefer their bastardized DSL. I don't blame them.
How do we bridge that gap? Python => Ruby => Python is bad enough. What about M4 or AWK or Makefiles or Haskell or god help us all some shitty custom format ala Puppet?
You might as well demand everybody on Earth speak English.
That's working out quite well so far.
> That's working out quite well so far.
Not really. English is a minority language, spoken as a primary language by only about 5.5% of the world's population.
http://en.wikipedia.org/wiki/List_of_languages_by_number_of_...
One more juicy anecdote: there are more Chinese people learning English right now, than there are English speakers in the U.S..
You don't need to be an expert on the language used in your package management. 'second language' is a rather apt analogy.
Yes, I basically agree. I think there's a difference between a native speaker and someone who can order food in a French restaurant -- I mean, without getting snails when what he really wanted was escargot. But I agree, it's not a very important distinction.
Then they claim that theirs is "easy" but they've only done the first half of "simple things should be easy, hard things should be possible".
A properly autotooled package offers an amazing level of control about where to put things.
Cabal just tells you up front when this will happen which can be annoying. That said, the prior mechanism requires that you notice when version mixes are causing issues.
A couple of months ago, I wanted to write a simple Web App using Yesod. The only way to get a working installation of yesod and all the dependencies was by using the sandbox thing. This however meant, that after every code change, you had to wait ~10 secs for the project to rebuild. That doesn't sound like much, but it gets annoying very quickly.
I think it's unfortunate that Yesod gets billed as the premiere web framework for Haskell because it does require some extra work to get compiled.
In fact, the entire Stackage ("stable hackage") project, I believe, was originally developed in order to get some tooling which would allow Yesod to be consistently built. If you think you're going to be primarily using Yesod then I suggest you switch to Stackage [0]. Stackage prevents install issues by building the entire Stackage group together periodically to ensure that there are no build problems. The downside is that Stackage is partial and slow to track Hackage.
You may want to try Happstack: http://happstack.com/page/view-page-slug/9/happstack-lite-tu...
Yeah, that's the biggest hit to my productivity, when working on a moderate sized Yesod project. Otherwise, the experience has been great.
Or use snap, I've never had build hell with snap. But if yesod works for you, awesome! :-)
I've been meaning to try and gather some empirical data as to whether this is actually the case.
If you have libraries A and B which each require mutually exclusive versions of library C, what is stopping the compiler from adding both versions of library C to the compiled code for A and B to call separately?
Obviously this would result in larger binaries and should generate a build/install warning, but it seems like this would solve the overwhelming majority of Cabal's dependency resolution issues.
There's clearly something I'm missing since this has been an outstanding issue in Cabal for quite some time. Would someone with a better understanding of GHC/Cabal's compilation/linking/installation steps mind shedding some light on this?
* A gets a value from its version of C.
* It passes that to your app.
* Your app gives it to B.
* B gives it to its incompatible version of C.
In particular this is, for whatever reason, a frequently occurring problem with the `bytestring` library. There are a lot of SO questions related to figuring out why the compiler is rejecting the use of a ByteString type even though it appears it should work... but that's because the compiler error is eliding the package version difference.
On the other hand, if C is present in A's interface, then it s version has to agree with all other versions of C visible in the same unit of code (package/module/namespace).
Thanks for the clear explanation BTW.
It's just hard to read that into Haskell since "modules" are not as well represented in-language and exist somewhere between Haskell-the-language and Cabal-the-package-manager/ecosystem.
This is also a better fit in JS where the community seems to prefer packages that don't expose many real "objects" with methods and stuff. Packages tend to expose bare data-bags and functions from what I've seen. That makes "I got an object without the methods I expect" errors less frequent.
I personally feel that approach is too unsafe and definitely wouldn't have been a good fit for Dart, but it does kind of sort of work for npm.
Of course, it's totally broken in the presence of dependencies cycles, which is why npm now has "peer dependencies" and is right back to having to resolve shared constraints.
Suppose C is a JSON library, for example. In my code I fetch a JSON object from A, with library version C1. What happens when I pass that object to library B, which uses C2? Is that an immediate type violation because C1 and C2 are treated as different types? Do you live with runtime failures when, say, B calls a C2 method not in C1? Do you you try to create some sort of magic object conversion and hope that the data isn't too different? Or do you try to require all libraries to describe how to convert data from every version to every other version?
Even if the two libraries never interacted and data could be proven never to flow between them, there's a real question as to what version of the library is available from the main code.
Despite my concerns, I suspect that you could make this mostly work if you were cavalier about possible runtime errors, because actual conflicts would be rare. But from what I understand of the Haskell community, they're not big on "cross our fingers and hope it works at runtime" approaches.
The symptoms are a result of the confluence of a few things (mentioned elsewhere in this discussion), some of which are quite positive, which make the problem substantially harder. Which isn't to say there's no problem, or that nothing could or should be better - I'm glad it's getting attention and I'm glad people have ideas for improvements, though both of those have been true for a while.
I experienced this immediately after I read Learn You A Haskell and it made me give up on the language. I develop on Windows (currently?) and I was passionate about creating my first hobby project in Haskell. But every direction I turned, I ran into an issue where a dependency or transitive dependency expected some linux library to be there and I couldn't install.
Maybe I'm just spoiled and need to be more open minded. I come from Java-land where I take "write once run anywhere" for granted. I eventually switched to Clojure but that was unsatisfying for different reasons. I wish there was a language with Haskell's purity and type system but Java's ability to "write once and run anywhere".
I think Frege is very close, but I feel uncomfortable learning something with such a small community (damn... I am spoiled).
In my experience, this sucks in basically any language.
As a counter-anecdote, my experience is that Windows is frictionless for developing PHP, Node.JS, C#, Java, Scala, Elixir/Erlang. It's also close-to-first-class for Python. Ruby to a lesser extent, with a lot of the community just assuming you're on OSX, period.
I've often been impressed by how, despite the hacky nature of the Node.JS community, a lot of stuff simply just works there. For example, getting PhantomJS, a full-fledged scriptable headless browser, available for a project is a single "npm install" away on all three major OSes.
I remember a get-Windows-working-better rallying call at a Ruby conference I was at 4 years ago, and as far as I can tell, things have not changed much since then. The OSX thing isn't quite fair though - everything works on linux too, because that's what people deploy on. In fact the trend seems to be toward developing against containers running your production version of linux, rather than OSX directly.
I suppose a lot of the issue comes from the fact that Cabal is very careful about everything. It seems like most package managers I use have a "try and maybe fail" attitude, while Cabal seems to have more of a "guarantee success or fail" attitude.
My config is here: https://github.com/seagreen/vivaine
But seriously. It's so difficult to realize how much of a game changer a great package manager can be. I've never tried Haskell, but is this really THAT much of a problem? I hear so much praise and worship about Haskell...
I'm actually intrigued to see what comes out of this. When the Haskell community turns its mind to something like this, something quite interesting tends to come out.