New implementation of Git in OCaml
github.com
github.com
I feel like OCaml is one of the programming world's best kept secret - if it had better tooling and better marketing, it could have taken over the world. I'm cautiously optimistic about ReasonML for this reason.
It's outputted js is fantastic.
It would known as F# :)
Next, everything builds so slowly, and just trying to set up Core, utop, and ocp_indent is a trial in patience, watching the same dep build again and again, confusion about OPAM switches.... nope nope. OCaml/OPAMs's got a distribution problem. They've pushed the complexity off to the end user and the experience is terrible. Jane Street planting their flag on the language and encouraging massive dependencies like Core, or shit like "corebuild" is even worse. I would never depend on anything OCaml-based on it unless it could be installed by my OS' package manager (Arch/AUR doesn't count, because it builds). That rules out most popular OCaml tooling, and ruins the development experience.
I'd prefer OCaml to Haskell, I find it more practical, but I feel much better depending on Haskell-based software, which generally "just works." As much as they're a wart, Haskell syntax extensions are a real benefit. You can build new fancy Haskell software with older GHC versions.
I think your overstating the pain though. Even in other languages, best practice demands you pin a library version so you wouldn't have some of those issues.
I think you mean FB's ReasonML. Buckle is Bloomberg's OCaml-to-JS compiler.
Howso? Stable Rust has been nothing but 100% solid for the past 1+ year I've worked with it.
I dunno, maybe you had a bad experience but considering how well Cargo and the stable compiler work I don't thing it's fair to categorize Rust as never working correctly.
There's a push to remove "optional dependencies" which are the reason why opam dependencies rebuild again and again: http://rgrinberg.com/posts/optional-dependencies-considered-... For example in the Mirage project we've been working on this https://discuss.ocaml.org/t/ann-major-releases-of-cohttp-con... but it has caused some breakage here and there.
jbuilder (from Jane Street) is excellent: expressive, easy to understand, builds packages extremely quickly, is actively developed, has minimal dependencies and a lovely manual http://jbuilder.readthedocs.io/en/latest/ It takes care of generating boilerplate for other tools like merlin (which to be honest I never got around to manually configuring). There's also work to integrate it with utop https://github.com/janestreet/jbuilder/issues/114
jbuilder also supports building multiple libraries in one big source tree, so we could switch to a package lockfile model: the author uses opam to create a solution to the package constraints and checks in the specific versions known to work, the build clones the dependency sources and jbuilder builds it all simultaneously. I'm keen to try this on one of my larger projects so that "git clone; make" just works, irrespective of where the host OCaml comes from.
PPX syntax extensions depend on specific compiler versions, so when (for example) homebrew updates to OCaml 4.05 you might find that extensions you need have not been ported yet. ocaml-migrate-parsetree aims to fix this problem http://ocamllabs.io/projects/2017/02/15/ocaml-migrate-parset...
There's obviously still plenty of work to do, but I think things are improving!
But a lot of things I'm trying to use aren't quite there yet. You mention jbuilder and the generated .merlin... it just doesn't work with current opam packages (at least not with ppx - I'm not doing anything fancy, just basic project with core). To fix this, I had to learn about opam pinning features, and then I had to carefully choose the jbuilder commit that works (the one after the bug has been fix, but before it breaks the compilation of bin_prot that I also use) and so on... As for Merlin, it's great, but the documentation is minimal. When it doesn't work as expected, it's hard to find where to look.
Core and Async are great, but besides 'Real World OCaml' that is a bit outdated and insufficient, there is little documentation and not much help on Stack Overflow.
That being said, I'm not complaining and I'm grateful to the guys developing these tools. But I think newcomers should expect some difficulties if they want to do anything serious using these most recent tools.
Whenever a package is added to the repository, the CI tests (the latest compatible version of) every package that depends on it to check everything still works. However, non-current packages may break if they're missing upper bounds (it doesn't check all previous releases).
However, since opam-repository is just a Git repo, you can clone a known snapshot of it at some point in time and keep using that. That allows you to continue using old versions of packages without the risk of new software causing unwanted upgrades.
> My impression is that INRIA is too eager to add features and tweak the language.
That's strange. I've always had the opposite impression - that they maintain compatibility at all costs, including leaving sub-optimal APIs in the standard library.
> Next, everything builds so slowly, and just trying to set up Core, utop, and ocp_indent is a trial in patience, watching the same dep build again and again
The Core alternative standard library is pretty huge, indeed (I don't use it). I just tried installing all the tools you mention in a fresh container:
$ time docker run ocaml/opam:debian-9_ocaml-4.05.0 opam install core utop ocp-indent
6:03.75elapsed
So, 6 min to set up a dev environment with those tools. That installed 67 packages, so compilation took about 5 seconds/package on average. I'm not sure why it would build the same dep twice - I haven't seen it do that.
"That's strange. I've always had the opposite impression - that they maintain compatibility at all costs, including leaving sub-optimal APIs in the standard library."
When I started using OCaml, many standard library functions weren't tail recursive---you couldn't get the length of a list with more than ~10,000 elements, for example. The library definitely seems like an afterthought compared with the language. (And that would be why there are so many of them.)
Just pick your algorithmic use case for a good excuse to use list and not an array from here:
But often programs are used long after they are designed, and it is great if they can degrade gracefully as you move outside their original design parameters.
Out of curiosity, how feasible is it for opam packages to use lockfiles like npm is nowadays? Then packages which depend on some other package will continue to build against the exact same version they always did, until and unless they are explicitly upgraded to the latest version of that dependency.
> I'm not sure why it would build the same dep twice - I haven't seen it do that.
Maybe https://discuss.ocaml.org/t/why-does-opam-recompile-so-much/... provides a clue to that.
But,
"I'd prefer OCaml to Haskell, I find it more practical, but I feel much better depending on Haskell-based software, which generally "just works.""
I take it you haven't been bitten by cabal? It's a nightmare every time I touch it.
On the other hand, I've been using stack (https://docs.haskellstack.org/en/stable/README/) with hakyll, which seems to improve the situation dramatically. It's just that first build for a new project that's very slow.
Haskell could solve that with binary distributions - but that requires more resources than are devoted to it, it seems.
http://cml.cs.uchicago.edu/ -- the CML idea
Three SML implementations with CML support
http://www.pllab.riec.tohoku.ac.jp/smlsharp/
https://github.com/smlsharp/smlsharp
https://github.com/massivethreads/massivethreads
It was conceived at Tohoku University for use in Japan's high performance computing projects. SML# uses LLVM as a backend, has a nice FFI and EDSLs for SQL and JSON and a non-moving GC. All stemming from the use case it's designed for.
Docs are more of a WIP right now, but I'm confident they'll get a handle on them: https://reasonml.github.io/api/index.html
I'm very happy to see the standard library documentation is available for Reason, that alone tells me the Reason community cares about improving the experience for new developers (which is not the impression I get from the OCaml community).
In many projects where one has to "stand on the shoulders of giants", OCaml has a limited ecosystem to draw from.
A practical compromise is F#, which is an ML language for the .NET platform. And even with Microsoft's heft behind it and a rich .NET ecosystem, F# is still a niche language used in finance and a few other specific domains.
A functional language "taking over the world" is a tough proposition. That usually requires a strong use-case (like scientific computing--and now data science--for Python) and fungibility of expertise, which in turn calls for a language with a very low barrier to entry.
Even people using eg C++ no longer look at you too funny if you explain that you prefer all your variables to be 'const'.
If you are into functional programming, you should rather choose Scala. Huge ecosystem, largely used in production. Compatibility with java mean you can have all the library you want. A lot of people love to trash sbt it's build system, but for me it always work as expected. And other tooling are just awesome (you even have IDE).
Do you have any sources for this?
For many on the MS treadmill this is seen as a negative, as you can't always get visual studio RCs and be months ahead of the competition on the next version of blend/WPF/silverlight/whatever.
For others, the community driven aspect and lack of obsessions with technical fashion in the MS product line (with the track record they have...), is seen as a _major_ feature.
F# has a much more active open source community and presence than C#, most of the devs are actively cross-platform in a way C# still struggles with, and it has a better .net stack than C# aimed at command line efficiency and relatively stable approaches to common problems.
F# is still a first class language from MS, and would be quite OK on its own these days. Regardless, there is no chance MS will "lose interest" in F# given the impact on data science, machine learning, cloud computing, parallel computing, and stream-processing F# has. MS invested heavily in filling out their language portfolio to avoid being excluded from the big servers and big clusters. Time has shown those concerns to be growing, not shrinking :)
If you like Reason, you might also be interested in BuckleScript.
I feel like OCaml is one of the programming world's best kept secret
It would be interesting to see an alternative implementation like this get enough feature parity that projects like the BSDs could migrate and still be protocol and worktree compatible so most users could continue using the GPL2 implementation.
Edit: Seems people here are unaware that the BSDs treat GPL software like cooties, see e.g. [1]. They wouldn't start using source control that gave them less freedom than SVN, an with CVS there's the expectation that OpenCVS might get finished.
Shipping an unmodified copy of git and calling it should be fine even in proprietary code, no?
Yes, you can literally do this. It's not a derived work because your code is entirely independent of the GPL'd code and the GPL can't possibly cover code that you wrote independently. Simply interacting with some external GPL'd program does not expose your code to any GPL requirements.
For example, many databases are GPL'd and provide their entire functionality over RPC. Many CLI programs are GPL'd and (of course) provide their entire functionality over CLI. Neither of these cases place GPL obligations on clients, and it's no different for any given library which you wrap with a CLI.
A contrived wrapper that is not an independently useful program does not allow you to circumvent the GPL.
You are leaning very heavily on a technical distinction (linking vs. not linking) instead of on a holistic analysis of what a "derived work" is.
Now imagine that I rather than edit the video files themself create a playlist, in a (fictive?) format which has the ability to play subsequences of the linked files, and then creates a version of the series which presents the complete opposite message of the original. Would a court find this to be a derivative work of the original videos? What if in my need to edit the video I want to insert certain new short sequences, and thus I distribute a second video along with the playlist, but this video is useless in itself since it just contains a number of short clips in sequence. Would this video be considered a derivative work of the original - remember it serves no purpose in itself except along with the playlist and the original? To me it is not clear, but if they are not derivatives, then this will practically render the concept moot. Since most derivatives could potentially be formulated as the original + a diff. Now I would expect in this case that the combination of the playlist + my video file + original series be considered a derivative work, remember that in the arts transferring a work from one format to another (a dramatization for example) is considered to be a derivative work, even if all the actors would be seen on stage to read from the original book.
As I read it, this is the center of the FSFs argument. Now whether this would apply equally to software I do not know. But I have a hard time imagining that copyright is not as easily circumvented as you seem to indicate.
I think it's pretty obvious that the GPL derives it's power from copyright. Reasoning: If the GPL does not apply, then you don't have any right to anybody else's code (by the Berne Convention), so it gets its power by granting you more rights than you would have had by default. Make sense?
> The FreeBSD Project aims to produce a complete,
> BSD-licensed operating system allowing consumers of the
> system to produce derivative products without constraint
> or further license obligations.
Also https://www.openbsd.org/goals.html and https://www.netbsd.org/about/redistribution.html#why-berkele...
Requiring OCaml in the base operating system could equally be a reason to avoid this, whatever licence is used.
I'm not sure I understand this concern in the majority use cases. Why would you need an OCaml compiler installed?
Every few years I take a look at it, notice the license and move on. There's no reason whatsoever for a language not to have an implementation under a permissive license.
If you still have issues with that .. I would really like an explanation.
But for example, Ada Core's GNAT is licensed under GPL but doesn't have that exception, which makes the binaries it outputs be licensed under the GPL as well, to make you buy the Pro version of the compiler.
Good lord. Surely, no one is working on reimplementing CVS in 2017?
The CVS repository (https://cvsweb.openbsd.org/cgi-bin/cvsweb/src/usr.bin/cvs/) seems to actually have quite a bit of recent activity, so it seems to be a serious project that someone intends to deploy in the real world. There are a handful of things that the OpenBSD project does that I feel are quixotic (continuing to use CVS for everything, in general, is one of them), but this one takes the cake. CVS was good when compared to what came before (and in some regards even compared to Subversion), but there's no comparison with git (or any of the modern DVCS, really).
IIRC, when I last looked at this, they were doing dual implementations in OCaml and Scala, but now it looks like it's being done in Rust.
>Means no more downtime, no possibilities of censorship, be it from states or from companies.
Wrong.
>historically, patch-based systems have been very simple to learn and use, but slow, whereas snapshot-based systems can be extremely fast, but are usually hard to use for more than simple operations. As an example, cherry-picking is not intuitive in git/mercurial
How is cherry-picking not intuitive?
>Category theory has certainly been an inspiration for Pijul, but categories are neither algorithms nor data structures in themselves. In order to get the semantics we wanted, especially the handling of multiple files, rollbacks and unrecords, designing and implementing new algorithms and data structures was at least as useful as learning theoretical stuff.
Yet another thing with relatively little practically use, but hey, it uses category theory! It must be good! Oh look, it's also functional!
All of that said, props to Pierre-Étienne for putting in the time and effort to make this. He probably learned a lot.
Edit: for the record I have Googled the topic and am overwhelmed by how much is out there. Looking for personal recommendations.
For what it's worth, I got a surprising amount of mileage from learning Idris[0], of all things. Even if you ignore the dependent type system, it is still essentially Haskell but with a lot of the warts removed, making it much simpler to grok.
Beyond that, the usual recommendations apply: Learn You a Haskell[1], and Real World OCaml[2] were what got me started.
[0] http://docs.idris-lang.org/en/latest/
If some concepts remain unclear, Learn You a Haskell[1] is a good complement, though it doesn't go as deep as Real World Haskell. For example there is no coverage of monad transformers, which is a pity.
Typeclassopedia[2] is a great overview of common category-theoretic abstractions used in Haskell programs.
[0] http://book.realworldhaskell.org/read/
In any case, that book ain't bad either. It's better than Learn You A Haskell.
I also like "Discrete Mathematics Using a Computer" (https://github.com/ryukinix/discrete-mathematics/blob/master...). It focuses on teaching discrete mathematics and introduces some Haskell as executable mathematical notation on the side.
My experience has been that, generally speaking, programmers of language [X] don't have a particularly good grasp of what makes a good introduction to language [X], and I think that effect is even stronger with a difficult language like Haskell.
That said, I was about to read some of the new material in HPfFP, and I should also read the analogous wikibook content as an experiment. Thanks for linking it.
I remember a particularly vivid example of some of my coworkers at Google suggesting the Go tutorial, starting at https://tour.golang.org/welcome/1, to someone completely new to programming. They couldn't even understand how it's not perfectly clear and simple.
That being said the book that I've seen most success with getting someone from absolutely zero to "can start making progress on their own" is "How to Design Programs" (http://www.ccs.neu.edu/home/matthias/HtDP2e/index.html). It helps that the authors regularly teach absolute beginners.
Haskell, OCaml, etc take those functional idioms and apply strong typing, algebraic data types, pattern matching and something like the Option/Maybe type - but they build on the same functional concepts in Realm of Racket.
Caveats to the above:
1) Racket has an optional 'typed' variant which gives you type checking.
2) Rust is (IMHO) a little weaker with regards to pure functional programming than OCaml, but very nicely integrates pattern matching, type checking and algebraic data types in.
When I wanted to build something beyond a toy program, https://pragprog.com/book/vmclojeco/clojure-applied helped me immensely. It goes beyond "what you can do" and delves into "when you should do it". IMHO every language should have a book like this.
He focuses on teaching the syntax, semantics, and idiom of Standard ML, a classic functional programming language that's closely related to OCaml, Haskell, and others. He has a gift for explaining simply and clearly and keeps his lecture videos short (~10min) which helps to absorb their info. Check it out: https://www.coursera.org/learn/programming-languages
https://confengine.com/functional-conf-2017/proposal/4052/gi...
Edit: I get it now: it's an OCaml library that allows you to interact with git repositories and provides many (most?) operations.
In this case, he saw a limitation about the Garbage Collector of git (which can use a lot of your memory) and the push command. It's like just: "I want to understand what is Git (in the specification) and I will do". So, the big goal of my work (payed by the MirageOS team) was to switch to a non-blocking (memory predicted) implementation of Git.
I did not have any criticism about this approach. Sometimes you just want a project to work and ocaml-git was developed in this mind.
However, ocaml-git is used now by some big companies (like Docker, Tezos, etc.) and need a strong prediction about the memory consumption first. Then, the push command is a big lack of the last implementation. And, finally, the GC and encoding of the PACK file is the key to solve all of these problems.
As I said in this PR, I worked with Thomas Gazagnaire and some others peoples of the MirageOS eco-system to improve ocaml-git, implemented the Git GC, Git push and tried to avoid any problem about the memory consumption in a server context in the low-level API.
So, yeah, it's a +19k -8k PR, of course. But it's not like a OCaml's noob and tries to restart the world like: yeah, I recoded Git in OCaml in my only opinion and don't care about what was it happens in the OCaml world, Haskell world and industrial world, like just for fun.
In this PR, I explained IRL and in comments what happens. Why I did this choice compared to something else. The point of Thomas Gazagnaire (who reviewed my PR) and my point. What the MirageOS team expect and what I did.
So, clearly, yeah you did not read my comments and just see a big PR like a noob to try to strike all of this project but this is not what happened unfortunately for you. It's a result of a big discussion between Thomas Gazagnaire (specifically), others peoples and me to find the best for all (in the implementation and in the API).
Now, I can say this PR will be merge. I need to polish some details, improve the API and test it. So, yeah this big PR will be merge because it's what expect the creator of ocaml-git, what expect the MirageOS team, what expect my boss and what expect others users of this library if you just interest by the issues.
What I'm saying is you are not using git effectively. It is much easier to read and understand merge requests the smaller they are. Someone who didn't write the code should be able to go through your merge request in one sitting and understand all of its implications.
You may have written a good "implementation of Git," but you have also demonstrated that you don't know how to use it effectively.
From the OP's other comment, it sounds like this merge strategy was the preference of the repository owner. Why substitute your own preferences for theirs?
ETA: Also note that the PR was reviewed several times over the summer. GitHub has a feature that allows you to only review changes to a PR since the last time you reviewed. You can also manually use `git diff` to review that difference.
I'm always amused by people telling others what is more readable and understandable as if it was a hard fact and applied universally to all situations.
In reality, you can have a +19k diff which is easier to understand than a +100/-100 diff, and it happens quite often. Have you ever read a PR and clicked the "view" button to see the whole file? Have you ever clicked on the arrows to expand the context of a particular diff line to learn what the heck a given name is? Consider that you wouldn't have to do this if it was all in the diff in the first place.
The readability of the code is an elusive quality, with largely inconclusive research. There's very little in terms of facts to rely on. The best you can say is that a particular way of presenting the code, or changes, feels better to you. Good for you, but don't try to force that way on others, as you're guaranteed not to improve the readability and instead encounter a violent pushback.