bitwalker

Instead of "babysitting" your compiler, you're "babysitting" your application at runtime - I fail to see the difference.

That's not always true, rank-1 polymorphic functions (i.e. functions polymorphic in their argument) don't usually require type annotations. Higher-rank polymorphic functions though do generally require them, because without the annotations type inference is undecidable in the general case for such functions. For example:

    pair x y = (x, y)

This function creates a pair for any two types, and doesn't require any annotations.

Yeah, there does seem to be an undertone of "it's such a pain to have to prove things to the compiler that I know are right", even though the vast majority of the time the compiler is right and you messed up. I never quite understand the feeling that one wants to pin the blame on the compiler when you make a mistake and get that feedback right away. I guess in some cases, like Rust, where the compiler is really particular about enforcing certain rules, that might be frustrating, but one reaches for a tool like Rust because of those rules and the guarantees they provide, so it's kind of silly.

I'm not sure why one would ever use ADTs for the situation you described, I totally agree it's not a good fit, but it's not what they were designed for anyway. But based on what you've described either a plain StringMap, or if you need to store different data types, a GADT with different type constructors for each of the types which might be present in the data that you want to work with (i.e. String, Integer, StringMap, List, etc.).

The StringMap is trivial, and the GADT requires declaration, but is straight-forward pattern matching/construction from that point onward, so I don't think either option impose any "real" additional cost here.

Navigating/extracting datastructures via lenses or recursion/pattern-matching is clear and concise, definitely not onerous by any means.

I'll gladly admit that my argument hinges on using a state of the art type system, but in my opinion, if we're going to debate the merits of static typing, then we should be looking at the best it has to offer. It seems like perhaps you haven't had an opportunity to work with Haskell or something like it (want to be clear that's not a judgment statement, and I'm solely basing this on your last comment, if you have, sorry about that!), which is fine, I mean, I only ended up learning some of it due to diving down the rabbit hole of type system research while designing my compiler - but if the first thing I had to compare a dynamic language to is Java, then yeah, it's no contest I would use Clojure instead.

I think something important to remember is that I don't think one should be a "static typer" or a "dynamic typer" - dividing the world into right/wrong, good/bad, black/white is almost always the wrong point of view. If I'm automating something, or doing some exploratory programming, I'm probably going to reach for bash/python/etc.; if I'm writing a command line tool or simple service I'm probably going to use go; working on a concurrent system, I'm probably going to reach for erlang/elixir/go/pony/etc. - but if it's even remotely complex, I'm always going to favor the option with the best static type system, because I know it's going to help me (and anyone else working on the system, especially newbies) keep things sane. Over half of what I just listed do not have static type systems, but I still have a clear preference - it is possible to use the right tool for the job without compromising your ideals. Part of why I started working on my own compiler was that I wanted to design something that both provided the strong type system, while still functioning well as a "get shit done" language, something with the concision of Haskell, but the flexibility of ML (permitting both functional and imperative paradigms), a comfortable FFI, and good tooling, including the ability to run scripts interactively (for example, via shebang scripts). I haven't found a language that mixes all of that together in a way that satisfies me, which is why I still use a variety of languages for different tasks. So I guess what I'm saying is, try not to divide people into two camps - it's possible to live in both at different times, but still prefer one or the other :)

Static types represent proofs, depending on the language they may be very weak proofs, but nevertheless, that's what they are for. I see nothing wrong with that statement. Another way of putting it is "the compiler verifies your proofs"

Good points!

1: That's true for anything which isn't a primitive type, or built in to the language, sure. Most of the time though, the type you have to define is not exactly complex, take records for example (using a Haskell-ish syntax):

    data Person = {name: String, age: Int} deriving (Eq, Show)

That's something not only easy to define, but something that you'd _want_ to define anyway even in dynamic languages, i.e. via `defstruct` in Elixir, or `class` in Ruby/etc. The language (at least in Haskell, and in the language I'm working on) can help you derive the common interfaces you'd want (structural equality and inspection/stringification in the example), and then that's it, type inference takes over from there. In Haskell/ML, you typically only need type annotations when working with higher-ranked polymorphism (e.g. functions which operate on polymorphic functions), and you can go a long way without needing to write code like that. In other words, the burden at this level is extremely lightweight, and I would argue that it imposes no more burden than you would already undertake in well-written dynamic code (i.e. defining structures/classes).

Once you get into a place where you are writing abstract code to work across types, with no type constraints, where higher-rank polymorphism is likely to be front and center, then yes, you will spend some extra time up front annotating those functions with their polymorphic type, and that can be annoying sometimes. Even that's not a given though, in Haskell for example, one can define a typeclass (such as Eq, for equality), and write code which operates on instances of that type without needing to do anything more than:

  foo :: (Fooish a) => a -> a
  foo a = # do something Fooish with 'a'

That function is polymorphic across any type which is an instance of `Fooish`. I mean, I guess one could consider that an unbearable burden, but to me that's at least as much "extra" as I would do in Elixir with typespecs, and in another language via function docs at a minimum. Writing abstract code like that in a dynamic language probably still requires you to perform some checking of the arguments to ensure that they are actually of a type that is valid for the function.

You can go into production writing code without your compiler proving that what you expressed to it is valid, but if something is wrong, you don't have any tooling (short of debugging) to fall back on, you have to figure out what went wrong, try to fix it, deploy again and hope you actually fixed the problem. With (good) static type systems, you may be slower to production that first time, but the compiler has ensured that you haven't made any mistakes based on what you told it you were trying to do. One can of course still make mistakes with a type system helping you, but at that point it's a conceptual mistake, i.e. you've told the compiler to do A, and it did A like you asked, but the right thing was actually B.

Nevertheless, I understand the desire for a compiler to just do what I say, regardless of whether the compiler thinks it's ok; but most of the time when I do that, it means the next person working the code behind me isn't going to understand it - if it's not clear to the compiler, it's probably going to be unclear to a reader of the code too. That isn't a hard and fast rule, but I have certainly seen instances of that.

In my experience, most of the dynamicism I rely on in such languages is for metaprogramming, all of which I would rather do at compile-time via macros anyway, and thus doesn't require a dynamic type system. For everything else, I'm having trouble thinking of an example where types would interfere, other than writing code which ignores things like checking if a result exists or not, or is a datastructure vs an error - and I would rather have the proper handling of those things done and enforced by the compiler rather than ignore it until something blows up in production. That's my take though, and for those who are desperate to get something in to production ASAP, it's probably not the right fit.

2: I'm not sure that working with arbitrary data leaves you in a situation where you are no longer within the realm of safety the compiler can provide you. You will have code that cares about aspects of that data, imposing some kind of logical structure to it, whether that's encoded in a type or not. The only difference between dynamic and static processing of that kind of data is that the dynamic code has to search for the structure it expects, while the static code defines that up front, and only has to try to match the data to that structure - working with it after that point is not going to differ much between the two. With dynamic code you'll still need to check if a field is present or not, and whether it's of the type you expect it to be. It is certainly easier to create arbitrary heterogeneous structures in a dynamic language (I won't argue that!), I don't buy the argument that a static type system is somehow ill-suited to the processing of such data.

I will say that it is annoying that before you can start writing code to process the data in a static type system, you have to define the types/interfaces that the data will take - sometimes I would rather just mold the raw datastructure via traversing it and pattern matching on it - but I think in the end that extra time up front ends up a wash with a dynamic type system because you spend less time on the tail end chasing bugs that the compiler can catch.

For Rich's argument to resonate with me, I think I just need a good example of what he's talking about - every time this topic has come up, I haven't been able to come up one, and I haven't seen anybody else do so either. An example where Rich (or whoever) thinks the dynamic solution is clearly superior, and no static solutions come close.

At the end of the day, I say use the right tool for the job, and if a dynamic language clearly solves a problem better than the static one, then hey, I'm not complaining (after all, it's why I'm using Elixir/Erlang to solve problems, though I don't think the problems they solve are necessarily at the dynamic/static divide).

1:

In a static type system with inference, both examples would just be

    (add [x y] (+ x y))

as the `+` operator is enough to infer that the parameters need to be a number. So your point is true for some statically typed languages, but certainly not all (ML-likes, Haskell-likes, and others with strong type inference).

2:

In my experience, ADTs (and pattern matching in particular) make coding problems much easier to solve (and the code itself considerably simpler) than the alternatives in many languages. For example, I'm writing a compiler in Go, for fun, and a lot of tasks would be far easier and more concise to express with ADTs/pattern matching. Another example is working with pattern matching in Elixir/Erlang, especially with binary data, which is beyond easy to express a parser for with pattern matching. Elixir and Erlang are dynamically typed, and have gradual typing via Dialyzer (though it's rather different from clojure.spec I believe), but pattern matching is pattern matching, and it's indispensable in my opinion.

Using something dynamic, like Lisp/Javascript/Ruby/Etc. would make it much more difficult to navigate the code for something like a non-trivial compiler (in my experience), and more difficult to reason about how it will execute. The other thing I don't see mentioned very often is that if the compiler can't reason well about your code, then it can't optimize it very well. Even with a JIT, you are paying a performance cost that is non-trivial, unless you happen to be using a language with a particularly good JIT, performing tasks that the JIT is good at optimizing at runtime. For a lot of tasks, this overhead doesn't matter, but I would rather lose some flexibility to be able to use the same language for a broader array of tasks.

Recently I was taking a stab at translating some code for an Earley parser from some dynamic languages (Javascript and Python) to Go, again for fun, and in both cases found it was impossible to do so because of the way the dynamic features of the language were abused - basically I would have to throw out everything and write it from scratch. The code was hard to reason about, because while reading it, you think you are working with values of a certain type in one function, but the same value somewhere else is getting treated like a different type. You basically have to execute the code to see what it does. Such flexibility can make for more concise code, but when the code _isn't_ concise, it can make it incredibly difficult to untangle. That's just my experience though. I'm an Elixir/Erlang programmer during the day, so I'm not anti dynamic languages by any means, but I have definitely found it easier to build certain types of applications in statically typed languages.

I don't know, I'm a fan of the name to be honest, I wouldn't change it just because of the possibility of accidental association.

My impression (as someone working on a compiler of my own, using LLVM), is that while you get a lot out of building on LLVM (portability across platforms, a collection of "free" optimizations, and a well defined low-level IR), it comes at the cost of a huge toolchain (takes about 5-10 minutes to compile from scratch on my laptop), and it is not stable across releases (the extent to which you are affected depends on the features your compiler depends on though). I think for toy projects it is probably more interesting to write the code gen yourself; and for projects focused on a specific platform, it is perhaps desirable to handle it yourself. Some people probably just don't want the dependency. My perspective is that I don't want to reinvent the work on cross platform support, or the optimization passes, and would rather take advantage of those efforts by depending on LLVM. I don't know that this makes sense for this project though.

I am! Thanks for the kind words! I don't hear from people using my stuff too often, so it's always good to get some positive feedback :)

This is great! Thanks :)

Mostly for my own edification/enjoyment. It's something I've wanted to do for a long time, and once you have that itch, it's just a matter of time. I've built some toy compilers before, but this project I'm aiming a lot higher to see if I can combine some recent research in type theory (algebraic subtyping as an extension of ML-style type inference, effects) and the bits of languages that I would really love to see meshed together (rich pattern matching, including pattern matching on binary data, macros, zero/low-cost abstractions, Erlang-style concurrency with actors/supervision/transparent distribution) and see if I can still find a sweet spot where the language is still approachable, has good tooling, etc. I've accumulated a lot of research and notes, but only recently started working on an actual implementation. At this point nothing is set in stone, since I'm not certain all of those things are actually compatible in a single language, but those are the goals I have in mind for now, so we'll see where it takes me.

As for why I chose Go, I wanted something which was pretty portable, with decent performance, easy to build command line tooling in, but at a slightly higher level of abstraction than C; that way I could conceivably port it to C or C++ relatively easily if I felt like that was a better way to go down the road, but could mostly focus on getting things done in the short term. I'm not a huge fan of the OCaml toolchain, even though I like the language, and it's not very approachable for other people if they wanted to pitch in. Go isn't an ideal language to build a compiler in, but it's not bad either. I'm comfortable with it though, and that's all that really matters for a project like this.

Yes, the compiler is written in Go, and implements a subset of OCaml, the language.

This is well timed! I've been starting work on my own strongly-typed functional programming language using Go + LLVM, and this looks like a great reference for implementing some of the parts I hadn't yet thought out. Though I'm not going the OCaml/MinCaml route, it's still extremely useful to see some of the key compiler stages implemented, since it helps give me a guideline for what has worked for others. Thanks for posting!

I suspect most people which state that C has a weak type system are really talking about the fact that C has a weakly _enforced_ type system. You can break the type system's rules rather easily (or perhaps it's more accurate to say, the type system is too permissive), either way it doesn't provide you the same guarantees a stronger type system provides). At least that's my take on it.

The weather here is beautiful a little more than half the year, but winter here is definitely a hard pill to swallow for those not used to it.

You may have been able to bump the process priority to give that process more scheduler time if it's a critical process in a hot path. Perhaps you tried that though. Definitely have to be careful with it, but always an option to keep in mind!

I think the discussion around deployment may have been unnecessarily tainted by their experience using edeliver - it's an automation layer for building and deploying releases, but as mentioned it is configured via shell scripts and because it does a lot, a lot can go wrong.

The basic unit of Elixir (and Erlang for that matter) deployments is the release. A release is just a tarball containing the bytecode of the application, configuration files, private data files, and some shell scripts for booting the application. Deployment is literally extracting the tarball to where you want the application deployed, and running `bin/myapp start` from the root of that folder which starts a daemon running the application. There is a `foreground` task as well which works well for running in containers.

My last Elixir gig prior to my current one used Docker + Kubernetes and almost all of our applications were Elixir, Erlang, or Go. It was extremely painless to use with releases, and our containers were tiny because the release package contained everything it needed to run, so the OS basically just needed a shell, and the shared libraries needed by the runtime (e.g. crypto).

My current job, we're deploying a release via RPM, and again, releases play really nicely with packaging in this way, particularly since the boot script which comes with the release takes care of the major tasks (start, stop, restart, upgrade/downgrade).

There are pain points with releases, but once you are aware of them (and they are pretty clearly documented), it's not really something which affects you. For example, if you bundle the Erlang runtime system (ERTS) in a release, you must deploy to the same OS/architecture as the machine you built the release on, and that machine needs to have all of the shared libraries installed which ERTS will need. If you don't bundle ERTS, but use one installed on the target machine, it must be the same version used to compile your application, because the compiled bytecode is shipped in the release. Those two issues can definitely catch you if you just wing a deployment, but they are documented clearly to help prevent that.

In short, if there was pain experienced, I think it may have been due to the particular tool they used - I don't think deployment in Elixir is difficult, outdated, or painful, but you do have to understand the tools you are using and how to take advantage of them, and I'm not sure that's different from any other language really.

Disclaimer: I'm the creator/maintainer of Distillery, the underlying release management tooling for Elixir, so I am obviously biased, but I also suspect I have more experience deploying Elixir applications than a lot of people, so hopefully it's a wash and I can be objective enough to chime in here.

Elixir does an amazing job of showing you why a test failed, but as for debugging, up until the latest release, 1.5, there wasn't really a debugger, it had pry, which lets you play around in the environment at the point where pry is called, but you can't step though code. 1.5 introduced a more useful debugging experience, but you can't step line by line, you can only step from breakpoint to breakpoint as far as I'm aware, but I haven't played much with it yet. It's getting there though! I should note that I haven't really lamented the lack of a good debugger, and I came from .NET where I had one of the best debuggers available.

I hope you don't mind if I just copy my reply to Hates_: "I don't yet, as I've been pretty heads down. I'll put one together this week and reply back once it's up."

I'll make sure to write back to you both :)

I don't yet, as I've been pretty heads down. I'll put one together this week and reply back once it's up.

I'm actually working on a book which I think covers a lot of what you're looking for (and was what I wanted back when I got started with Elixir a few years ago). I'm covering architectural patterns for the items you mentioned as well as flow control and digging in deep on how to really leverage the fault isolation mechanisms of OTP. I'm the maintainer of Distillery, the release tooling in Elixir, so I'm also covering a lot of the gritty details involved there, such as how to work with hot upgrades and such, as well as dealing with issues in production using some of the excellent operational tooling available.

Unfortunately I probably won't be done until the end of this month, but if you're interested, keep an eye out on ElixirForum.com, since I'll announce it there when it's available.

Just want to point out that Records are still part of the standard library, and can be important for integrating with Erlang code using records which you need to work with. Even though they are avoided in Elixir, because structs are better in every way, they are still alive and well in Erlang.

> CoreOS is an operating that can only run Docker and is exclusively intended to run Docker.

This shows an astonishing level of ignorance for someone who claims to have done their research.

> First, the main benefit of Docker is to unify dev and production. Having a separate OS in production only for containers totally ruins this point.

What? This makes no sense. Your images will be the same between dev and prod, even if the host running the containers is different - which is really the whole point - if you build and run an image in dev, it should run identically in prod.

> If you like playing with fire, it looks like that’s the OS of choice.

We spent the last year+ running containers on Centos7 with no problems from the OS. Whatever issues we did encounter were either transient bugs with Docker or our own configuration. Perhaps we got super lucky, but we were running 120+ containers on 12 hosts, so I would've expected at least some evidence of significant problems within that timeframe if it were really such a risky setup.

> It’s not possible to build a stable product on a broken core, yet both Pivotal and RedHat are trying.

We've been running OpenShift Origin since March of last year, it's been very stable during that time - the few issues we did encounter were due to our own mistakes, and were usually fixed just by changing some configuration and restarting the host.

While there are undoubtedly problems with Docker, and likely many of the issues you brought up are very real, there are many teams like mine that use it successfully, and painlessly. Docker isn't the tire fire you want to make it out to be.

I think what he meant is that from a programmer's point of view, Erlang appears to be preemptive (there is no explicit yielding of control, ala golang or lua, the two I can think of off the top of my head), even if the scheduler itself is cooperative. The place where this most obviously reveals itself is with NIFs, where the scheduler can no longer preempt the process, and is blocked until the NIF finishes executing. At least that's my understanding.

As a long-time C# dev, now full-time Erlang/Elixir dev, I've written things like this before myself when attempting to bring some of the FP tools I loved back to C# for my own benefit - so this looks really familiar in that sense. This is by no means an actual competitor to Erlang's actors - for one thing it lacks any form of supervision, which is really the most important aspect of Erlang's actor model (i.e. the "let it crash" philosophy). There are other things missing as well (such as the OTP behaviours, e.g. gen_server, provided by Erlang).

That said, this looks like something I'd definitely use, or at least experiment with in C# if I was still doing work in the language - though for the actor bits I would probably lean on Orleans.

My understanding of Crystal is limited, but I think of the two, Go is the only one you would probably consider comparing it to, because of its native compilation. I don't think it has a very strong emphasis on concurrency, so comparing it to Elixir likely doesn't make much sense. Take that with a grain of salt though, as I'm not super familiar with the language.

Some libraries contain native dependencies, and those have to be cross-compiled. You also need a cross-compiled runtime if you are bundling it and not using one already installed on the target.

You actually don't have to bundle the runtime, but you can also specify an "alternate ERTS", i.e. one that has been compiled for the target architecture, and bundle that one instead. This is how you "cross-compile" releases.

Deploying releases has been fairly painless in my experience, but I also have had my head in Elixir and releases in general for a long time, so I may not be a great sample. You get a tarball containing everything the app needs to run, and all you do is extract it and run the boot script - other than entirely self-contained binaries like Go produces, I don't think it gets much better. Deploying without releases is probably a lot more painful though, due to the number of dependencies required, particularly web applications (Erlang, Elixir and Node). I recommend anyone deploying Elixir applications look into releases - it makes life much easier (imo).