Comment by hnedeotes | Hacker News Reader

Comment by hnedeotes | Hacker News Reader

Parent Full thread

> You know what's an even worse cognitive overhead? Languages without strong typing, like Elixir.

I see this mentioned many times, but when I do my impression (which might be wrong) is that you haven't written Elixir or Erlang for any thing more serious than tutorials or docs examples. Besides, elixir is technically (?) strongly typed - but not statically typed. Regarding async/await, compared to the things you get in the BEAM - that part specially is like comparing a stone wheel car moved by oxen with a space shuttle that has the same costs (or less), to launch and operate. It might sound snobbish, but sincerely I don't know what to say when I read that.

Although not compile time, pattern matching allows you to define as strict, and in some cases, stricter conditions for functions than many typed languages, without any additional cognitive overhead or complex type definitions that you need a PhD to understand.

def do_it(<<a::binary-size(2), ":", b::binary-size(4)>>), do: IO.puts("the argument is a binary string, of the form xx:xxxx")

Even conditions where it depends on more than the types, like with complex types such as maps, structs, tuples, lists, where parts of those conform to a certain pattern, in any combination needed.

I've been using Elixir for five years and I think it's sorely missing a better type system. I've done fine without it and have gotten entirely used to it but that doesn't mean things wouldn't be better with one.

I only really started noticing as I started to build more complex Typescript systems how much it helps. My teammate shipped a bug in Elixir yesterday that would have been caught by a static type checker

I know there's plenty of people working with Elixir (and Erlang) that feel that way. I can also imagine that in very large codebases dialyser might not be enough and it would be great to have it. Also that obviously, outside web and telecom there's definitively software which I wouldn't want written in them, but not in TS either, and probably it would have more tests than code, property and fuzzy and all the remaining things.

I also won't say that it doesn't happen but I do believe if the same effort that goes into writing typed typescript and tests, aka, using dialyser and writing tests, that there shouldn't really be any difference? And when you include things like guards, explicit pattern matching, casting your data at the entry points of your system that it provides a much more robust experience?

But that's all optional - what I've seen is that theoretically there's nothing you can't express with the base syntax (+ecto) that you could with most (as in the common ones) type systems, perhaps the difference is that since in a typed language it won't compile you are forced to do those things, define all enums, all their variants, exhaustively match, etc?

Green threads are great, right up until the moment you have to interoperate with something written in another language / using another VM. Then it's a mess (see also: Go).

The nice thing about async/await is that, because it's all just a bunch of syntactic sugar over callbacks, any language that supports some kind of callbacks with state, can be mapped to async/await - even C! On WinRT, for example, such interop works across C++, C#, and JavaScript.

This is largely why languages that push this model tend to form rather tightly closed ecosystems, IMO. Which, in turn, leads to their lower adoption - and thus, no particular implementation of green threads can become a de facto standard.

I think I might be missing what you mean? You can write an async/await implementation from scratch in a few dozen lines of elixir if wanted, but you also have Tasks that provide that already written for you (with support for being included into supervision trees, etc)?

How does JS interoperate with something outside V8?

A JS library that wants to interop with something outside of its runtime (which is not necessary V8 - it isn't in WinRT, for example) can do so through FFI facilities. So long as said FFI supports all the same things that C does, it supports callbacks via function pointers. And if you can pass a function pointer + data pointer to some API, you have a stateful callback - i.e. a promise/future/task. And you can map any such abstraction to the same in another language.

But the moment you introduce green threads, the caller and the callee both have to be aware of that particular implementation of green threads. So even if you can do C FFI, and whatever you're trying to call can also do C FFI, you can't interop async calls across that boundary without some kind of callback arrangement.

Here I'm a bit out of my depth.

I think with Erlang you don't need to know >that particular implementation of green threads, but you need to implement the required specifics of the VM FFI (which in practical terms might be what you meant) - but this has a reason, in that, for the VM to provide its guarantees it needs to be able to count "cycles", refs, etc in order to preempt the execution of any function/process at given points, so that no single process can block the scheduler. It also needs to transform things from and into data types usable inside the VM.

I think you can also use "dirty" nifs, but here you might crash the VM so you really need to know what you're doing (and you still have to translate things to and from the vm).

And there's also ports, and C Nodes. These have a higher sync cost but can be treated by the VM as if they were processes/erlang nodes.

Lastly, you can also just shell from the VM to an OS level process, or use sockets, etc.

But I doubt this is anything 99% of the people writing JS do in JS? One of the things people using Elixir or Erlang is exactly because of the guarantees and programming model of the VM.

Again, might be missing what you mention because it's not an area I have explored in any meaningful way.

That's the thing - it works for Erlang/Elixir, because it tends to live in its own ecosystem with its own libraries etc. If you are working on something where there's an existing large ecosystem in, say, C++, you'll be reinventing the wheel. Or jumping through hoops with a multiprocess implementation (pipes, sockets etc), and having fun synchronizing those.

Now, not all projects are like that - but reusing libraries from other languages is common enough in large projects. Thus, languages that can't accommodate that use case, don't become truly mainstream. Within their niche, they can be much more pleasant to work with, though. So I don't think there's anything wrong with Elixir per se, and for some tasks, it makes perfect sense - but it's not a very general-purpose tool.

Note, by the way, that I'm not talking just (or even mostly!) about JS, but rather about async/await in general - e.g. also in C#, where that syntax originated, or these days in C++20. On Windows, if you write "modern" (UWP) apps, regardless of the language used, they make a lot of async API calls for stuff like UI - and the implementation is all in native code, running in the same process as your app.

Yeah, it kinda limits the scope of adoption/interest in a way and I think everybody working with it would like that it wasn't the case but I don't think that gap can be worked out easily (or in a practical way) because without the code being written to accommodate the requirements of the VM it can't do what it's meant to do.

If you link a piece of code that can crash the whole VM or steal the processors schedulers then the value in writing supervision trees, restart strategies, compartmentalising your runtime concerns into processes, plus the tradeoffs made in the vm/language design themselves go down, because a single invocation can throw it all out of the window.

(and note, this is not to say the "outside" code is of less quality or anything, is just that when writing "inside" the vm, if you don't account for an unknown problem that happens only sometimes but place it under a proper chain of supervision (and this is much easier to do than writing 100% bug-free I've covered all cases including heinsenbugs code), it will be contained and not bring down the entire VM along with everything it was doing at the time, like all open client sockets or work it was doing, or impact other users when it happens)

Elixir/Erlang has some of the most mature ways of interoperating with external code of any language.

It's literally what Erlang was invented for 30 years ago, building telephone switches where the main logic was on the Erlang VM, interfacing with logic in C which interfaces with the switch hardware.

There are multiple ways of doing it, each with different tradeoffs of safety and performance. You can write FFI code using NIFs, but the Erlang culture of reliability frowns upon it unless you have real performance needs. A good example would be doing cryptography. If the FFI code needs to handle concurrency itself, it can, communicating with Erlang/Elixir via messages. You don't need to care about the threading model. But that's generally not a great architecture. You should rely on the VM's processes to handle concurrency and make the FFI code just be libraries.

It's very common to write performance critical code in a compiled language such as C++ and talk to it over a "port", which spawns an OS process and communicates with it over stdin/stdout. An example is high frequency trading, where Erlang is used to supervise low level code.

You can also write a "C node" which allows a standalone program written in C or Java to talk the native Erlang network distribution protocol. So you just send messages to it.

This kind of thing is very common in Elixir embedded systems, and a joy to work with. See https://www.cogini.com/blog/elixir-and-embedded-programming-...

> Green threads are great, right up until the moment you have to interoperate with something written in another language / using another VM

You mean like this? It's not a problem in elixir.

https://www.youtube.com/watch?v=l848TOmI6LI

I don't know why green thread based languages do not export a pooling interface on their FFI.

Pooling much nicer interface to implement than callbacks, and a quite easy concept to plug one interface into another.

I'm not sure I follow. Can you explain how two runtimes, each with its own green thread implementation, would interoperate in this manner to make a call across the boundary? Say, X (written in A) wants to asynchronously call Y (written in B) and wait on it, passing it Z (also written in A), which Y needs to call several times to perform its task.

X does not wait on Y. It starts the task, and adds the polling interface for the task into its internal polling list. When something is available, calls the continue interface of Y as many times as it needs, just as it does with the OS data polling.

The same happens for Y calling Z.

By "wait" I mean logical wait for the result of the async operation - e.g. to use it as an input for another operation - not a blocking thread wait. What you describe sounds like callbacks to me?

Polling loops are completely different from callbacks.

What I described uses functions to read partial or complete results, and functions that indicate if there is any result to read (and yeah, there must be some blocking thing to indicate whether there is any result at all available). It's something much closer to async/await, except that there is something to tell you if await will block.

It is much more complex than callbacks, but it enables much higher parallelism and somewhat higher single-thread throughput. Also, runtimes with green threads already have it implemented anyway, because it's the preferred OS (any high performance OS) interface for async.

Async/await is just syntactic sugar over callback-based continuations, though.

And in .NET at least, you can easily tell if await will block or not, by looking at Task.IsCompleted.

Reply on news.ycombinator.com