Go: Process isolation rediscovered as a programming pattern | Hacker News Reader

Go: Process isolation rediscovered as a programming pattern

popalg.org

30 pointspetar31 comments

I think providing a language construct to help you do something badly and covering up your mistakes by rebooting is a step in the wrong direction.

Surely, you can debug your programs, but catching slow leaks in complex software is hard, even in Go.

I don't think this warrants restarting your goroutines. Go is still a young language, who knows what tools for debugging will come along in the future.

It depends what you're building. If what you're building is something which should never fall over and die, and you can afford to (because the interesting applicative state is stored elsewhere) kill-and-restart is a perfectly valid strategy.

It is in fact one of the standard strategies in erlang supervision trees: a subtree crashed, log the signal somehow and spawn a new subtree. No point in interrupting the service if you don't absolutely have to.

If you have memory leaks in a subtree, you can just kill it and let things recover in a standard manner, while you're investigating the cause of the leak (or not, you may have more important stuff to do)

> Go is still a young language, who knows what tools for debugging will come along in the future.

While that's true, Go has not exactly been at the forefront of language innovation so far and I've yet to see truly good tools for memleak investigation and debugging in more innovative languages...

I think providing a language construct to help you do something badly and covering up your mistakes by rebooting is a step in the wrong direction.

That's a huge part of the Erlang philosophy.

Are the concepts described in the article the same as Erlang or Erlang/OTP, or is there more to it?

I'm not very knowledgeable about either Erlang or Go. But from what I read, Erlang doesn't share objects between threads whereas Go does. This is a serious difference, because as the article mention you have to decide what to do if there are references from other threads. That's why the parallel with processes breaks in the case of Go. Cleaning up the resources of a share nothing (or little, and in a centralized place, see shared mem) process is easy and fast. Cleaning up the resources of a share most/everything thread... not so much.

But that's the point is it not? TFA's building patterns to try to implement in Go some of the most basic facilities of Erlang, and try to work around Go's misfeatures (such as default shared memory).

So the concepts described in TFA do indeed already exist in Erlang.

Indeed. Erlang has this feature, I didn't intend to imply otherwise. What I meant was that this mechanism would be more difficult to implement in Go.

> What I meant was that this mechanism would be more difficult to implement in Go.

Well of course, since it needs to be implemented in the first place whereas Erlang provides it as part of its core features.

> Are the concepts described in the article the same as Erlang/OTP, or is there more to it?

seems to be identical to what already exists in Erlang. it is pretty easy, imho, to have an appropriately defined 'terminate' in a 'receive' clause, of an erlang process. on receipt of the said message, erlang process can exit out of the event-processing loop, thereby terminating the process.

not sure why author thinks this is not present in Erlang.

this is part of the core language, and has nothing to do with OTP, which is a framework built on top of the language.

edit-1 : clarified OTP's role (or lack of it).

Not OTP, just Erlang processes (memory isolation) and process links (an apoptosis signal, though here it seems pretty limited, to killing a supervisor only whereas Erlang can propagate death signal quite arbitrarily, including from children to supervisors or to unrelated processes without the dying process having any knowledge of the link).

I don't understand why TFA says this does not exist anywhere else (let alone suggest it could be implemented in Erlang when erlang already goes quite far beyond that)

The author seems to have rediscovered the advantages of managed operating systems.

Here's a relevant article for those not familiar with the concept:

http://research.microsoft.com/apps/pubs/?id=69431

Why "managed"? Doesn't process isolation also works in normal operating systems?

I think the GP is getting at the author's discussions of the limitations of GC. In some managed OSes that have come out, GCs are built in. Thus, file and socket handles would be garbage collected.

But the author's proposal seem to be opposite to this: as he points out, it's what current "normal" OSes do: having a command to terminate processes (and thus, collect handles).

Cool stuff. I think it's similar to custodians in Racket (the artist formerly known as PLT Scheme):

http://docs.racket-lang.org/reference/eval-model.html#(part....

If the problem looks like memory management, then perhaps something that looks like garbage collection is the answer?

Of course, it doesn't map trivially, since there are no existing references to goroutines. But, it may possible to determine if a goroutine cannot communicate with anyone - it is garbage - if there is no one listening to any of its channels. The one exception to this rule I can think of are daemon-style goroutines that would interact with the system through external calls. Such goroutines could be labeled as such at the creation site, indicating that the goroutine manager should not manage them.

Discussion on go-nuts mailing list: https://groups.google.com/d/msg/golang-nuts/ar3QSMI4ooE/9LGK...

I see what the author is getting at, but I don't see the reason for a language to provide this isolation. Why not just use an OS-level process? Why not create a library or framework to do this?

> Why not just use an OS-level process?

OS-level processes are very expensive (and the OS themselves may set quite harsh limits on e.g. the number of processes you can have[0]) leading to lower flexibility.

Furthermore, process-based IPC tends to be untyped (unless you're willing to pay for the price of serialization and deserialization of higher-level structures).

Finally, the ability to link processes together and react to events (mostly death) of unrelated processes (no SIGCHLD and no SIGHUP) are limited.

[0] it defaults to 532 on a non-server Snow Leopard system, for instance...

nptl (read, any modern kernel) Linux processes are cheap, they share an implementation with threads in fact. Note that this only means they can be quite cheap, if you dirty a large amount of memory you lose CoW of course.

> nptl (read, any modern kernel) Linux processes are cheap

They're not 300-bytes cheap. They're not you-can-have-a-million-of-them-on-a-desktop-box cheap.

Erlang's processes are.

The other commenter has a good answer and I will re-iterate (I'm the author): Sure you can everything the old school way. The point is that being able to replace processes with in-program constructs is both less expensive resource-wise (memory, OS engagement, etc.) as well as more convenient to program (and thus making the programmer more productive). Language design questions are very much about how to make the programming process more productive and less error prone. So the focus should be on convenience. Of course, you can do everything the old-fashioned way (but that's not the point).

Language design questions are also about how to not make languages complex and add too many features to them. I'd argue that creating or modifying a language should be a last resort.

After all, there's nothing to stop you from implementing this in Python right now with thread-local storage, and I'm sure you could achieve the same effect with other languages if you don't like the idea of the GIL or slow interpreted languages.

I think it's a good idea. Slow resource leaks are very hard to debug. Sometime it's easier and cheaper to just restart the process. I seem to remember some fighter jet subsystems can do fast reboot. When they crash, it can be restarted in a faction of a second. If the rebooting approach works in a realtime environment like that, it sure would work in the day-to-day environment.

My first reaction is that this is a terrible idea.

Very often you create a goroutine to send messages down channels. Those messages are objects. Those objects may go to multiple other goroutines.

Kill the originating goroutine and you've just messed up all of the other goroutines that thought that they had objects.

The runtime, or the supporting library would have to take this into account. The messages would have to be in shared memory and would in any case have to be managed.

I can see how this might be used for certain specific things. Because of this specificity, a library would be more appropriate than adding it to the language, though.

Basically, mutability (including being deleted when other processes didn't expect) and message-passing turn out to go poorly together. It isn't impossible to jam them together with enough work, but you end up with a wildly more complicated system in the end... if you end up with a coherent "system" at all.

Erm... wouldn't the GC take care of that and keep the objects alive as long as they're in the channels, or something?

He's talking about an extension that lets goroutines be preemptively killed, and their memory freed. The mechanism he discussed would make an end run around regular GC. My point is that it wouldn't play well with it.

Seems loosely similar to how I'm using .Net4's Tasks from the Task Parallel Library. I didn't particularly focus on the effects of the garbage collector when I came up with my approach, but it does at least achieve point A in his opening paragraph ((a) is simpler to use)

Getting this article in text/plain.

Reply on news.ycombinator.com