Three ways to think about Go channels
dolthub.com
dolthub.com
At least old-school syntax like “GOTO foobar_step_23” and “LABEL foobar_step_23” is grepable).
I greatly prefer programs that “color” functions sync/async, and that use small state machines to coordinate shared state only when necessary.
Go technically supports this, but it doesn’t seem like it is idiomatic (unlike rust, the go compiler won’t help with data races, and unlike C++, people don’t assume that the language is a giant foot-gun).
The parent is really complaining about idiomatic Go. Which is fair, because idiomatic Go feels like writing Java 6 (that is to say, making lots of compromises due to limitations of the language).
No. An exception is a datatype. Not completely unlike an error type in concept, but intended for erroneous conditions that could have theoretically been caught at compile time (i.e. the programmer screwed up), as opposed to conditions that are not decidable at compile time (i.e. something happened in the outside world).
In Go, panic creates an exception, which may be the source of your confusion. The exception contains any metadata you passed to panic along with other data, like a stack trace. This is not panic in and of itself, though.
> but when writing idiomatic Go one would prefer error types over using panic
Not really. Idiomatic Go says that errors are in no way special and is faulty to think of them as being special. They are just values like any other. Go exceptions allow attaching any value as metadata.
The official line is that panic stack traversal should not cross package boundaries, but internal use is perfectly acceptable. Even the Go standard library does it.
Why is it that Go attract so many "experts" who have clearly never used the language even just once?
You might impose it upon yourself when you start to understand the pitfalls of using exceptional handlers for anything other than exceptions, but such is engineering. Everything comes with tradeoffs.
Within a package it can work, but then you'll find language support lacking. For example defer is scoped to a function so you need to write pretty unusual code to make the equivalent of a try block.
Go _doesn't_ have any form of checked exceptions, though, which are required by the compiler to be caught by the caller.
Of course, Go code uses panics much more sparingly than Java or C# or JS or Python use exceptions.
func outer() {
defer catch() {
if r := recover(); r != nil {
fmt.Println("Recovered:", r)
}
}()
// ...
doSomething() // Can panic
doSomethingElse() // Can also panic
}
const outer = () => {
try { doSomething() } catch(err) { console.log(err) }
try { doSomethingElse() } catch(err) {
console.error("dang it bobby", err)
throw err
}
}
func outer() {
r, err := doSomething()
if err != nil {
fmt.Println("Recovered:", r)
}
r, err := doSomethingElse()
if err != nil {
fmt.Println("Recovered else:", r)
}
}Why not?
func outer() {
func() {
defer catch() {
if r := recover(); r != nil {
fmt.Println("Recovered from doSomething:", r)
}
}()
doSomething() // Can panic
}()
func() {
defer catch() {
if r := recover(); r != nil {
fmt.Println("Recovered from doSomethingElse:", r)
}
}()
doSomethingElse() // Can also panic
}()
}
Hell, if you long for try/catch for some reason, you can even get creative:
func try(fn func(), catch func(any)) {
defer func() {
if r := recover(); r != nil {
catch(r)
}
}()
fn()
}
func outer() {
try(func() {
doSomething()
}, func(r any) {
fmt.Println(r)
})
try(func() {
doSomethingElse()
}, func(r any) {
fmt.Println(r)
})
}
The recover is still function-wide in your example, which is what I said. You can nest funcs just to deal with this, but it's ugly, and code reviewers won't like it. I use Go on my team, and yeah I'm not as expert as the team who created Go here, but idk why you keep saying I've never used it. My complaint about "if err" is pretty common among Go users, who would understand the joke of calling it "Errlang."
Yes, it suffers all the same problems. And then doesn't even help you with the errors once you get them! You then have to resort to this kind of craziness to do something with the error:
try {
doSomething()
} catch(error) {
if (error instanceof FooError) {
console.log('foo')
} else if (error instanceof BarError) {
console.log('bar')
} else if (error instanceof BazError) {
console.log('baz')
} else {
console.log('unknown')
}
}
err := doSomething()
switch {
case errors.Is(err, ErrFoo):
fmt.Println("foo")
case errors.Is(err, ErrBar):
fmt.Println("bar")
case errors.Is(err, ErrBaz):
fmt.Println("baz")
default:
fmt.Println("unknown")
}
But I get the impression that those who find benefit in passing errors using exception handlers don't handle errors. If you don't have to worry about errors in the code you write, then I think there is a good case to be made that errors as exceptions is a better approach.
That is, after all, the difference between scripts and systems. Scripts can simply fail, leaving the user to try again. Errors as exceptions, or something in the same vein, make sense as the predominant mechanism in scripting languages. Systems, on the other, have to deal with failure. You don't get to just bubble it up and let the user deal with it. This is where errors as exceptions becomes a nightmare. Go is unabashedly a systems language. It is not meant to be a scripting language.
Different tools for different jobs.
They aren't different jobs, though. The two most common uses of JS are backend systems (like you'd often use Golang for) and web frontends, not scripts. Backends will usually catch errors in one place, an HTTP or similar handler, sending back the appropriate status code with maybe a payload. Golang backends do something similar, which is why you see so much "if err != nil... return err" in practice.
Python is more for scripts aka CLIs, but Python backends are fairly common too. And Golang CLIs are also common.
But at least we have a rule checking that you actually did something with the return statuses.
If you want to use a Result/Optional type I think that is way better than exceptions. As long as the compiler can enforce that you are acknowledging (or explicitly ignoring) both the happy and error cases. Ideally the error case for the Result/Optional type would also have an attached stack trace.
So, if I'm understanding you correctly:
An exception is an unexpected failure at runtime that could have been caught by the programmer. For example, you might have a switch block with a default block saying "this can never happen". The programmer might miss a case in the switch block leading to an exception. This would be analogous to an unchecked exception (RuntimeException) in Java -- these exceptions are not required to be caught.
In Go, `panic` creates an instance of the type `exception`. When `panic` is executed it attaches metadata like the stack trace to the created exception. When I say "exception" this is the behavior that most people think about.
An error is for an expected failure, like a file potentially not existing when performing I/O operations. Like you said (and this matches my understanding), error types are just a normal type -- really any type that implements the `Error` interface. There are a lot of utilities functions/libraries that act on that `Error` interface, like assertions for unit tests, the multierr library, etc.
`Error` is similar to a checked exception (Exception) in Java, though it misses the behavior you expect like requiring it to be caught, and the try-catch syntax. Additionally, you don't have the stack trace/metadata automatically added.
Go programmers seem to _heavily_ prefer errors over exceptions, even when they should be using `panic`.
My problem with errors in Go are that they are too easy to ignore accidentally and they don't contain stack traces, so they can be hard to track down. I inherited a codebase where we had _hundreds_ of unchecked errors being silently ignored.
I understand that these problems go away if you have linters or a very detail-oriented team, but I unfortunately have no power over the actions of my team before I join. It's also _very_ hard to convince a team that they've been doing things incorrectly by ignoring errors.
This is a problem for values of all types, not just errors. One I'm not sure we've figured out how to solve[1]. There are a few languages out there that force variable assignment to try and address the problem, but even then there is really nothing to say that you haven't accidentally ignored the variable assigned.
> It's also _very_ hard to convince a team that they've been doing things incorrectly by ignoring errors.
To be fair, if you are able to completely leave out entire blocks of logic without anyone noticing even under the most cursory of testing, perhaps it wasn't actually needed? Forgetting entire code branches isn't exactly a subtle bug.
[1] Short of going all the way to formal proofs.
But even assuming it is done sometimes, is the developer going to actually handle it? I don't know how many times I've come across "catch" blocks that are empty or something equally inappropriate. Nothing was gained. It turns out that programmers will still forget no matter how hard you try to hold their hand.
I'm not sure there is any other solution than to test it, and once you get into testing, entire blocks of logic missing are going to stick out like a sore thumb. Forgetting an entire branch is not exactly a subtle bug. At that point it really doesn't matter what language constructs you do or don't have available.
You are right, but the difference is that the programmer is making the choice to ignore that error versus accidentally ignoring the error.
And fair enough. It would be just as easy to forget to do that as it would be to forget to handle errors in any other language. There is no silver bullet here.
The difference is that the happy case is well-tested. If you aren't handled the error-free route you probably will notice that very quickly unless you aren't testing your code, even manually.
e.g. it doesn't matter if you ignore normal variable assignments because the programmer will usually catch that themselves. They will not likely catch all of the possible error assignments without some help.
> To be fair, if you are able to completely leave out entire blocks of logic without anyone noticing even under the most cursory of testing, perhaps it wasn't actually needed? Forgetting entire code branches isn't exactly a subtle bug.
In this case we were silently ignoring errors which caused multiple types of issues that we had to then manually track down. This is an entire class of defects that can be avoided with better tooling, either at the language level with checked exceptions, or with a linter like errcheck.
I think exceptions are pretty much a dead letter in new language designs at this point? You could get people on board for real algebraic types and matching.
Exceptions are a smaller deal than async/await. Basically instead of manually writing "if err" everywhere, it's implicit if you aren't catching it. The most popular languages do this. Rust "?" syntax is also a decent compromise.
We don't need to have one language to rule them all. Its fine for languages to make different choices and trade-offs.
It's a weird cult-like thing.
And go is reaching the point where like Java it gets crammed down your throat. Which is probably why golang should start adding even more features
It took me quite a while (years) to go from thinking that returning error (conventionally as the last return value) was a bit clunky to thinking that this is actually a very good idea. I don't mind the if err != nil pattern. I can't think of a single experienced Go programmer that is sufficiently annoyed by this to complain. I sometimes see people new to the language complain about this, but it tends to fade as they adapt.
With the ability to join errors, do sensible comparisons etc, errors work well for me when needing to express high level errors and their low level root causes. (eg the high level error may be "failed to read configuration" and the root cause might be "permission denied when opening file X").
I've come to realize that I like Go style errors better than exceptions for two reasons. First off, a lot of code in, for instance, Java uses unchecked exceptions. Which means that the whole idea that you have to handle exceptions goes out the window.
This, of course, varies from codebase to codebase, but the it is common enough that having exceptions available in a language doesn't mean people will use them in a way that forces discipline. Which kind of makes exceptions pointless. They just become syntax for emitting and handling errors. And awkward syntax at that. Which is the second reason. Exceptions tend to introduce extra scopes you have to deal with. This is annoying and in some cases it can get somewhat complicated.
Exceptions, the way they are commonly used, are a weak enforcement mechanism made more dangerous by people pretending it isn't. If you want discipline, neither exceptions nor Go errors are the way to go. You probably want to look to something like Rust.
I have a hard time seeing how exceptions would make Go better. Yes, if someone could come up with some syntactic sugar for Go 2 to deal with errors, that would be nice, but I have seen no proposals for this that represent actual improvement.
`if err != nil` is actually extremely readable and to justify new syntax for expressing this it would have to be a _lot_ better. I'm open to listening if people can come up with something.
Exceptions didn't end up doing what people hoped it would do in Java. We should learn from that and try not to pointlessly repeat this mistake. Besides, Go has already made its choice. Retrofitting it is only going to do harm.
Also the link to SliceTricks goes to stale content
All parallel execution is asynchronous, but not all asynchronous execution is parallel.
async/await interleaves execution in a cooperative fashion: a "fiber" runs until it yields by doing something blocking (such as blocking I/O, the most often cited use case). There are no data races and no need to use synchronization primitives since there is never a case where 2 fibers modify a piece of data at the same time: because execution is interleaved, the operations are always ordered one after the other. Whenever a fiber blocks, execution is returned to the scheduler, which resumes execution in one of the other fibers which were blocked.
fork(pthread_create)/join(pthread_join) parallelizes execution: a "thread" runs until it is done or is killed externally (the scheduler can also pause execution). A thread X can wait for another thread Y to be done by calling join(Y). There is a need to use atomics or other synchronization primitives such as channels (or mutexes, or barriers, or semaphores) when 2 threads share a variable in order to make sure only 1 at a time modifies, because although threads can be ran in an interleaving fashion (and sometimes they are interrupted (pre-empted)), they are usually ran in parallel on different CPU cores/threads (if you have a SMT CPU, which most nowadays are: n CPU cores and each CPU core has 2 "CPU threads").
P.S. A CPU thread != OS thread
> async/await (as most commonly implemented)
Javascript is famously single threaded, which is probably the runtime on which most async/await code runs today.
But yes, Rust for instance has the Tokio async runtime which is parallel, although it also has cooperative async runtimes.
For those who are reading this and feeling unsure: Go has parallel execution, and it calls `fork()` `go`. Go doesn't feature a `join()`, for that you must use channels:
func main() {
done := make(chan bool)
go func() {
fmt.Println("goroutine running")
time.Sleep(2 * time.Second) // simulate some work
done <- true
}()
<-done
fmt.Println("goroutine finished")
}
e.g. in C#, the following is true:
using var http = new HttpClient() {
BaseAddress = new("https://news.ycombinator.com/")
};
// Runs sequentially
var page1 = await http.GetStringAsync("?p=1");
var page2 = await http.GetStringAsync("?p=2");
// Submitted sequentially, everything after the first yield completes in parallel, IO does not block the caller
var page3 = http.GetStringAsync("?p=3");
var page4 = http.GetStringAsync("?p=4");
// Waits for page3, then page4 to complete, they are effectively parallel
DoSomething(page1, page2, await page3, await page4);
But later you mentioned threads running in parallel on different CPU cores.
The way I always understood it, is that this is the only real form of parallelism able to happen on a CPU. But that does seem kind of incompatible with your statement at the beginning.
Individual CPU cores are parallel because they are executing multiple instructions simultaneously (even though they are from the same instruction stream).
The other way around. Alternating two executions on one CPU core is concurrency, but not parallelism.
As sibling notes, you have it exactly backwards. It should read "All parallelism is concurrency, but not all concurrency is parallelism." I thought maybe this was a typo, but then you said:
> If you just think of concurrency as an instance of parallelism, you start to realize there are many ways to achieve concurrency, and thread-safe queues are one of the better ways to do it.
This is demonstrating parallelism as an instance of concurrency, not the other way around. A thread-safe queue can be processed by multiple tasks running interleaved in a single thread in which case it is executing concurrently, but not in parallel.
> Individual CPU cores are parallel because they are executing multiple instructions simultaneously
This is an implementation detail of the CPU. Not all CPUs execute instructions in parallel.
Unlike GOTO, there isn't really a better alternative for synchronizing parallel programs. Parallel programming (or even concurrent programming, for that matter) is just harder than sequential programming, and if you don't know what you're doing (regardless of whether you use channels or not) you're going to end up with a mess. Channels help to tame parallel programs, but they can't compensate for programmers who don't know how to write correct parallel programs.
And while Rust can prevent against data races, data races are a tiny sliver of parallel programming bugs. Far more common are race conditions and deadlocks, against which Rust is powerless.
> I greatly prefer programs that “color” functions sync/async, and that use small state machines to coordinate shared state only when necessary.
Async/await similarly doesn't solve the problem of undisciplined programmers making a mess. If you give undisciplined programmers async/await and shared state, they'll make a mess as easily as they will with goroutines and channels. If you're hiring people who can't be trusted with shared memory parallelism, then you have to take away parallelism or mutability or you have to train them to write correct parallel programs.
Notes On Structured Concurrency goes into some depth on this comparison:
https://vorpus.org/blog/notes-on-structured-concurrency-or-g...
If channel sends are being used as invocations, then this description can apply just as well to channels as it does to the go statement.
Yeah, this is why we use `sync.WaitGroup`s and similar abstractions in Go. Using bare channels (like using `goto`) is fine for small local things, but if you're building abstractions you should use patterns like those discussed in the article (for example: https://go.dev/play/p/UoB8ECDbaTw --I'm sure this can be abstracted further).
Any language that supports true multithreaded parallelism provides foot-rail-guns, no matter if you’re locking, asyncing or channeling. For concurrent data structures, you can’t lazily rely on the compiler to catch your errors. Not even in eg Rust. You should never sprinkle concurrent primitives just because.
For every channel created, you need to determine how the invariants of both the channel itself and the contents that pass through it. This is, for the most part, an easier exercise than with locks, because channels provide a way to think about ownership, and crucially, transfer of ownership between threads (goroutines). It also decouples the response to an event from the production of it, whereas with locks you have to dispatch the “next action” from the code that produced the event. I’ve never had a language that’s so easy to do things like “spin up 10 concurrent requests at a time with individual timeouts, cancel outstanding requests if any of them succeeds or canceled by the user, and ensure all tasks are torn down before returning”. It’s near-trivial to get right with channels, contexts and waitgroups.
In either case, since concurrency is so difficult, I would say that the Go docs and resources are quite lightweight on how to use their primitives correctly. Right now, it’s a bit “now finish the rest of the fucking owl”. You need discipline to use them, but there’s no parental figures around to tell you how to watch out for the dangers.
In Go you can use a type alias to make this work:
type ExitChan <-chan nil
Unfortunately it doesn't really enforce anything at the compiler level, but at the very least you can grep for it.
For example, lets look at "string". Most examples are gonna just call it a "str" or even "s". But if what you have is someone's name you should get called out for calling it a "str" rather than "name".
You shouldn't name all your mathy variables x, y, i, j.
You shouldn't name your channels "chan" or "c". Unless its just generic channel code.
I don't mind short domain variable names. But "chan" isn't that. It's like calling all your ints "num" or all your strings "str".
There's a linter which I like[1] that checks variable name length based on the scope of its usage.
The idea is a variable name can be short if it is used in a small lexical scope (e.g. a function parameter in a one-line function) because it won't affect readability much. But names should be descriptive if used in a larger scope (e.g. a constant used across a file).
This just sounds like badly written chaotic code. Unfortunately there are some people who seem to think "Go has channels, therefore, I need to use channels as much as possible", but that's almost always a mistake.
Channels shouldn't be used very frequently, but they can be very handy especially in combination with select and/or buffered channels. Usually there should be a clear and "obvious" API.
There are multiple ways to use channels wrong, the compiler will not help you and most of the docs and "tutorials" will not cover that. The best thing you can do in Go is stay as far away from channels as you can.
The worst is when people use channels simultaneously with mutex.
With async/await, everything is still a function. Your IDE and debugger help you understand when and where a function is called. When debug-breaking on a function you have the stack trace.
Async/await gets you fast context switching for IO ops as it is a single OS process (no threads for the OS to constantly switch).
But once you go beyond a single OS process, then you need something like messaging or shared memory on top of async/await to communicate between OS processes/threads.
I think Golang jumps straight to messaging which is better for multiple cores, but misses the first step of async/await which is easier to reason about when you have a single process on a single core.
Golang needs to have a way to manage the channels better. Naming them and waiting on them would simplify a lot of crusty stuff. Naming is becoming possible, goroutines can already have pprof labels (that are even inherited between goroutines!), so just adding pprof labels to stacktraces will help a lot.
But unfortunately, Go creators are allergic to anything that brings thread-local variables closer.
I do find the Actor model a bit nicer at an engineering level, simply because I find being able to refer to things by name to be a bit more intuitive, and since Actors (at least in Erlang) can manage their own state it's a lot easier to kind of shape them into doing anything you want (e.g. buffered or non-buffered). I'm much less familiar with the formal definitions of Actors though.
That was one odd thing that stood out to me about Go. I am coming from Erlang so a process having process ID we can keep track of, terminate it, trace, etc, is fundamental to be able to reason about and operate a system. Advertising that they can handle millions of lightweight goroutines but then having no obvious way to identify them, and monitor their lifetimes is kind of strange.
On the lower level, I sort of understand why they did it: they focused on typed channels. So they can have multiple channels of different types potentially talking to the same goroutine. Now, having both named goroutines and channels would be more complicated, and they tried to keep things "simple". Erlang's processes on the other hand, have implicit mailboxes, there is no "mailbox1" and "mailbox2" there is only one, but it's also easier there because there is no static typing. There things are simple because only processes have identities but not mailboxes.
As a thought experiment, I could see Optionals, Named Channels, Collections etc being implemented in some common but non-stdlib library. Of course now we’ve recreated all those languages people hate when they talk about go… but those features exist for a reason.
This is the most frustrating thing about Go for me. They use thread-local storage within the stdlib, but refuse to let us plebs have access to it.
Consider a Node app that calls a C++ web service. The C++ web service returns "503 Service Unavailable" because your request causes the server to segfault. The useful stack trace for fixing this problem is on the C++ side in a core dump, not whatever Node prints when the HTTP request fails.)
What you want to do on the Go side is have your sender be able to relay an error to the receiver, and to wrap errors:
func doWork(w work) (result, error) {
return errors.New("too lazy to do work")
}
func doWorkQueue(q <-chan work, r chan<- result) {
for w := range q {
result, err := doWork(w)
if err != nil {
r <- errorResult(errors.Wrapf(err, "doWork(workID=%v)", w.ID))
continue
}
r <- successResult(w, result)
}
}
...
go doWorkQueue(...)
submitWork(work)
for r := range resultCh {
if err := r.Err(); err != nil {
log.Error("failed to do work: %v", errors.Wrap(err, "recv result"))
}
}
main.go:20: failed to do work:
main.go:20: recv result:
main.go:9: doWork(workID=42):
main.go:2: too lazy to do work
(At work we use a hacked up copy of github.com/pkg/errors, which grabs the entire stack trace at each of the errors.Wrap/errors.New call sites. This results in an exceedingly verbose trace that takes up your entire screen, but is at the very least... thorough.)
The reason that error wrapping is essential is because stack traces don't capture critical information, like what iteration of the loop you're on, how many times your retry loop ran, which work id failed and returned "too lazy to do work", etc. The error wrapping is where you get to add this in. This is, again, the same as every language... the random exception you throw in your HTTP client when the server is down isn't aware of the work item id that caused this exception, so you have to catch and re-throw with that information or you have an undebuggable mess.
What's nice about Go is that it's really easy to add this contextual information, either with fmt.Errorf in the standard library, or with very small functions that capture some information automatically (errors.Wrap/errors.Wrapf). I will say that not doing this is my #1 complaint, and it's exceedingly common to just "return err". I have spent 4 years fixing the work codebase to wrap errors, and people still add new unwrapped errors (because our linter allows an escape hatch, errors.EnsureStack, which some people really like). It then results in some oncall engineer wasting a week debugging a simple problem. Sigh! But that's humans being humans; Go makes it very easy to do the right thing. You just have to tell your team to do the right thing and to make them want the right thing.
[1] https://blog.jgc.org/2024/03/the-formal-development-of-secur...
I often dream of Go with Rust’s borrow checker. Sigh…
There’s nothing in the type system preventing data races. But that’s orthogonal to channels. They just help you to structure the code.
There are just so many things that are just slightly wrong to make it unpleasant. Better than many other things, but still kind of frustrating. I really think unbuffered channels should have been the default.
I think the point of Pike's quote is that, when a goroutine gets a pointer received from a channel, it gets the ownership of the values referenced by the pointer and other goroutines give up the ownership. This is a discipline Go programmers should hold but not a rule enforced by the language.
A simple and useful degenerate case of this is that whenever any message is sent, complete ownership of the entire transitively-reachable set of values contained in that message is by default transferred to the receiver. This is worthwhile even if you must explicitly construct some safe value to pass in order to maintain this promise. This keeps things generally easy to reason about without the full complication and power that Rust offers. At the very least, whenever I violate this, I have lots of comments about what and why on both sides of that transaction in my code base.
(I do sort of wish there was a variant of a "go" statement that I could use that made me statically promise that all communication in and out of a given goroutine must be solely in the form of copied values, so I could guarantee that goroutine was an "actor". This is, admittedly, just me wishing very pie-in-the-sky. I doubt it could be turned into a practical proposal.)
You're talking about the fact that Rust's lifetime analysis was initially a simple lexical check, and has evolved toward properly understanding control flow beyond just lexical scope (e.g. it is aware that both branches of an if/else cannot be taken).
This has nothing to do with ownership, a completely separate part of the type system from lifetime analysis, and how it prevents data races by controlling the sharing of data between concurrent processes, which is what this discussion is about.
I would dearly like to see Rust advocates be more careful when they talk about this aspect of their beloved language.
The ownership semantics of safe Rust programs prevent programs from being written, and in the process, eliminate entire classes of bug. But they also prevent a literally infinite number of correct programs from being written, without resorting to the unsafe escape hatch.
Sometimes that's a good tradeoff. Often you can design your program around those restrictions and enjoy the benefits they bring. But what you said is much too strong a claim! Ownership semantics prevent or inhibit a great deal of simple patterns which are in fact possible to implement correctly. The doctrine that every difficulty a programmer runs into trying to color inside the lines of safe Rust is a case of Rust preventing them from doing something wrong, is simply incorrect.
Rust's memory model is highly opinionated, and in fact, restrictive. The pitch, and it's a strong one, is that the benefits of working within that model are worth learning how to work within those restrictions. But it's also very clear that ownership semantics create problems as well as prevent them. Telling people that the tradeoffs are worth it is good advocacy. Pretending there are no tradeoffs is not.
My main point is, the general concept of ownership and having to care about it is present everywhere, though. If you prefer to say that it reveals it but imperfectly, I'm down with that. But I think it's important to understand it isn't entirely 100% responsible for creating it. Every shared-state threaded language has the issues, it just doesn't have a type system and compiler (imperfectly) helping you with them. It's not a license to write Go or any other language while oblivious to ownership issues just because the compiler won't complain.
It's simpler than that: Rust prevents you from doing a lot of correct things in the safe subset of the language. It does it for a good reason, because that lets the compiler prove a bunch of nice properties, but it's a fundamental tradeoff: in exchange for not having to deal with bugs in shared mutable access to data, it doesn't let you write programs that way. I actually think that Rust is a great choice for multithreaded programs which want to follow a one-writer-many-readers pattern, because that's very difficult to do correctly without the borrow checker.
But when you say this:
> The ownership problems are 100% there in other languages too, it just isn't a compiler error.
A straightforward read of that is that anything you can't do in safe Rust is a bug. That's very far from true, it's in fact backward: what you can do in safe Rust isn't an ownership bug, by construction. But I see this implication reversed in a lot of Rust advocacy, and it directly contributes to the Rust fatigue which you'll see on this very website, and many other places.
It's the difference between prohibiting potentially unsound ownership policies (what Rust does) and prohibiting only unsound ownership policies (which safe Rust does not).
> My main point is, the general concept of ownership and having to care about it is present everywhere
I completely agree with this, fwiw.
I'm aware that this is a very controversial take, because lots of people love duck typed languages, but after working in large codebases, they are a hard pass for me. There's a reason why TypeScript was created for JavaScript, Sorbet was created for Ruby, and type hints is so popular with Python. I think I've seen something about gradual typing being introduced in Elixir, but gradual typing is still a long ways from enforcing type safety. Until then, I'll stick with Go.
If I am mistaken and there is a "TypeScript for Elixir", then I would love to know about it.
I love working with Elixir. Go is something I tolerate.
I can recommend making a utility function which accepts a set of anonymous functions and a concurrency factor. I've since extended this function with a version which accepts a rate limiter, jitter factor, and retry count. This handles most cases where I need concurrency (batches) in a simple and safe way.
You can do channels (message passing) on top of async await.
As for uncovered topics or part 2 - long running Go channels can be a nightmare.
You need to implement kind of observability for them.
You must find the way to stop/run again/upgrade and even version payload to handle your channels/goroutine.
Common problem with channels overuse is so called goroutine leaks. Happens more often than most devs think. Especially, if lib writers initiate goroutines in init() to maintain cache or do some background cleanup job. It's good to scan all used packages for such surprises.
You might also find concepts like "durable execution" or "workflow" engines down the road.
I recently wrote a simple function that maps out tasks concurrently, can be canceled by a context.WithCancel, or if a task fails. The things that cancel the task mapper need to coordinate very carefully on both sides of the channel so that they're closed and publishers stop sending in the right sequence. The amount of switches/cancels/signals quickly explode around the coordination if you too cute on how to do it (e.g. read from the error channel to stop the work).
Frankly I'm not sure I still got it right [1]. And this is probably the most unsettling part. Rereading the code I can't possibly remember the cancellation semantics and ordering of the short mess I created. Now I'm wondering if mutexes would've made for more understandable code.
[1] https://gist.github.com/pnegahdar/1783f0a4e03dc9a3da43478994...
I think everyone goes through a bit of a complication phase with channels, I recognize your issues in code I've written myself, this is definitely not a "only a bad person would have this problem" post. But, yes, there probably is an organization that solves this problem. There's an art to using them properly. A good 50% of that art may well be that consumers should never close channels.
(Another one is the utility of sending a channel as part of the message in a different channel. It is intuitively easy to think that channels must be very expensive, but when they're not participating in a select, they're just a handful of machine words in RAM. It is perfectly sensible to send a message to a "server" process that contains the channel in it to send the reply to, because it only costs a few machine words in allocation and precisely the one sync operation it will ever participate in. Channels do not have to amortize their costs with lots of messages; a 1-message channel is practical. This also cleaned up some complicated code I had before, trying to prematurely optimize something that was already very cheap.)
The other thing is, if you haven't looked at https://pkg.go.dev/golang.org/x/sync/errgroup , you may want to. golang.org/x/ is the not-as-well-known-as-it-should-be extended standard library; things the Go team are not willing to put into the 1.0 backwards compatibility promise so they retain the ability to change things if necessary, but otherwise de facto as high quality as the standard library, modulo some reasonably well-labeled exceptions. Contra some claims that it is impossible to abstract in Go, many of these common concurrency patterns have been abstracted out and you can and should grab them off the shelf.
> Generally this is consumers closing channels, which is not a good idea.
I've heard this too. I updated my comment to include the code snippet. But any general tips on when the consumer needs to cancel the rest of the work because of an error?
I don't recall now but I think I ended up writing my own for fast-fail and generic support. Really hoping the x/sync packages get generic support soon!
> a 1-message channel is practical
Great tip, every time I add a little signal one I feel like I did something wrong, maybe that is the move.
Yes, another channel which everyone watches for cancellation.
In modern Go, that should be a context and it's .Done() channel.
This is another "I'm not saying you suck because I did this myself a while before I figured it out", but having a channel around just to cancel things is fine. It doesn't cost anything significantly extra. I have some still in my code bases where I've just never had any reason to upgrade to contexts.
The underlying feature that people are looking for is a TCP-like "either side can kill the other side" feature. This is what contexts are.
func read(ctx context.Context, c <-chan any) {
for {
select {
case <-ctx.Done():
return
case x, ok := <-c:
if ok {
fmt.Println(x)
} else {
return
}
}
}
}
func write(ctx context.Context, c chan<- any) {
for i := 0; i < 10; i++ {
select {
case <-ctx.Done():
close(c)
return
case c <- i:
}
}
}
The writer can still kill the reader by closing the channel, because closes are writes.
I prefer explicitly using contexts for cancellation, but in a pinch:
c := make(chan struct{})
foo(c)
bar(c)
close(c)