Hacker News Reader

Comment by JohnCClarke | Hacker News Reader

TBH: Context feels like a wart to me. It works, but it's not elegant. Golang has an aversion to thread^H^H^H^H^H^Hgoroutine local storage. Instead it provides this kludgey experience.

I really feel that Golang V.2 should invent a better, native, way of controlling threads of execution.

View on HN

Hmm, I think you think of context differently from me. For me context is something you use to manage execution. I mostly use it to notify different parts of the program that "you can stop what you are doing now". For instance if you are processing a request and the client went away. Or you have run out of time.

You're talking about context as a way to distribute data? I do that as well. For instance to provide auth/session data to requests, but that's usually just limited to one path in my software that does this. (I agree it is clumsy, but not because it is the "wrong" thing to do, but rather the API feels a bit dodgy).

If you are talking about something like thread-local storage, that's really a very different thing from both the control aspect of context and the request data aspect.

What extra functionality do you want for goroutine control and why do you think Go needs it?

What annoys me about Context is there's no way to tell if it's honored by the callee.

And when I'm accepting Context I'm annoyed at having to write handlers for it all through the stack having no idea how/if people will use it.

> What annoys me about Context is there's no way to tell if it's honored by the callee.

Everybody has to be decent enough to do their part.

> And when I'm accepting Context I'm annoyed at having to write handlers for it all through the stack having no idea how/if people will use it.

Thank you for doing yours :)

> What annoys me about Context is there's no way to tell if it's honored by the callee.

On the occasions where I've needed that I've used a WaitGroup and done wg.Add(1) at the point where I start goroutines and then have a defer wg.Done() as the first thing in the goroutine. I don't think the functionality belongs in Context. And if you put it there, you'd just end up complicating things.

> And when I'm accepting Context I'm annoyed at having to write handlers for it all through the stack having no idea how/if people will use it.

How would you propose you do it instead?

> On the occasions where I've needed that I've used a WaitGroup and done wg.Add(1) at the point where I start goroutines and then have a defer wg.Done() as the first thing in the goroutine. I don't think the functionality belongs in Context. And if you put it there, you'd just end up complicating things.

I'm not sure this is the same thing. The point of Context is to propagate cancelations or timeouts across multiple layers of your app and libraries, it's not supposed to be useful for directly started goroutines.

Perhaps not, but if you have no idea what ought to be cancelled how would it help you to know that something has been cancelled?

What changes would you make to Context?

I really disagree with this. A function taking a Context is a really important signal to me about the semantics of that function. I also much prefer being able to see context values explicitly passed around, instead of values that magically appear out of the ether, without a clear code path to find out where they came from, what goroutine it's bound to, where that goroutine came from and its lifecycle, etc.

The context is just a big bag of stuff. you don't know what's really in it. Ends up almost any method that needs something from the big bag ends up having a context parameter, but you don't know why that method needs it.

Separating these concerns (cancellation vs. bag-of-request-scoped stuff) might make sense. I'm specifically talking about the cancellation side of contexts. I don't think there's a good answer to this other problem of whether those two concerns should be combined into one mechanism, the options that I know about all have mixed tradeoffs.

I still think an explicit bag-of-stuff is better than an implicit one though.

Believe me I've tried to write Go code that doesn't follow the Go conventions and can't get my PRs approved even with tiny differences. So the idea that I could separate these two concerns might be a good one, but in practice it would be impossible.

Treating a context value as a bag to fetch data out of is the first mistake. They should only ever be used to control things like deadlines and whether or not a part of a function executes. IMHO they should disallow attaching values to a context.

Yes/no/maybe? context is one of the few ways to get contextual logging and tracing to work in a almost general way in Go. But I have also used it to pass the authenticated user to the handlerfunc. I dont dig it, but it works and avoids the need to keep a static map of request pointers somewhere to figure out which user was in this request...

Right, you don't know what's in it, but you know you need to forward it if you start a goroutine.

I also need to forward it to nonroutines because there is a logger in the context and almost every function wants to use the logger. So essentially, we have to almost alway pass context as the first argument to any function unless its some private function trivial function.

Yes, but I will curse everyone who didnt pass the logger so we lost the logcontext :(

I always thought that context should just be a goroutine local thing always available, automatically inherited when `go` is executed (obviously with the option to set an explicit one).

I think of contexts as being Go's answer to dynamic variables in earlier languages, like Lisp, and less like thread local storage (like Pthreads). Much like how Go works with errors, explicit is favored over implicit -- being able to see the context pass through, and whether a function expects a context, tells you a lot about the function you are about to call.

If a function does not take a context, you know it probably cannot be interrupted, just like when a function does not return an error, you know it should not fail. In my work projects, this is also a cue that the function does not do any logging since we always carry a zerolog.Logger in our contexts enriched with trace information about the request and handler.

This also makes life easier for me as a reviewer -- I can see the context passing, I can spot when there is a bad pattern, like retaining a context, or failing to handle an error. It does not require me to maintain a detailed mental map of which functions employ dynamic variables or can throw exceptions.

I think Jonathan Blow's programming language, Jai, has an implicit context available to any function. However in Jai the context has a lot of implicit functionality, on the top of my head at least logging plumbing, and allocator plumbing.

An alternative would be a language that has structured concurrency built in. [1]

The rules around goroutines and context seem to point in the direction of structured concurrency. For example, if any goroutines started in a function get cleaned up before return then that's following the rules of structured concurrency.

Thread-local storage is bad because it's implicit and causes bugs when used with concurrency; if you farm out some work to another goroutine, it will break.

[1] https://en.wikipedia.org/wiki/Structured_concurrency

> Thread-local storage is bad because it's implicit and causes bugs when used with concurrency; if you farm out some work to another goroutine, it will break.

Can you elaborate on why being implicit is bad and how it causes bugs?

I understand that shared data (via pointers) may cause race conditions and other unexpected behavior, so let's say we require that the thread-local storage can only store values (with value semantics).

If you could point out any issues with that, I'd greatly appreciate it.

It's been a while, but the underlying issue is that threads aren't always one-to-one with server requests. You can have a request where some work is handled by multiple threads. Or, a single thread can do some work for multiple requests.

So one possible bug is that you have a function that implicitly depends on thread-local storage, and then you move some work to another thread and call the function there, and it doesn't work because its dependencies aren't there. You need to manually set up the thread-local storage of each new thread.

Another bug is that if a thread does work on multiple requests (say, a task queue), some thread-local storage could leak data from a different request.

In larger systems, it might even be worse: one request can be farmed out to multiple servers and then you need to pass the context along over the network when doing rpc. This only works for serializable data, but things like deadlines can be propagated, and a request id that ties it all together is useful for logging.

"Which request am I working on" is something that's transient and often doesn't map directly to OS-level objects. (Although it does map one-to-one in simple cases.)

So to paraphrase, the problem is around context passing between threads, which makes sense.

Sounds like that could be solved by a language surfacing the context at thread boundaries like the `go` statement, possibly channels.

Thanks for the answer!

Are we missing a thread pool/executor like abstraction for workers? If we really want callers to be able to control concurrency primitives deeper down the stack, we should coalesce on an executor paradigm that the library can use as it's work queue.

The only way to do that would be to introduce a concept that is orthogonal to functions (and their signatures) / errors, and that would mean an incredible increase in complexity. I doubt the Go authors will do that.

I think it splits people the same way that err values split people, which is that Go makes more things values in service of making the control flow plain.

Errors as values would be nice if they weren't so terrible ><

The tooling around them is nearly non-existent in stdlib, and if you want to understand errors from stdlib you will have to guess at string content.

And like, why is there not a stack trace by default?

Reply on news.ycombinator.com