Context should go away for Go 2 | Hacker News Reader

Context should go away for Go 2

faiface.github.io

4 pointslladnar40 comments

I agree with the author wholeheartedly. I see the solution that contexts solve, but it seems like a hacky bolt on that made an otherwise beautiful language cumbersome.

A solution? Well, I'd had a few passing thoughts but never really vetted them much or thought of implementation, just throwing things at the wall. I'll share anyways.

A) A supervisor type system. the 'go' command would return some id, eg

  x := go runthis()

where x is the id. This could then be used to identify, kill, etc things at a supervisor level.

B) Maybe more esoteric, but treating go routines as implicit channels. There's a few ways this could go, but it would be similar to the above, except the return would be an channel. This could allow passing signals to the goroutine, and also, retrieving values from the goroutine. Eg

  x := go run()
  y := <-x  // y is the return value of x
  or
  x := go run()
  x <- 1 // terminates the x goroutine

I don't think A would work because the Go team has been opposed to goroutine id's, the main detractor being it would be used to create thread local storage.

I like B, but a major use case for Go is http servers, and I can see that being awkward when most frameworks don't expose the creation of the gorountine.

> I don't think A would work because the Go team has been opposed to goroutine id's, the main detractor being it would be used to create thread local storage.

I find this a weird stance. Idiomatic Go already has pervasive mutable globals, and life before main. Aside from cancelation the other thing that `context.Context` poorly abstracts is goroutine local storage. Having go routine ids and the new sync.Map seem like a backwards compatible way to add these features to go now.

I really like B, I'd prefer to see killing the goroutine by closing the channel and then adding a

ChildChannel.dependsOn(ParentChannel)

for explicitly deciding when you want to end a whole tree of computation.

Like Erlang's links, sort of.

I thought Context was also used to partially solve the "middleware" problem in http servers. Ex. I have an authorization middleware that takes the http request parameters and returns an authorization object that is available to request handlers.

Way back when, Martini solved this problem using reflection (which was met with vitriol). Other packages solved this with their own handrolled context object (which was always a map[string]interface{}). I personally always liked how Martini handled it (you got to "keep" your types), but understandably it included way too much magic.

The response to Martini always confused me. The language and community (at first, anyway) was all about terse identifiers, type inference, interface inference, etc. But then, e.g. with the continual repetition of `if err != nil {}`, or the "just use stdlib" approach to HTTP, it injects a lot of ceremonial verbosity back into the ecosystem.

It's not the verbosity that bothers me. It's the arbitrary "verbosity is bad/verbosity is good" choices that have emerged over time.

Go is consistently anti-magic, but not particularly terse (although the last time I wrote a Go program next to an equivalent C++ program, it was about 20% shorter). Instead its syntax strives for whatever the hell Rob Pike means by "lightness".

Roughly, languages like Java add length by long identifiers and numerous modifiers to convey that something belongs to particular semantic category that behaves in a particular way.

Go adds length by having fewers such categories and forcing you to explicitly code up the behaviour. Short identifiers (and proper use of scoping!) are orthognal to this, except insofar as they reduce the pain.

I guess I'm thinking mostly about the talk Rob Pike gave called "public static void" [1], where he probably describes pretty well what "lightness" means to him, or at least what it meant to him 7 years ago. It seems largely about verbosity.

[1] https://www.youtube.com/watch?v=5kj5ApnhPAE

Can you explain how reflection helps here? Are you saying that people still pass tracing objects around, except they don't have to be context.Context?

That's nice, but it doesn't sovle the fact that the auther (rightly!) doesn't want to add extra complication to interfaces like io.Reader

Reflection, the way Martini used it, was for dependency injection. So in our imaginary Go language, the io.Reader interface could be satisfied by either of two functions:

  Read([]byte)
  Read(context.Context, []byte)

In a way reflection let you do operator overloading, so in your case you could just call a.Read(p) and others could a.Read(ctx, p). In Martini's case, if you wanted your handler to be cognizant of a cancellable context, you would just add the context parameter.

No context does 2 things :

- send signals by exposing a channel

- serves as a bag of values (the infamous .Values method ).

Context was invented because Go developers kept writing leaky servers as they were referencing *http.Request outside an http.Handler (for session management for instance) . If you do that, you need to be able to signal that the request treatment is done or you will have a memory leak.

I guess they added .Values(interface{})interface{} because they could.

The post is right about the problems with the context object, but is flat wrong when it claims that the only problem it solves is cancelation.

Cancelation is just one aspect of a generic concern: that a tree of function calls (and more importantly, RPCs) is a causal chain of events that external systems (including humans) will want to monitor and control. Trace IDs, auth tokens and other things need to be propagated somehow, but the code that they propagate through shouldn't care about them.

So its nice to have context.Context in the standard library so that we don't have a proliferation of per-library hand-rolled solutions which then have to be virally propagated.

I can't speak about Go specifically, but something I'd love to see in a language is a general "context" mechanism for passing data from a very high scope to a very low scope, without changing the functions inbetween, and without resorting to global variables.

A (not very good) example in pseudocode would be:

    complicated_operation(param) with context (log_out=myfile);
    // complicated_operation calls a function,
    // that calls another function...

    void log(string str) context (File log_out=stdout) {...};

This would be a bit like TLS, but has the advantage that the context reverts after the original call.

Usually, when I have this problem, I just stuff all the functions into a class, and make the "context" private fields. But then you end up with classes that really "don't want to" be classes, the lifetime of the context state is longer than necessary and unclear, and it feels like a hack. Also, in many languages going from free functions (and plain data and closures) to classes takes a bit of rewriting due to different syntax.

The only language I'm aware of that has these "dynamic scopes" is (apparently) Emacs Lisp, but it should be possible to fake them in C++ quite well.

It would also make the situation in the article a lot cleaner, IMO.

This might be a little out of scope, but C#/.NET has a few similar constructs. If you do:

    using(TransactionScope scope = new TransactionScope()){
         //do stuff
    }

Any connection created while that block is active will 'enlist' in the transaction associated with that scope. This works (I think...can't find it now) by using something called 'CallContext', which is more or less a static key-value store that's persisted up and down the call stack (including through async continuations, not sure how that works).

If the 'CallContext' is static, doesn't that mean it's global and shared between threads? Well...in C# that's not necessarily true since the thread is accessible via a static property and it's possible to store data against the thread (which is exactly what 'CallContext' does, plus some magic that moves the data during async continuations in case the execution thread changes)

Scala has implicit parameters. For examples, see http://daily-scala.blogspot.nl/2010/04/implicit-parameters.h... (simple examples) or https://stackoverflow.com/a/9538177 (more useful ones)

IIRC scheme and probably common lisp have something like this:

(call-with-dynamic-bind ((var1 val1)(var2 val2)...) stuff...)

It lets you introduce dynamically scoped variables that work pretty much the way you described.

In Common Lisp this is just let, where the variables are dynamic variables.

(There is a dynamic binding operator called progv, but the situation for that is when the variable names are computed.)

I don't think so. _let_ is lexically scoped. What I was remembering was a syntax called parameterize in chicken scheme that would bind a set of values to a set of dynamic variables. You can see an example of it here: https://github.com/caolan/snowy/blob/master/server.scm#L85 I don't think common lisp has an equivalent (and I'm pretty sure it's not part of rnrs in scheme), but it should be really easy to hack one together.

In Common Lisp let is either lexically or dynamically scoped, depending on whether the symbol is marked "special" which is the situation if the symbol was previously used as the name in a defvar or defparameter form to define a dynamic variable.

So dynamically scoped contexts are achieved just by binding these special variables. By convention, they are usually given names beginning and ending with an asterisk ("earmuffs") to put them into an effectively separate namespace.

That parametrize is reminiscent of fluid-let; another Scheme hack to simulate dynamic scope.

Aaahhh! Yeah, that makes total sense. I was thinking about newly introduced symbols using let. I recently jumped from scheme to common lisp so I'm used to lets just shadowing other variables.

Most of the major scheme implementations I've used have had real dynamic scoping of some sort. I don't think it's very well supported in the standard, but they all seem to have it as a proper language feature.

The convenient thing about special variables just using let is that you can use them in any binding construct:

    (with-output-to-string (*standard-output*)
      ... code ... )

Here, code's output to the default standard output stream is captured in a string.

Common Lisp has dynamic scope via dynamic variables.

> but it should be possible to fake them in C++ quite well.

Been there done that; had a dynamic_var<T> template some seventeen year ago. It provided thread-specific rebinding and all.

TCL has dynamic scope for those who don't know.

This keeps repeating "Go is a general purpose language", but is it?

https://golang.org/doc/faq#What_is_the_purpose_of_the_projec...: "By its design, Go proposes an approach for the construction of system software on multicore machines."

That page points to https://talks.golang.org/2012/splash.article for "A much more expansive answer to this question". That article states:

"Go is a programming language designed by Google to help solve Google's problems [...] More than most general-purpose programming languages, Go was designed to address a set of software engineering issues that we had been exposed to in the construction of large server software."

I find it interesting that one of the reasons GO refuses to provide a way of identifying a routine's unique context is to prevent abusing that as a way of having a session key attached to it... and then the standard libraries go and make a ctx variable to do EXACTLY that.

I think a solution would probably relate to looking at client connections.

well, the ctx can be passed between goroutines and still give the ability to cancel the logical operation, no matter in which goroutine it happens to live

I really appreciate the argument in this post. That being said, I wish the author provided a suggestion of what the cancellation mechanic would be

I like the idea of a standard signaling + param passing struct{}. I don't like the idea of context.Context slowly taking over the whole golang stdlib.

Then again, I'm positive I'm not as smart as numerous google (and golang) engineers that have already thrown around ideas so maybe this isn't as big a deal as it seems.

Not sure I have a solution, but I'd love to see someone come up with one before context "infects" every function related to I/O.

> Go is a general purpose programming language, just like C, C++, Java or Python. For example, I’ve been using Go for about 2 years and I’ve never written a single server in it.

This made me twitch because ultimately no argument should start with, "I use it this way and not that way, so change what you're doing." This was followed by:

> For this reason, when designing the Go language and it’s standard library, we need to approach it from a general purpose language perspective.

My understanding of what's being said here is simply: "I code in Go in this particular way, so it should be made to suit those requirements." Just because you don't write servers in Go it doesn't mean others don't. I personally enjoy the production ready HTTPS server and client on a daily basis. I'm constantly writing APIs in Go.

All I'm saying here is don't start by saying something doesn't suit your needs so it needs to change.

The rest of the article was great and gave me more insights into how Go works, so I thank you for that. And it's possible there is a better solution for context that should be considered for v2.

Thanks for the good read.

I think it is fair enough for the author to say that since Go is a general purpose language, its APIs should not end up infected by a viral pattern needed for servers.

So far, I think Go does a good job of using context.Context where it is needed and not letting it leak into things like io.Reader. But it is a tricky balance to strike, because the the author is right that the context pattern is viral by nature.

To a degree I can understand his, and your, view point here. However my understanding, which could be wrong, is that Go was designed to solve problems specific to Google. In my opinion that doesn't make it a general purpose language, that makes it a purpose driven language.

It's general in nature in that you can ignore certain features and use it as a general language, but ultimately it has a built in HTTPS web server for a reason, right? And a programmer and an SRE, Context is an excellent idea for tracing requests through the whole stack -- again, there was a purpose to this design.

Well I can't speak for Google about why it funds Go. My own observations are that Rob probably made Go just to be a language that he likes and that it is also a good fit for the way things are done at Google.

I'm in the odd position that when I go to work as an SRE at Google, I hardly use Go, but at home I like to use it for hobby projects which are rarely servers. When I do use Go at work, it is for tools that I run on my workstation.

So (a) even at Google there is more to life than servers, and (b) Go benefits from being attractive to people who are not at Google and/or not writing servers.

They fund it because they wanted a language to suit specific goals and be purpose built to address certain problems. Why does anyone make anything? To solve a specific set of problems -- positive side effects are a bonus, such as the solution being general purpose enough that it can be used for other tasks/problems too.

Ansible has one goal in mind: configuration management. It can also talk to my toaster and send a Tweet when I pop some bread in there. I guess Ansible is a "general purpose" configuration management tool, except it's not :-)

Terraform can be used to run a PowerShell script on a Windows box it never provisioned. I guess it's a scripting framework now.

I think Go can be used as a general purpose language, but that's not how I see its design ethos. It caters to that crowd, but does have an underlaying goal I believe.

As long as people are happy and things are working for them then all is well, I guess.

This repeats "Go is a general purpose language", but is it?

https://golang.org/doc/faq#What_is_the_purpose_of_the_projec...:

"By its design, Go proposes an approach for the construction of system software on multicore machines."

They also refer to https://talks.golang.org/2012/splash.article for "A much more expansive answer to this question". That article states:

I'm glad to read this, I've seen context prop up in more and more code for a while now and it feels like a bit of a wart on what was a good, simple programming language.

There's a few more warts that have been added since Go 1.0, presumably to preserve backwards compatibility. Magic comments, the forking of the syscalls package, the vendoring experiment, the alias functionality, etc.

Question, how do other concurrent languages like Java or C# handle the concept of 'thread context', and cancellation?

To me it sounds like having to pass a context everywhere is something that should be handled on the language level, just as concurrency.

yes, like an expended channel reference. This way you can open up this reference to handle channel management/cancellations.

In RethinkDB which is C++ with a home rolled m:n threads library, it passes in an "interruptor" which is like a one shot chan struct{} you can wait upon or select over with other chans. The main difference is that the functions which take an interruptor and throw an interrupted_exc_t exception. You could add language support for doing this sort of stuff automatically, or very conveniently, though it would need a bit of care when lambdas or closures get involved.

The article didn't really give an example of using contexts but from what I can tell they are a way to add additional semantics to a function? The article mentioned cancellation and implicit parameters at least.

My first reaction was that monads would solve the problem. Since this is go hardcoding cancellation intonthe language probably is the best available solution, though.

Reply on news.ycombinator.com