Non-Blocking Parallelism for Services in Go
goldsborough.me
goldsborough.me
for {
select {
val := <- someChan
doSomething(val)
}
}
Instead:
for val := range someChan {
doSomething(val)
}
1. It reads better
2. It leaves the loop when you call close(someChan), which is neat
But that's just my take. I'd be curious if anyone has any reasons for the single-channel-read-in-a-select-block approach
For this reason the article I would say is also suboptimal, and is non-idiomatic. It's something you'd do in other languages, but should be avoided in Go.
In my opinion you should just start a goroutine. The goroutine could block on a semaphore/channel to limit concurrency, but there's nothing inherently more costly about having the goroutines themselves be the queue instead of like the article having a list.
Yes, it'll probably take a bit more memory to create a goroutine than to add to a list, but almost always I'll take that to get more correct behavior.
But also, your suggestion doesn't handle the requirement "Calling the service should not block the caller", does it?
The article overengineered, by far. You don't need infrastructure for this. It's just:
for work := range workGenerator() {
go process(work)
}
sem := sync.NewSemaphore(runtime.NumCPU())
for work := range workGenerator() {
work:=work
sem.Acquire(1)
go func() {
defer sem.Release()
process(work)
}()
}
sem := sync.NewSemaphore(runtime.NumCPU())
for work := range workGenerator() {
work:=work
go func() {
sem.Acquire(1)
defer sem.Release()
process(work)
}()
}
Yes, if in doubt then block the caller. I mainly provided it as an example because the article had it as an explicit requirement.
You're right about errgroup. One can fit only so much in a HN comment. And I didn't want to distract by making it look like "no, you should use my favourite library instead".
But again it depends. If an error is handled by doing log.Fatal(), then there's no point in using errgroup.
I'm also not passing ctx, which much (most?) nontrivial code should pass.
c := make(chan struct{}, 10)
for job := range jobs {
c <- struct{}{}
go func(){
defer func() { <-c }()
}(job)
} sem := semaphore.NewWeighted(int64(runtime.NumCPU()))
I like the push approach outlined here:
http://marcio.io/2015/07/handling-1-million-requests-per-min...
I have done a more generic version of this with success.
You have N workers, each reading from a separate input channel for work to do.
Then you have an outer "channel of channels", holding those worker channels. When you want to submit work, select a worker channel from the outer channel, submit, and push the worker channel back on the outer channel when done.
It breaks contexts (and all that implies, like credentials, tracing, timeouts) and stacks (making it harder to debug).
You should just start the goroutine instead of enqueueing it.
More info here: https://news.ycombinator.com/item?id=25831844
If you want best-effort asynchronous job processing semantics, then it's totally appropriate to use something like what's described in the article -- ideally with a lot less code :)
If you want request-response, then, yeah, this isn't appropriate, and I agree that you should do whatever work in the request goroutine, blocking as necessary.
The issue with spawning goroutines all the time is that spawning, while cheap individually, can become expensive if done in large numbers (like 1 Million times a minute as in the article). Is that not considered problematic ?
E.g. the godoc for the context package itself says "Do not store Contexts inside a struct type". I understand the reason to be that it's easy to make the lifetime nonobvious, and make it "strange" (hard to read and reason about) if it's not very strictly kept under control. In other words it becomes a foot-gun as the code base grows.
And even then the stack is still unhelpful.
Benchmarking for your own workload beats anything else no matter what the language, of course. But it may change in the future, too. Maybe the next version of Go actually makes it faster? In other words it's not wrong, but great care should be used.
One reason it can be faster to spawn goroutines is that IIRC spawning a goroutine actually makes the current OS thread start running that. And if it completes then it can jump back to the spawner. In other words spawning goroutine can save thread creations and context switches, while a goroutine pool will likely incur an OS context switch, with cache implications and other complex interactions.
But yes, measure with your actual workload is king.
It's all tradeoffs, I guess.
This means that the queue could grow unboundedly, but that is ok, as this type of structure is not meant for constant request handling but rather for bursts of requests that can happen asynchronously (and in parallel), and allows us to limit the maximum number that can run while also never blocking callers if there are no available go routines to process their request when submitted.
Here is a minimially-viable example of how I do it:
https://play.golang.org/p/mD_jpMdoY_g
Contents here:
package main
import ( "fmt" "time" "sync" )
func main() { fmt.Println("Hello, playground") wg := &sync.WaitGroup{} wg.Add(1) numRequests := 100
inCh := make(chan int) go func() { resCh := make(chan int) queue := []int{} inFlight := 0 max := 20 completed := 0
for {
select{
case m := <- inCh:
queue = append(queue, m)
fmt.Println(fmt.Sprintf("Enqueing: %d", m))
case <- resCh:
inFlight -= 1
completed += 1
}
if len(queue) > 0 && inFlight < max {
v := queue[0]
queue = queue[1:]
fmt.Println(fmt.Sprintf("Submitting: %d", v))
inFlight += 1
go func(_v int) {
time.Sleep(1 * time.Second)
fmt.Println(fmt.Sprintf("run: %d", _v))
resCh <- v
}(v)
}
if completed == numRequests {
wg.Done()
}
}
}()
for i := 0; i < numRequests; i++ {
inCh <- i
}
fmt.Println("Waiting for completion...")
wg.Wait()
fmt.Println("All processes completed. Exiting.")
}queue = append(queue, m)
Hell even C/C++ is going to have a hard time, you need a VM with garbage collector!
Goodbye Karma!
Coding without types is not productive if you want to do meaningful work.
Clojure does not exist without the Java VM.
I have a hard time seeing how these two languages could possibly be considered best-in-class for concurrency, or their memory models. Loads of concurrent code as been written in these languages despite their concurrency and (shared!) memory models, not thanks to it. Imperative, shared-memory concurrency is the hardest way to do concurrency. In that respect, Go at least provides some syntactic sugar to encourage good practices. Either we're talking past each other or one of us is missing something.
>Coding without types is not productive if you want to do meaningful work.
That's questionable. And not just because it's hand-wavy (what does "meaningful work" even mean, here?). It's trivial to come up with counter-examples to even the most charitable interpretation of "types are required for meaningful work". If your statement were true, Rails, Django, Clojure, JavaScript and hell, even PHP wouldn't be such raging industrial successes.
>Clojure does not exist without the Java VM.
That's false. It's a hosted language, yes, but it's specifically designed not to be JVM-dependent (see, for example: ClojureScript).
Are you expressing thoughts or sentiments, here?
Let's say, here's an example, explaining why the different components compose into good/bad design, performance, whatever. Sometimes/usually full idiomatic code makes sense, other times not. What's idiomatic might change too.
There might be very valid or invalid reasons companies use internal patterns. They all tend to turn sour after commoditization. If a stable library fully encapsulates a separable need, might be good to use it.