camdencheek

0

Oh, we do also heavily parallelize. This blog post is just focusing on the single-core perf.

From the mouth of this post's author: yes.

Definitely possible! However, I prefer to avoid cgo wherever possible for more than just the overhead.

In my experience:

- It complicates builds by requiring a C toolchain

- It makes single, static binaries more difficult

- It makes portability more difficult (not that assembly is portable though)

- It causes difficult-to-debug issues (I recently ran into an issue where MacOS signing changed, causing all my cgo binaries to be killed on startup)

- Debuggers don't work across Cgo boundaries (they do with go ASM!)

I think Dave Cheney said it best: https://dave.cheney.net/2016/01/18/cgo-is-not-go

AFAICT, that still doesn't support 8-bit integer dot product? (To clarify, I was using int8 to mean 8-bit integer, not 8-byte)

Copying in a response to a similar question on Reddit:

I should really add some discussion around BLAS in particular, which has an good implementation[0] of the float32 dot product that outperforms any of the float32 implementations in the blog post. I'm getting ~1.9m vecs/s on my benchmarking rig.

However, that BLAS became unusable for us as soon as we switched to quantized vectors because there is no int8 implementation of the dot product in BLAS (though I'd love to be proven wrong)

[0]: https://pkg.go.dev/gonum.org/v1/gonum@v0.14.0/blas/blas32#Do...

I did consider Avo! I even went as far as to implement a version using Avo since it has a nice dot product example I could use as a starting point. But ultimately, for as small as these functions are, I felt that Avo was an unnecessary extra layer to grok. Additionally, it's x86-only, and I knew in advance I'd want to implement an ARM version as well since we also do some embeddings stuff locally.

If I were to ever take this further and add loop unrolling or something, I'd absolutely reach for Avo

Thanks for the feedback!

> Will that end up on Github?

It's already there! I just haven't cut a release since the change.

> Most often I want to return just the first error.

In many cases, I do too, which is why (*pool).WithFirstError() exists :)

> original errors can be inspected as intended

If you're using errors.Is() or errors.As(), error inspection should still work as expected.

> Often goroutine overhead is negligible and I would bound concurrency in dumber ways

Yes, definitely. And that's what I've always done too. However, I've found it's surprisingly easy to get subtly wrong, especially when modifying code I didn't write, and even more especially if I want to propagate panics (which I do, though that seems to be a somewhat controversial opinion in this thread). Conc is intended to be a well-known pattern that I don't have to think about too much when using it.

> I think(?) conc does that too, but it could use documentation

It does! I'll update the docs to make that more clear.

> nice work

Thanks!

> The default concurrency GOMAXPROCS is almost never what I want.

FWIW, the default concurrency has been changed to "unlimited" since the 0.1.0 release.

> Aggregated errors are almost never what I want.

Out of curiousity, what do you want? There is an option to only keep the first error, and it's possible to unwrap the error to an array of errors that compose it if you just want a slice of errors.

> Using it would place a burden on any reader that isn't familiar with the library

Using concurrency in general places a burden on the reader :) I personally find using patterns like this to significantly reduce the read/review burden.

Fair criticism. The nice thing about the current API is it works with any input that's iterable (channels, slices, readers, etc.) and any output (callback, channel, append to slice, etc.). In most code I write, I avoid channels because I find them easy to misuse. I used channels in the examples because it's the easiest way to represent "some potentially unbounded stream of input."

> panics aren't "goroutine scoped" in terms of their potential impact

I'm with ya there. However, there are also many classes of logic errors that are not goroutine-scoped. And there are many panics that do not have impact outside of the goroutine's scope. In my experience, this is true of most panics.

In practice, panics happen. They are (almost) always indicative of a bug, and almost always mean there is something that needs fixed. However, if a subsystem of my application is broken and panicking, there's a pretty good chance that reporting the panic without crashing the process will provide a better end user experience than just blowing up.

Yes, that means I'm accepting the risk that my application is left in an inconsistent state, but coupled with good observability/reporting, that's a tradeoff I'm willing to make.

(bonus: this is especially true when propagating panics allow me to capture more debugging information to fix the panics faster)

> run different things in the background and gather the results in different ways

I'd be curious to see an example of the type of task you want to be able to do more safely

> The WaitGroup looks suspiciously like errgroup

I heavily used errgroup before creating conc, so the design is likely strongly influenced by that of errgroup even if not consciously. Conc was partially built to address the shortcomings of errgroup (from my perspective). Probably worth adding a "prior art" section to the README, but many of the ideas in conc are not unique.

> In go land, this seems desirable.

I mostly agree, which is why `Wait()` propagates the panic rather than returning it or logging it. This keeps panics scoped to the spawning goroutine and enables getting stacktraces from both the spawning goroutine and the spawned goroutine, which is quite useful for debugging.

That said, crashing the whole webserver because of one misbehaving request is not necessarily a good tradeoff. Conc moves panics into the spawning goroutine, which makes it possible to do things like catch panics at the top of a request and return a useful error to the caller, even if that error is just "nil pointer dereference" with a stacktrace. It's up to the user to decide what to do with propagated panics.

Whoops, yep, thanks for pointing it out. Just fixed it

Nope, just a catchy short form of "concurrent"

This is exactly correct. Behavior is equivalent, performance is not. It's probably still not a great example because if reading from a channel already, you're probably better off spawning 10 tasks that read off that channel, but the idea of the example was that it can handle unbounded streams with bounded concurrency.

Hi! Author here. Conc is the result of generalizing and cleaning up an internal package I wrote for use within Sourcegraph. Basically, I got tired of rewriting code that handled panics, limited concurrency, and ensured goroutine cleanup. Happy to answer questions or address comments.

My favorite project to give kids who are just dipping their toes in the water is a Madlib. It prompts you for the blank words, then prints out the complete story with the blanks filled.

The flexibility of Fluentd and the community around it are great, but ultimately the resource intensity proved to be too high and too unpredictable for our use case.

We ended up building our own general purpose log parser in Go to support our hosted log monitoring platform. Though it's still somewhat young, performance is great (beats Fluent Bit in most of our benchmarks), and it's nearly as flexible as Fluentd in terms of configuration. If you want to check it out, we recently open-sourced it and are always looking for more feedback: https://github.com/observIQ/stanza