Show HN: Bild – A collection of parallel image processing algorithms in Go

Memory management for a one-and-done image conversion isn't much of an issue. The program will pretty much allocate, run a few very fast GCs since almost all memory usage is in a handful of big arrays (some people seem to have this idea that all GCs are multi-millisecond ordeals, which isn't the case), and then at program termination just throw everything away. A program like this could get away with an allocator implementation that allocated normally but whose "free" is a no-op [1].

What would slow something like this down is not using the relevant assembly code to accelerate working with these images. A very quick glance at the convolution package, chosen on the grounds that if anything is ASM it probably is, shows that this really is "pure Go", in the sense that it doesn't use Go's ASM support. The compiler will not optimize that to speak of, certainly there's no autovectorization, so it's going to be significantly slower than an optimized program that does make correct use of vectorized operations.

Though, in theory, this can be cleaned up over time. Go does have the support built-in for fixing that in a reasonably nice way, as long as you don't mind that it has its own assembler dialect.

[1]: Not 100% related, but close enough, the part towards the end: http://www.drdobbs.com/cpp/increasing-compiler-speed-by-over...

Less that the anti-GC crowd thinks, specially when the right algorithms get used, one does not allocate like crazy, and just like in Rust pays attention to where data gets allocated and how.

Last time I looked, the `image` crate was the defacto standard for doing image manipulation in Rust, and in my opinion its API is/was pretty bad. (Had several different types for "an image," which made things annoying, and I recall hitting unreachable!() in a public function once).

I can give you a real answer. If you use go's image and color interfaces then boxing can stress the allocator if you process pixel by pixel.

But assuming you represent and process your image as a packed slice of rgba or something, the gc wont be in your way at all.

Source: led team that built production go image resizing and conversion services at walmartlabs.

maybe a good start, seems this compares rotation for 90 degree angles and resizes of natural multiples (2,3,4..) if so that's not enough to consider for production consideration.

Check out benchmark suite, seems more comprehensive: https://github.com/jcupitt/libvips/wiki/Speed-and-memory-use

I ended up building a transcoder within the past few months using Lambda that is handling, as of today, 500k+ videos per month and growing. This was replacing a set of EC2 instances similar to what you described.

A few caveats I ran into: 1) You are limited in how much /tmp on a Lambda instance can hold (512MB total). For the transcode I'm doing on these boxes, some of the videos are that size just to download. It will fail. I have a backup using the older method to handle these very large instances. 2) These, obviously, need a pretty decent amount of RAM to run.

But the benefits are really worth it for us. Elastic Transcoder, for our needs, was going to be >$15k/mo). Our current transcoder cost is around $400/mo. The previous, EC2-based iteration, was a step up for us, but it sometimes took a while to start. Doing it with Lambda has actually been less expensive, provided a faster experience for our customers, and was significantly easier to build than the EC2-based option.

Not bad for a project to see if I could write something in Go.