Halide: a language for fast, portable computation on images and tensors | Hacker News Reader

Halide: a language for fast, portable computation on images and tensors

halide-lang.org

146 pointsespeed23 comments

https://www.youtube.com/watch?v=3uiEyEKji0M

Here is a relevant video from the creator. It has some fun illustrations to help with what its going on.

/edit - letters

In addition to this video overview by Andrew Adams from the Stanford course [1], there's also an MSR talk given by Jonathan Kelley [2], and there may also be a Google TechTalk somewhere on YouTube (don't remember for sure so just search for "Halide" or by one of the authors' names).

And of course there's also the videos of lectures of the MIT 6.815/6.865 course on computational photography given by Frédo Durand (those videos are linked to from the Halide home page under the Getting Started section [3]).

[1] Stanford CVPR2015 http://halide-lang.org/cvpr2015.html

[2] Decoupling Algorithms from the Organization of Computation for High-Performance Graphics & Imaging (MSR talk given by Jonathan Kelley) [video] https://www.youtube.com/watch?v=dnFccCGvT90

[3] MIT 6.815/6.865 Digital & Computational Photography lecture videos by Frédo Durand http://halide-lang.org/#gettingstarted

The funny thing is that YT itself has been using Halide to process all videos, perhaps as early as when that one was uploaded.

And anyone using Google Camera has been running Halide code, from Pixel phones all the way back to Glass, which took decent photos despite having hardware that, due to the form factor, was less powerful than the Galaxy Nexus that came out 1.5 years before.

Do you have a source for this? Maybe examples of code in-situ?

The use by YT and GCam has been mentioned, among others, at https://suif.stanford.edu/~courses/cs243/lectures/l14-halide... (last slide)

More mentions by Google: https://www.blog.google/products/pixel/pixel-visual-core-ima...

https://productforums.google.com/forum/#!topic/phone-by-goog...

The HDR+ siggraph paper (hdrplusdata.org) states that they use Halide, as does the newer siggraph paper on the Pixel's portrait mode (http://graphics.stanford.edu/papers/portrait/wadhwa-portrait...), and the paper on the Pixel's video denoising algorithm (http://www.chiakailiang.org/papers/ehmann_2018_rtv.pdf). The Pixel camera is riddled with Halide code, but of course it's all closed-source.

I can also vouch for the youtube use, having been tangentially involved in it and given permission to state that they use it.

Halide seems like it might have immense potential for fairly general applications in scientific computing, but all of the documentation, tutorials and examples seem dominated by image processing. Could someone elaborate on the limitations of Halide with respect to applications like finite difference and finite element methods, lattice Boltzmann simulations, etc.? These are all ultimately reducible to basic linear algebra and stencil kernels, which seem well suited to the design of Halide. Should computational scientists give it a serious look, or am I missing something?

Halide is generally good for pipelines that do math on multidimensional arrays, so quite possibly. It has a property that I think would make it hard to use for some applications in scientific computing though - while you can mutate in-place, you can't mutate an array ("Func" in Halide) after it has been used by another pipeline stage. This means that the dependency graph between your pipeline stages has no cycles except for self-cycles, which lets us do lots of clever stuff like bounds inference to guarantee the program is correct regardless of the schedule. I can imagine things like flow simulation having a hard time with that constraint. In the deep learning context this means that CNNs are ideal, but RNNs can be awkward or impossible.

Andrew: have you considered adding something akin to Rust's lifetimes here? While you'd have restrictions on the schedule, it seems like it's still let you express who in the "graph" messes with the data as tightly as possible.

Among other optimizations and advancements, Halide provides compiler support for advanced latency-hiding scheduling/optimizations beyond the capabilities of generic compilers, which essentially enables you to generate highly-parallel optimized CPU/GPU compute kernels to make memory accesses free.

There was a paper released recently to do deep learning on Halide[1] that I'm excited to see merge[2] with the main Halide codebase.

Seems like the creators of Halide are in this comment thread, does anyone know when this is happening?

[1]: https://people.csail.mit.edu/tzumao/gradient_halide/ [2]: https://github.com/jrk/gradient-halide

It works but still needs a bit of cleanup. I was planning to get back to it after an upcoming conference deadline. Here's the PR: https://github.com/halide/Halide/pull/3220

Previous discussions: https://news.ycombinator.com/from?site=halide-lang.org

I'm interested to see how well Halide's approach would pair with the GPU optimizations being put into GraphBLAS [1].

[1] https://hn.algolia.com/?query=GraphBLAS&sort=byPopularity&pr...

Halide doesn't currently have good ways to handle sparse matrices, unless they have some sort of special structure that lets you pack them into rectangular shapes (e.g. a constant number of non-zeros entries per row). It would be an interesting language extension. The big thing to add would be a way to reduce over the non-zero entries more efficiently than checking them all.

Mainly https://news.ycombinator.com/item?id=7544855 from 2014.

Also https://news.ycombinator.com/item?id=10410832 from 2015.

If the creator of that page is reading this, please fix the color of the code example text. Right now it's a dark shade of gray that is hard to distinguish from the background until syntax highlighting kicks in. While you're at it, maybe figure out why it takes almost ten seconds for the syntax highlighting to kick in.

I agree, although it was more like 2 seconds on my laptop. It was almost unreadable to me until the colors leapt in.

Sorry, poor choice in the default Bootstrap style we adopted years ago. Under normal conditions the styles kick in in a second or less, but I've fixed it nonetheless.

Reply on news.ycombinator.com