Writing graphics software gets much easier | Hacker News Reader

Writing graphics software gets much easier

web.mit.edu

52 pointsmaxko879 comments

Ack, while the paper looks to be legitimate (and has big names like Marc Levoy on it), the beginning of this write-up is terrible. Facebook absolutely did not buy Instagram for its filters as they are well understood technically - FB bought Instagram for its exploding community. Furthermore, as of at least Instagram 2.0, filters were all heavily GPU accelerated and can be performed in real-time on the phone, i.e. massively parallel and unlikely to need any benefit that Halide might provide.

One of the original paper authors here. Some things got lost in the translation into a news piece. Here's our take on Instagram:

Instagram is only able to perform the most trivial filters expressible as point-wise operations (mostly curves and vignetting). These are easy to parallelize. If we want to go beyond that (e.g. clarity in Lightroom, which uses a more complex algorithm called Local Laplacian filters), we need to be able to write fast, complex, image processing pipelines on embedded devices more easily. Currently these sorts of things are possible on embedded (see Adobe's various offerings for iOS), but they take expert engineers a long time to write.

…and even with simple point-wise filters, Instagram had to manually code all its filters for OpenGL ES to run on the GPU to get reasonable performance. Last I heard (months before the acquisition), only one or two people at the company could write new filters, since they were the only ones with enough esoteric knowledge to build reasonable imaging pipelines in GL ES, even as simple as those pipelines were.

The point here isn't that it becomes possible to do things no one could do before, but that it becomes radically easier so:

1. More people can build fast imaging pipelines than previously had the skills to do so. 2. Even those who had the specialized skills can build and explore fast imaging pipelines radically more easily.

Still, this is a research language and compiler, so there are plenty of limitations and rough edges. The main point is to demonstrate and explore a potentially better way of doing things in this domain, so future languages and tools can learn from this experience. But we have found the actual tool to be quite powerful in practice, and so are making it available to the public.

Results are impressive with "snake image segmentation" alone but OK with other algos. http://people.csail.mit.edu/jrk/halide12/

I didn't get how algorithms expressed in this language are faster, if it is just a 'Representation'. Is it because of its efficiency in parallelization? If so, any hand tuned assembly should be on par with it. any other thoughts?

Btw, the title "Writing graphics software..." is misleading as 'Halide' addresses only Image processing on CPU/GPU but nothing to do with 3D/2D graphics software.

I have some utility for image processing algorithms but I am no expert in the field, nor do I have any desire to become one - I just have some ideas I want to implement.

The idea that I can write my algorithms in this language and target them at the CPU initially (to keep costs down while focusing purely on photo processing), then have it running on Amazon's EC2 GPU infrastructure later with no changes when I want to (and can afford to) expand into video processing is pretty awesome.

And all of that while having the code apparently run faster than hand tuning (which I have no interest in doing) would allow. Implementing a parallalizable hand-tuned algorithm does seem like it should be faster, but even if that were true I don't have the time or expertise for attempting that.

If you read the paper, you'll see that the detail is a little different from the headline claim. In section 6.3 they go through comparing C++ at various levels of optimization to equivalent Halide code, and eventually reach C++ code that is faster than the Halide code. However, to do that, even the tuned C++ ballooned from 122 lines of code to 370+ lines, while the Halide implementation was 35 lines, and had allowed them to discover optimizations quickly, by separating the concerns of the algorithm and scheduling. The C++ code is also now very specific to the architecture it's tuned for.

So the win here is in terms of maintenance and effort, and the ease of porting to different architectures. Hence, I expect, why they also show comparisons with Matlab.

The paper has sample code: https://docs.google.com/viewer?url=http://people.csail.mit.e...

Every language I read about is dubbed the second coming, and I rarely hear about it again. Why is that?

It's very difficult for languages to "escape the lab" as it's often put. I'd say the biggest factors are:

- Developers are often reluctant to learn new tools.

- Project managers are reluctant to use new languages, seeing a small community as a risk.

- Academic projects have different priorities than industry. Implementations done by academic teams often need substantial work before they meet industry needs.

This boils down to visible necessity. Without that, it's difficult for a language to reach critical mass. But the visibility part is tricky, because it's really a marketing problem, not a technical one. Sometimes organizations are blind to their own pain or even are attached to it, believing it's necessary (ORM comes to mind). Sometimes a language spreads because someone less risk adverse has very visible success, creating a pressure to copy their choice (37signals and Ruby for example).

But language research is still worthwhile even if it withers after publishing. The ideas propagate, and often end up as features/libraries in more broadly accepted languages (erlang probably inspired akka for example).

Reply on news.ycombinator.com