Kornia: Differential Computer Vision | Hacker News Reader

Kornia: Differential Computer Vision

github.com

69 pointsArtWomb5 comments

BTW title should be s/Differential/Differentiable

As a non-native-english speaker foreign to ML and CV, what does that mean?

This library is super cool because it implements a ton of important classical computer vision methods in a way that is compatible with automatic differentiation. So you can combine modern deep learning methods with classical methods

In case you're asking about the parent comment (s/Differential/Differentiable), it means the title should be "differentiable CV" (instead of "differential CV").

"Differentiable" describes some computation for which derivatives can be computed. "Differential" is a more general term which means that something has to do with differences, e.g. differential equations deal with equations that specify how things change together.

With the recent surge in deep learning came significant improvements in optimization techniques and hardware, making it feasible to formulate some computations in a differentiable manner. Doing that allows one to optimize the computation process relatively efficiently, at least in theory. Some other examples: differentiable programming[0] (other differentiable techniques are a subset of this), differentiable rendering[1], differentiable signal processing[2].

[0] https://en.wikipedia.org/wiki/Differentiable_programming

[1] https://arxiv.org/abs/2006.12057

[2] https://github.com/magenta/ddsp

I'm really interested in using this library, but I have to admit I'm struggling to think of use cases. I can see differentiability being useful in feature extraction and maybe some uses of augmentation like colour correction, or spatial tranformations -- for example in the image registration example they give -- but overall the kinds of operations offered by the library tend to be used for preprocessing, and not really needed in the trainable pipeline.

For instance edge detectors and even most feature detectors are basically hard-coded kernels, are they not? And you wouldn't apply them on intermediate representations, they are only really valid operations for the input layer, and not needed during back-propagation.

I find the examples interesting, but mostly are things that could be done using OpenCV and don't exploit the differentiability in any meaningful way. But maybe I am missing something.

Reply on news.ycombinator.com