Hacker News Reader

Comment by suyash | Hacker News Reader

Is it still relevant in the days of Tensorflow, Pytorch and other high level frameworks that solve computer vision with ease?

View on HN

Yes. It's a different kind of computer vision. Not the kind where you just throw an expensive NN at stuff and hope for the best, but one where we're actually dealing with operations at the pixel level (corner / edge features, image warping, stitching, calibration etc). No neural nets, just very fast operations on the actual image data.

It's useful either as a preprocessing step that you feed into further NN processing or -- super old school -- you are modeling your problem by hand instead of letting a NN figure it out (for better or worse).

It's also got a decent NN implementation. You can have both.

Absolutely. Not all computer vision problems are AI related and not all AI problems are solved by pytorch. If I need a program for, say, soft real-time processing of some industrial automation video streams to run on a low power system opencv is my #1 choice. My 2c.

OpenCV and "AI" can work well together; see YOLO: https://pjreddie.com/darknet/yolo/

In that example, they're using OpenCV to display images and read from the webcam for a demo. At one point, they make it clear that OpenCV isn't used for any of the CV parts:

> We didn't compile Darknet with OpenCV so it can't display the detections directly. Instead, it saves them in predictions.png. You can open it to see the detected objects.

Yes, it is totally relevant. Creating a product that processes images/videos/photos typically requires composing some small models, applying classical functions to work with polygons, edges, transformation matrices to rotate/shift coordinates and pixels, resizing, working with colors, working with bright/contrast curves, open/write files, classic filters, etc. PyTorch and Tensorflow don't have this or use in their examples OpenCV functions to do it.

It totally is: OpenCV does pixel-level image processing, in a deterministic and generally highly optimized way. For some computer vision, this is definitely enough, and it will have a much better performance than doing it with neural networks. It might be complementary to higher-level frameworks (typically based on machine learning models), e.g. for object detection/classification.

I know a chap that uses it for Arduino-powered drones. He is not a coder.

I agree that projects like OpenCV should be better-supported.

There are a couple of issues, though, that always crop up, when talking about supporting open projects:

1) I don't know of any corporation, anywhere, that actually donates for no reason at all. There's always a hook. Sometimes, it's just brand-building (logo on the free swag stuff), sometimes, it's to attract future employees (for instance, donating to a project that is maintained by a certain set of students from a college curriculum, etc.), sometimes, it's to influence "hearts and minds," and sometimes, it's a pure investment. They need a self-interested reason.

2) I don't know of any corporation, anywhere, that donates, with no expectation of influence. That's one reason why lobbying is such a big deal. There's a fig leaf of "no quid pro quo," but everyone knows that the donator is expecting a return.

3) I don't know if anyone has noticed (</s>), but the corporate environment tends to be, just a bit, on the competitive side. I know many corporations may not be willing to donate to a cause that will serve their competitors, as well as themselves.

I think that foundations help. They can set up a "step and repeat" page, with donors, but keep the branding (and influence) off the actual donations.

Corporations could donate in a way that benefits them. For example, Intel contributes a lot of code to the Linux kernel. Presumably they are at least partially contributing stuff that makes it run better on their chips. But lots of people run Linux on Intel chips. And while working on stuff that benefits Intel, their engineers are probably going to do other ancillary tasks that could be helpful. So it seems like a win-win.

It should be socially acceptable for open source programs to offer the deal: we’ll consider patches from your engineers, but in exchange you have to chip in enough to support one of ours (to keep working on stuff we find interesting), and a little bit more (need some engineer-hours to review your patches).

It sounds quite biased in favor of the project at first, but if the project is popular, the company could get quite a bit out of it.

Is it a bad thing if corporations donate towards features that benefit them? That is a signal to the developers to allocate their energy towards useful ends.

In addition to helping keep the library in development member organizations of OpenCV get some perks as a result of their support such as appearances on our live streams, press releases, and pull request priority. We have tiers for Bronze ($6k/yr), Silver ($30k/yr), Gold ($100k/yr) members with more benefits for each tier.

I very much doubt you can do any image processing on Arduino. A typical Arduino has only 2048 bytes of RAM, like, what kind of an image can fit in there?

I used OpenCV and Yolo on an RPi4 for object detection. Worked great. Fun project and I learned a lot from the OpenCV community.

I think the ability to learn computer vision from scratch (and free) like I did is one of the best aspects of open source generally.

But yeah, an Arduino would be tough.

I don't know. I can ask him.

He has just been raving to me about how great OpenCV is. I know he's been using ChatGPT4 to give him C drivers for his custom rigs.

Like I said, he's a hardware geek, and actively hates coding, so I'm not exactly sure what he's been doing.

I will say that his drones and RC vehicles work a treat. I drive them around, occasionally.

There’s all kinds of Arduinos now. It’s not your typical ATxxx microcontroller.

https://store.arduino.cc/products/uno-r4-minima

Could be something like this: https://www.pjrc.com/store/teensy41.html

From memory (heh) they come with 1Mb and you can add another 8 or 16...

It's a 600Mhz ARM M7 also, you can do a fair bit or work with that.

Even taken at face value such pipelines often still rely on OpenCV for I/O, data cleaning and preprocessing.

There are CV applications where techniques you'd currently get consensus on calling 'AI' would be overkill (unnecessarily intensive to run) or relatively poor performing.

Say you just need to do some edge detection for example.

Of course the 'AI' line is blurry and moving, probably 'computer vision' (vs. 'just processing some image data', which of course it is anyway) has been too.

Definitely. There is still a very large class of problems that are amenable to image processing pipelines and algorithms . And a very large number of problems where there is insufficient training data to simply throw deep learning at it.

Absolutely!

I'm not into that space as much lately, as I have been years ago, but in a lot of newer marketing videos I can easily recognize the OpenCV signature markings. For example from tesla videos (not the typical public facing ones, but on the NVidia tegra side) or robotic startups

We decided for OpenCV (on a project that provided vision to a production line, think 50 images per second). The speed and computational cost advantage weighed in favour of OpenCV. I don't think this has changed so much in the last couple of years.

as far as I am aware opencv has a lot of classical Computer Vision stuff that is missing in tensorflow/pytorch. In case you need speed, or can't have data annotation, or something where deep learning is not super useful (slam) it's still very relevant I'd say.

There are lots of ways OpenCV complements Tensorflow, pytorch, and other high level frameworks. They don't need do all those things themselves, because they assume you have OpenCV around for stuff like image io, preprocessing, manipulation, transformation, color space conversions, edge detection, filtering, segmentation, morphological transformations, feature detection, video io, real time video capture, camera calibration, 3d reconstruction, optical flow, object tracking, qr code detection, gui creation, etc.

In addition to the other replies:

Check out the official or unofficial examples for TF, PyTorch, and OpenVINO for CV applications; they all use OpenCV as a dependency, eg to get get the image data from a camera or RTSP URL, process the image, draw debug info and display the results etc.

Some of those things could be embedded nicely into OpenCV, though. Without me having to learn about all the underlying concepts, finetune a model and learn how to deploy it on various devices. OpenCV is a wrapper around lots of common image functionality, and it could be extended to cover modern deep learning approaches.

Only if hallucination is not a problem for your particular application.

I really dislike the word "hallucination"

It's not a "hallucination" it's a "miss-prediction".

And the more-classical methods of CV are still very capable of miss-predicting.

No matter what method you choose, you must test how accurate it is, NN have the benefit of being fine-tuned for specific applications as well. If you want to be extra sure then you likely want to use both.

That was also my thought. I have used OpenCV 10 years ago for some small things, but today I would simply look for any existing neural network on HuggingFace and maybe train some custom classifier on top if I need sth custom. I cannot really think of any possible case where this would not work better than what you have in OpenCV. And if speed is a concern, there are also many small models, or otherwise you can quantize them yourself, or maybe train yourself a small model with knowledge distillation. You can get this as fast and as small as you want, while still probably outperforming everything that OpenCV can offer.

I use OpenCV for blur detection and camera shake detection on mobile devices, to qualify the image before handing it along to an AI model to extract data. I can't think that 2 extra neural networks would be faster at this task than OpenCV is, even if they might be better given how subjective blur is.

And motion detection that runs on the camera itself, to decide whether or not to send the video stream to the expensive GPU servers in the cloud.

https://www.axis.com/developer-community/open-source/acap

ACAP Computer Vision SDK examples

ACAP version 4 example applications that provide developers with the tools and knowledge to build their own solutions based on the ACAP Computer Vision SDK.

The open source examples are focused on video analytics applications. Giving AI/ML developers the chance to experience the smooth and smart develop environment that ACAP provide, showing how the powerful capacity of Axis devices provides unlimited possibilities for developers to build new AI/ML applications.

Being one of the most active repositories we have, you can find examples written in C++ and Python performing interesting features that varies from object detection, QR decoder, and image capture, using popular open source ML libraries such as OpenCV.

https://github.com/AxisCommunications/acap-computer-vision-s...

How does it do the blur detection? I assume it's also just a convolutional kernel applied on the image in some way. So it would do just exactly the same as a tiny neural network with one convolutional layer. Thus, they would be exactly the same speed, if they would use the same underlying native kernels. However, the native kernels in TF/PyTorch/etc are more heavily optimized than OpenCV, thus I assume that OpenCV would actually be slower here. And also more complicated, as you need to mix two big frameworks.

Realistically I think you'll often want both depending on what you're doing. Especially for things like blur detection, what's your acceptable specificity, acceptable/scale of performance, where you're running the algorithm (on device vs cloud).

I'm not an expert at all but most image processing network I've seen generally involve at least a few plus a few other layers. I don't think you can get away with a single convolution, at least not that well.

OpenCV you could use Laplacian variance which looks like it's just a single line of code.

> cv2.Laplacian(image, cv2.CV_64F).var()

Many of the NN implementations look like their finetuned off google's ViT checkpoints. I really can't imagine these are faster (at least not without spending extra on GPU/TPU's) than Laplacian variance but I could be wrong.

And I assume you might be able to get better evaluation performance from a finetuned NN but depending on what you're doing, that's a ton of work compared to opencv.

https://pyimagesearch.com/2015/09/07/blur-detection-with-ope...

https://sh-tsang.medium.com/review-bdnet-blur-detection-conv...

regret to inform you that image preprocessing and IO is done in either openCV or PIL or both for every image processing NN i’ve ever worked with.

Yes - not sure how this has been overlooked. For Python/torch I tend to use cv2 with an albumentations augmentation pipeline, which is much faster than PIL/torch transforms.

sometimes Pillow-simd is nice as well.

Thanks - I’ll check it out.

I can’t run those HuggingFace models on low power devices (including medium power microcontrollers). I can do OpenCV on them at the edge though.

You can. For example: https://pytorch.org/edge (Just an example, there is a lot more, similarly also for TensorFlow.)

I’ve tried. They simply don’t run any decent useful models on an ESP32-S3 at a speed that has real use cases. Maybe one day, but not yet.

Whereas today, OpenCV and derivatives are already using the LX7 vector extensions and run fast enough to be useful.

I’ve worked on plenty of problems where all you need is to find what pixels are a specific color. While I’m sure a NN could be setup for that, why make things harder than they have to be?

Why harder? My point is, by using some DL framework, you would make this simpler. You don't need to have a big NN for that, or even any NN at all, and could still use some DL framework, and I would assume this is still easier and probably faster. E.g. finding what pixels are a specific color, this would be sth like:

    torch.all(image == torch.tensor([255,0,0]), dim=-1).nonzero()

To get all pixel pairs (x,y) where the color is red (255,0,0) of the image of shape (width,height,channels).

How do you generate the torch vector to do this? Often it's OpenCV!

torchvision.io.read_image?

This is like answering "the grocery store" when asked where food comes from.

Yeah, OpenCV is really easy to work with and use in quick scripts to process images.

I have used it to group images of the same subject/angle, then remove the most blurry ones.

Yes, big time. For instance, once you have segmented the image you should use something like openCV to extract contours of the segmentation mask.

> or high level frameworks that solve computer vision with ease?

That's overstating things more than a little.

Reply on news.ycombinator.com