Hacker News Reader

Comment by squigz | Hacker News Reader

Wow that sounds like a really interesting use-case for this. Can you link to some of those examples?

I don't have anything specific to link to but you could try it yourself with line art. Try something like a mandala or a coloring book type image. The model is trying to capture something that encompasses an entity. It isn't interested in the subfeatures of the thing. Like with a mandala it wants to segment the symbol in its entirety. It will segment some subfeatures like a leaf shaped piece but it doesn't want to segment just the lines such that it is a stencil.

I hope this makes sense and I'm using terms loosely. It is an amazing model but it doesn't work for my use case, that's all!

Actually a combination of LLM and VLMs could work in such cases. I just tested on some circuit boards. https://chat.vlm.run/c/f0418b26-af20-4b3d-a873-ff954f5117af

Thanks for taking the time to try that out and sharing it! Our problem is with defects on the order of 50 to 100 microns on bare boards. Defects that only a trained tech with a microscope can see - even then it's very difficult.

Seems like a exciting problem. Hope you get tools in the future which can solve it well.

It looks like there could be a lot of potential here for learning/repair/debugging/reverse engineering. Really cool application of this stuff!

Generally this is called automated anomaly detection.

Have you found any models that work better for your use case?

To answer your question: no but we haven't looked because Sam is sota. Trained our own model with limited success (I'm no expert). We are pursuing a classical computer vision approach. At some level segmenting a monochrome image resembles or is actually an old fashioned flood fill - very generally. This fantastic sam model is maybe not the right fit for our application.

Edit: answered the question

This is a "classic" machine vision task that has traditionally been solved with non-learning algorithms. (That in part enabled the large volume, zero defect productions in electronics we have today.) There are several off-the-shelf commercial MV tools for that.

Deep Learning-based methods will absolutely have a place in this in the future, but today's machines are usually classic methods. Advantages are that the hardware is much cheaper and requires less electric and thermal management. This changes these days with cheaper NPUs, but with machine lifetimes measured in decades, it will take a while.

Way late response: the off the shelf stuff is very very expensive as one would expect for industrial solutions. I was tasked to build something from scratch (our own solution). It was quite the journey and was not successful. If anyone has pointers or tips in this department I would truly love to hear about them!

My initial thought on hearing about this was it being used for learning. It would be cool to be able to talk to an LLM about how a circuit works, what the different components are, etc.

Reply on news.ycombinator.com