[flagged]
I've seen these agent-written fake anecdotes on Twitter, Reddit, and now here, all with the exact same formatting. They pretend to be real people with real anecdotes, but they're all completely made up.
> latency matters more than raw accuracy – think industrial inspection
it (rightfully) raises red flags in anyone when you hear someone confidently claim raw accuracy is _not_ important in things like _inspection_
These things aren't impossible of course but it's additional management over "place the device here".
Here's how you know that accuracy isn't the be all and end all of the discussion - we already deploy systems with less than human accuracy to monitor things, and when we use humans we very rarely inspect every single item. So there must be a tradeoff we're happy making in lots of industries.
Even if you're focussed on not missing anything, lower accuracy that comes at the cost of more false positives can be massively useful as you can then do a two step process (even with humans as the second step if you need). The goal of the first step is to ignore the 99% of totally fine items so you spend the costly process on just 1% of the items.
but i wouldnt stand on a soap box and yell that to the world without all that ^ nuance
but by the time i'm done with all that, i'm only preaching to the choir
That’s wild. And scary.
Hope HN has tooling ready to handle this ongoing onslaught of manipulation...
Already so, LLMs are trained on human-written text, and then spit out text they try to make human-like, so now a bunch of stylistic choices some humans made are "tellsigns of a human using LLMs for writing". It's not just bad, it's removing humanity from the humans.
Obviously the AI itself doesn't have any goal (that matters anyways), but the humans/organizations that set it up obviously have a lot to gain. Accounts of age/above karma thresholds are treated less suspiciously, so if you build up N accounts that way, eventually when you launch your product, each manufactured comment looks less fake as the accounts are already "established" at that point.
This is nothing new, been going on for decades already. Guess the scope kind of expanded and the required effort went down a lot these last few years though.
7B on 15W could be any of the Orin (TOPS): Nano (40), NX (100), AGX (275)
Curious if you've experimented with a larger model on the Thor (2070)
Huh? Why would industrial inspection, in particular, benefit from lower latency in exchange for accuracy? Sounds a bit backwards, but maybe I'm missing something obvious.
The point of their comment isn't that you would use an LLM to sort fruit. It was just an illustrative example.
Again: Nobody is using LLMs to (for example) sort fruit. But there are some industrial processes that prioritize latency over reliability.
But fine - what are these industrial processes where that prioritize latency over reliability and using a LLM - as mentioned by the OP - makes sense?
They're reconfigurable on the fly with little technical expertise and without training data, that's really useful. Personally in projects for people I've found models have fewer unusual edge cases than traditional models, are less sensitive to minor changes in input and are easier to debug by asking them what they can see.
https://www.lakera.ai/blog/visual-prompt-injections
https://www.theverge.com/2021/3/8/22319173/openai-machine-vi...
The lazy analogy the other way is that developing a custom system to do these jobs is like hiring a team of experts to spend 2 years designing the perfect crosshead screwdriver that fits exactly one screw (and doesn't work if the screw starts slightly rotated) when you have a flathead one right next to you that'll work and it'll work right now.
> and inviting nondeterminism in important systems.
Traditional ML is just as non-deterministic.
> they are also vulnerable to adversarial attacks.
Typically not relevant in these kinds of cases but also this is easily a problem in many traditional ML algos.
Have you worked on things like this?
And I still haven't seen a single example of anyone actually using a finetuned Qwen in industrial inspection, which leads me to believe than nobody is actually using it for that, but some people want to use it because it's their new favorite toy. You don't need a VLM to count cells in microscopy images, or find scratches in painted parts, or estimate output from a log in a saw mill. I can see the use case for things like describing a scene from a surveillance camera, finding a car of a certain model and colour, or other tasks that demand more reasoning or description. But in those cases latency is not super important compared to getting the right output, which was the tradeoff discussed from the start of this thread.
The last thing I'd want to deal with is to have a computer say something like "You're absolutely right, it was wrong of me to classify the metal debris as food".
> The last thing I'd want to deal with is to have a computer say something like "You're absolutely right, it was wrong of me to classify the metal debris as food".
The cnn will do that potentially more often and it can be because it’s just not seen enough examples of the debris at that angle or something else equally irrelevant to a human.
Of course, but that isn't what unclear here.
What's unclear is why a 7b LLM model would be better for those things than say a 14b model, as the difference will be minuscule, yet parent somehow made the claim they make more sense for verification because somehow latency is more important than accuracy.
And then it depends on whether there is a useful difference in performance between the two.