So how do you detect these attacks?
In this incident, we detected the packages quickly, reported them, and they were taken down shortly after. Given how high profile the attack was we also published an analysis soon after, as did others in the ecosystem.
We try to be transparent with how Socket work. We've published the details of our systems in several papers, and I've also given a few talks on how our malware scanner works at various conferences:
Very insightful.
You can't catch everything with normal static analysis either. LLM just produces some additional signal in this case, false negatives can be tolerated.
There are cases an LLM may be able to catch that their static analysis can't currently catch. Should they just completely ignore those scenarios, thereby doing the worst thing by their customers, just to stay purist?
What is the worst case scenario that you're envisioning from an LLM hallucinating in this use case? To me the worst case is that it might incorrectly flag a package as malicious, which given they do a human review anyway isn't the end of the world. On the flip side, you've got LLM catching cases not yet recognised by static analysis, that can then be accounted for in the future.
If they were just using an LLM, I might share similar concerns, but they're not.
when static analysis does it, it's called a "misclassification"
“Chat, I have reading comprehension problems. How do I fix it?”
Does the AI detect the obfuscation?
Thanks for the links in your other comment, I'll take a look!