Say you have a package that only triggers the evil activity one out of 10,000 evaluations? It'd have a 99.99% of passing validation but could still wreak havoc on a heavily used client side application.
Ditto for a package that simply delays it's malicious payload until some point in the future, after the scan has occurred? I guess this is a bit easier to test for by future dating the system clock, but again what if it's a random window? (e.g. every other day for five minutes, at 4:59pm UTC).
Having said that, obfuscated code (e.g., base64 encoded) or runtime payloads (e.g., downloaded from pastebin) will not be analyzed with static analysis, but it will tell you if the recent package version uses base64 or exec APIs in the code (file/line) for you to take a deeper look. We've some ideas on runtime lightweight sandboxing, and will eventually implement those.
...You mean, in the same manner everyone should have been auditing code all along, but hasn't, thereby creating the problem you're trying to solve?
It's alright. You're trying, and there's nothing wrong with that answer, but the people you're trying to sell a shovel to don't want to dig. That's the chief problem.
That's exactly where a tool like Packj can help. It can cut the digging time down significantly. Auditing hundreds of direct/transitive dependencies manually is impractical, but Packj can quickly point out a "risky" behavior (file and line number). Using obfuscation to hide malicious code itself is a suspicious behavior, which is flagged. After all, we have to start somewhere.
It’s not a bad idea to stake out a product that’s mature and ready to go when that happens. It doesn’t even have to be foolproof. It just needs to be earnest enough to check off some boxes and show that “best practice” effort was made.
Is this tool unix-only or just the sandbox thing?
Nevermind, found the answer regarding PyPI install:
> Packj only works on Linux.
But won't this require your target users turning off SIP [1] on Macs to enable dtrace? A lot of dtrace tools require SIP to be disabled before they can produce any meaningful output.
1: https://developer.apple.com/documentation/security/disabling...