Hacker News Reader

Comment by selcuka | Hacker News Reader

I believe this is something that automated code parsing (a la Github Copilot) could really shine. Those tools are able to explain what a block of code does, so it should be possible to catch many types of malware hidden in source code.

View on HN

If a human can't figure it out than an AI trained on human behavior (or more specifically, trained on comments left next to code) can't figure it out either. It's not like malware authors annotate their code with nice "here's where I exfiltrate all the user's secret data" comments.

That's not going to stop people from trying though. If people keep abusing the package managers like this, then publishing open source code is at risk of ending up like distributing binaries on Windows, where you need to fight dozens of virus scanners arbitrarily blocking your code for no apparent reason. Imagine opening a tab in VSCode only to see a popup warning saying it might be harmful, even though you just wrote it yourself.

This kinda happens already. I get prompted to trust stuff I wrote

That's exactly was vscode does for me.

Are you talking about this? https://code.visualstudio.com/docs/editor/workspace-trust That's different from how virus scanners work and not exactly what I'm talking about. Even Emacs does that.

That is different, vscode finally realized most people don't want stuxnet-class implicit autorun behaviour from random files and folders in your system.

I never claimed that it would be able to catch all malware, but it beats a human team reading through hundreds or thousands lines of code, for every package, after every update. I believe it can easily point out many types of suspicious additions to existing code.

A focused human will currently do better than automated tools, but actually getting a human to focus on this sort of thing is rare. Most projects I'm familiar with use a multitude of dependencies and only do cursory verification of changes at best.

An automated tool to check all depency changes for suspicious code and flag that to a human could be valuable. Whether or not that could be done in a way where wading through the false positives is worth it, I'm not sure, but it's a reasonable idea.

This is the kind of stuff that antiviruses did for ages, from signature, code simulation to heuristics.

They have a lot of expertise in this, it feels like they could branch out in finding malware in source code, instead of in binaries.

Source code has issues with obfuscation methodologies that can defeat a lot of techniques. It’s why companies are trying to build more analysis down into the kernel such as via EBPF. For example, concatenating a series of strings and characters that wind up reading from .AWS/credentials in the end is surprisingly tough to catch based upon simple pattern recognition alone, especially if it’s done in a subtle way such as with a spare buffer while doing other legit activities. So until the syscall gets issued and all substitutions resolved the user space analysis can be highly resource intensive or inaccurate

Right, code analysis to try to detect places where it reads from ~/.aws/credentials is never going to be reliable. The correct approach is not to run untrusted code in an environment where it can read your AWS credentials.

And if that's infeasible, to not run untrusted code in an environment where it has unfettered outbound access to the Internet.

Another approach is to only run credentials that are ephemeral, which is sort of what most SSO systems will do for cloud IAM. Instance profiles using IDMSv2 work as well, too. However some malware out there only needs a few seconds of dwell time to wreak some serious havoc so even ephemeral credentials may as well be the same as static credentials potentially, especially if your credentials are used to do permanent privilege escalation. All it really can do then is provide a time window of usage and make filtering through a SIEM much more accurate, which is certainly valuable for forensics at the very least and even more important in terms of law (chain of custody, irrefutability, etc).

How about flagging anything that looks like obfuscated code?

Apologies if this was not a joke, but just imagining the implications of this made me laugh harder than any comedy I've watched the past few weeks. My... umm... "scientific code" would be flagged in miliseconds :D

Binaries can be seen as "source code" for the CPU.

Isn't this impossible as per the halting problem?

It would be better to run the code in a sandbox and log all the syscalls it makes, similar to how Cuckoo's malware sandbox works.

Halting problem is only relevant if we are discussing a perfect 100% solution with no false-positives and false-negatives.

In real-life it is not of much relevance. I haven't seen a practical case in my life where I would conclude that it is undecidable to say what a function does (happy to see some practical examples if someone has some). For a theoretical program that "uploads credentials iff a sub-program halts" you would probably block it and live with a possible false-positive.

> undecidable to say what a function does (happy to see some practical examples if someone has some)

Any and all functions that involve templating/plugins/metaprogramming/etc (common in various web frameworks) that can effectively run arbitrary code and thus are undecidable. Also any function that does deserialization and thus will instantiate novel objects with their code, e.g. Python code that uses pickle will often be undecidable as its execution will depend on what exactly is unpickled.

Reply on news.ycombinator.com