I think being on GitHub (and seemingly open source) gives developers a false sense of security in that they assume the code is open and therefore community vetted and that the developer has nothing to hide.
I suspect people who would know not to download and run a random binary off the internet would download, compile and run projects from GitHub.
I mean, you can use static analysis or similar, but you generally can't check every line of code for every open source lib you pull in, let alone its dependencies.
Seems that, once you decide to use open source, you are actually making a choice to trust to some extent.
Let's use GitHub as an example. We have forks, and stars. Maybe we could also have some kind of build endorsement?
How one would verify that the endorser is worth your trust, I am not entirely sure.
Maybe endorsers could eventually be rated by CVEs found in their endorsements, and that would build trust?
This would actually be an excellent LLM coding benchmark,[0] in addition to a human endorser benchmark.
[0] If nobody is already doing this, especially retrospectively, and you do, then please at least give me a shout out. :)
Pay researchers to analyse repos without any. Post results. Link to the repo with mitigation PRs.
It’s insane this isn’t the standard already
This is the problem, the best we can do is pay via exposure. But that actually ain't nothing. Not just individuals, but also orgs could then make money from private contracts based on their reputations? This should be the benchmark of trust. Could there be anything better?
I would love other people to jump in and elevate this conversation.
Sure, CVEs might not be the ideal metric. Could you, or anyone else, suggest a better metric?
If GitHub is too lethargic to do even contemplate this type of change, maybe this could be a differentiator for GitLab?
On top of that it could have forced release binary scanning via VirusTotal/insert-malware-scanning-vendor-here.
If you work for a commercial company then you should not download the code from random users on Github for free but from commercial, safe repositories where the code is inspected, tested and verified. Or from reputable large commercial companies that are unlikely to put backdoors. Microsoft or Apple won't risk their reputation by backdooring an open-source library.
Also, I did some research and the sandbox is difficult to implement because you need to stub literally every facility (because Linux was not designed for sandboxing). For example, I had to write an emulation of /proc in Python using FUSE because many apps rely on reading files there but granting them full access leaks too much information about your system and is not secure. Now think how much time you need to stub every API, including undocumented APIs like /sys, ioctls and so on.
We know we can hit the windows key and type "sandbox"? (May need to "install" it from windows features.) Right?
There are software packages that let you snapshot the files and checksums, then compare again after you've run your test program / installer / whatever.
You can make this software "portable" so you don't have to install it every time. You can copy and paste into the sandbox from your windows desktop and drives.
Obviously this isn't sanboxie or nix or an immutable file system or anything, but let us not pretend it's 1996 and "GoBack.exe" hasn't been invented yet.
For some reason the same business model has not made many inroads for higher-level language ecosystems, although many companies are trying - for example the Python Conda distribution.
Although the "repo" is a list of manifest files that include third-party download sources. So even if there is an approval process it seems to be quite vulnerable to including malware.
Edit: Example https://github.com/microsoft/winget-pkgs/blob/master/manifes...
You don't have to assume that the code is community-vetted. If a repository has at least a couple hundred stars, lots of forks, and an active pull request cadence, then you know that at least some people have gone digging through it.
If not, then that's when you should break out the sandboxing tools and prepare to check the code yourself. At least it should be easy-ish to automatically check/block everything that has the potential to open a network connection, which defeats most profitable malware models.