Hacker News Reader

Comment by DannyBee | Hacker News Reader

While it's true that there is no pancaea, Semmle will not move the needle on vulnerability finding. This I have extensive data on.

(I mean this in terms of capability, not sudden popularity)

It would move the needle on a bunch else. It is a good tool for sure (and im very happy for them), i just think they will disappoint people by pressing this particular narrative, and wouldn't do so with a different narrative

View on HN

I'd be curious as to why you think that. Are you able to provide more detail on that claim? I have extensive experience with Semmle and my experience drastically differs from you.

With certain languages and a strong and diverse ruleset Semmle has it's strengths. In particular with native code (C, C++) and decent rules I have seen Semmle be very successful at finding certain classes of bugs.

The Datalog part is interesting! Do they have a bunch of rules to make graph queries work nicely, like Datomic pull syntax or maybe some pattern matching syntactic sugar? Is the underlying thing still an EAVT store? Is any of that information publicly available?

It looks like they have reasonable docs on their query language, in particular https://help.semmle.com/QL/learn-ql/about-ql.html#properties... has some info on the QL language.

https://help.semmle.com/lgtm-enterprise/user/help/generate-d... says "LGTM generates a database for each commit stored in a repository. Each database is a relational database that represents the structure of the codebase for a specific revision, or snapshot, of the code.", though a triple store could qualify as relational here. I couldn't find much more than that about the implementation details though.

Right. I found those docs but they didn’t look like datalog queries at all. Of course that doesn’t mean they don’t compile down to datalog :)

Could you share the details of your experience? My experience has been quite the opposite.

I have been using Semmle daily to automate much of the vulnerability discovery process and I am extremely satisfied.

We run it over millions of lines of Java code and have not yet run into scale or perf problems.

Developing custom queries and defining security invariants in a logic language is, quite honestly, a joy.

I would love to hear more about this data, as everyone I know who has used it for vulnerability finding has very good things to say. Semmle have also demonstrated its capabilities with some high profile examples.

Who have you talked to about it? Outside Mozilla and Microsoft?

I’d rather not name drop individuals or companies, partly because I don’t know that these entities want their business relationships public, but neither of the companies you named. I’ve also used it myself (you can too, at lgtm.com). I’m aware that MS is a customer, but I don’t think I’ve talked to anyone there about their experiences with it. As far as static analysis goes, which is inherently limited, it’s far better than anything else I’ve tried (which is most of them).

Fair! Thanks!

Reply on news.ycombinator.com