Hacker News Reader

Comment by mister_hn | Hacker News Reader

I don't understand trying to use the trunk version of a under-development compiler. Would you use the binaries built from an unstable compiler in production?

Given that here we talk about SQLite, what's the advantage to use clang 11 instead of a previous version?

View on HN

You're looking at the wrong side of sqlite usage. If you develop sqlite, why wouldn't you test it on every compiler/version combination you can get your hands on? Whether someone uses it in production is irrelevant.

Because you'll spend a lot of time chasing down bugs that are actually issues in the compiler. Unless you're trying to vet the compiler itself, this won't improve your software but cost you a lot of effort.

If you’re SQLite, you generally do care about popular compilers converting your code into something wrong because you care about your code doing something wrong in general. If there was a bug in Intel processors that made SQLite fault randomly during tests, I’m sure they’d be interested in that too.

Yes.

Two things though:

1. Hopefully none of your users run an unreleased clang from head, so you're not using anything a user will run into.

2. Consider if it is it worth the effort to validate every commit pushed to head, or whether you'd get more out of your investment if you only tested releases that received some more vetting (e.g. nightlies).

I don’t think SQLite is literally vetting every commit. (Although kudos to them if they are!)

> 1. Hopefully none of your users run an unreleased clang from head, so you're not using anything a user will run into.

Yes, but clang eventually gets released. You want to catch bugs in clang and get them fixed before that release happens.

Yes, but Clang is many months away from a release.

It is not the same as playing with a release candidate.

It's an automated fuzzer, it caught a bug very early, but the report got sent to SQLite instead of Clang. Probably because OSFuzz doesn't handle this distinction. (Though it might should. Or maybe usually new compiler versions catch problems in old code a lot more frequently than it catches compiler bugs. Dunno.)

On the other hand, you might catch either program or compiler bugs before it's too late and it has shipped to production. I think this would improve the software, and I'm also not so sure a lot of time is spent on this, and it wasn't even something caught by SQLite devs, but rather an external project which SQLite decided to work around for now.

There was a chance this miscompilation would not have been found until after the release of clang 11. Surely both the clang and sqlite developers benefit from people trying pre-release versions and reporting bugs?

LLVM follows an open model for commits. Therefore, their trunk being broken early in the release schedule is not a surprise. They are months away from a release.

Whether that development model is good or not is another question, but it is what it is.

If I read the article right, the SQLite people don't use clang 11, but OSSFuzz does and someone ran it against SQLite and filed a bug. This bug turned out to not be a big in SQLite but in clang 11.

I'm not sure why OSSFuzz uses a prerelease compiler, but maybe it's purposefully for this reason, to add some extra surface area for finding compiler bugs?

there's 3 versions of this comment floating around and another 2 floating around asking why this wasn't discovered earlier

combine both to find zen koan, test early and often

sqlite testing is pretty involved, and impressive. There are articles written on this, such as: https://www.sqlite.org/testing.html

I believe Chromium uses the under-development trunk of Clang, and bumps it forward every couple weeks - and that’s probably one of the most heavily used code bases in the world.

https://chromium.googlesource.com/chromium/src.git/+/master/...

OSS-Fuzz wants to get the new sanitizer features fresh from the oven I imagine to fing new bugs in fuzzed sw as early as possible.

Not working in sqlite, but postgres. We've found numerous bugs in gcc, clang and postgres by building / testing with unreleased compilers. And found usability issues with new compiler warnings.

That doesn't mean we use such compilers in prod.

The Google people do use them in production, it's the clang version they build Chromium with.

Why?

Reply on news.ycombinator.com