John Carmack on Static Code Analysis

Both produce high quality output in their domain. Fortify is widely use in the AppSec world.

Any time you have dedicated sales staff selling stuff to organizations with a sales process that needs to get buy-in from executives, your prices have to go way up to cover that cost. (Doubly so because the sales process is expensive for you, and your price point lowers your success rate.) Unfortunately many organizations have a rule that every purchase over $X has to be approved by management. Thus there are 3 price points that you can find proprietary software sold at:

1. Small enough that a developer can charge it to a corporate credit card.

2. Relatively modest recurring payments, individually small enough to pass on a corporate credit card.

3. Starting well into 5 figures.

It is certainly true that many of the static errors that Coverity and friends can find aren't possible in Haskell because it won't compile, but I'd like to see a lot more public experience with Haskell and larger projects to see what the result would be for other problems. A lot of what we now know about how C++ should be written is the result of decades of hard experience, and who knows what we might find in Haskell with as much experience.

http://www.altdevblogaday.com/2012/04/26/functional-programm...

(I prefer Haskell, it's much more beautiful. But that's more of a religious issue. OCaml is still way better than, say, C++.)

BTW, I once considered this (this being a language with Haskell's strengths, suitable for systems programming) a possible idea for a thesis, but at least for now it's put on hold. If anyone has any comments, I'd really love to hear them.

A couple of years ago I ran across an article from Coverity about the challenges that they had getting people to use their tool. One of the most interesting was that the tool would find real bugs, but the developer wouldn't understand that the bug actually was real, and would say, "This tool is crap." Then they would not get a sale.

They had this problem particularly strongly with race conditions. There are a number of checks that they took out because, even though they were finding real bugs, developers were convincing themselves that the bugs were not real even though they really were.

It really does not help that the developers who are asked to evaluate are likely to be the same people who made the mistakes in the first place, so all kinds of defensive behavior are to be expected.

(Full disclosure: I work for HP Fortify.)

- I like that the contracts are right there in the code and not in a separate file.

- The actual Contract methods only run once upon loading each class, so they don't cause a performance hit. I'd like a way to make them all noops (so they don't add any instrumentation) when running in "production".

- My thought is that if you can define pre- and post-conditions as you're doing, then you can cut out a lot of the checks in your tests, so that as long as the tests exercise the code, you can rely on the post-conditions to do quite a bit of the validation. Plus you're validating outputs all over the place constantly, so you'd probably catch edge cases you never even expected.

- I'm disappointed that your system seems to offer little more than type checking (unless I'm mistaken). I'd like something that gives me true pre- and post-conditions: basically passing a block of Ruby code for each one, with common idioms abbreviated so I don't need to write a whole block if the conditions are simple.

Good luck on the project!

It does impact Microsoft more, because Microsoft has taken on responsibility for games to be good (well, meet certain baselines for various definitions of good) through their TCRs.

[1] http://www.sonarsource.org

Imagine a tool that could analyze this:

  //sql table T having columns int x, string y

  //php
  $data = sql_fetch_from_column(T)

  //javascript
  var d = <?=json_encode($data)?>
  for(a in d)
      var b = a.x / a.y;  //Type error, int division by string not defined

    public Statement ageFilter(int minimalAge) throws SQLException {
      Connection conn = getConnection();
      String sql = "SELECT name FROM people WHERE nmae IS NOT NULL";
      if (minimalAge > 0)
        sql += "AND (age >= " + minimalAge + ")";
      return conn.prepareStatement(sql);
    }

I also find getting a good set of rules for phpcs is extremely helpful in PHP. You can use its sniffs to catch past mistakes and enforce consistent coding standards.

That's because `for(var i=0...` is completely bogus. What if you have two of those loops? Are you going to put `var` in the first one, but not in the second one? Are you going to use 'i' in the first one and 'k' in the second one?

What if you need some refactoring and those two loops switch places?

It's fine if you like Java, but JavaScript is different. When in Rome, do as the Romans do. I also wish there would be block scope, but merely pretending there is doesn't make it so.

It's better if you declare all of your variables at the very top of the innermost function, because that's where they effectively are (thanks to hoisting). It's better if your code reflects how it's actually working, because that will always make it easier to comprehend.

Once there is `let`, you should switch to Java's "declare on first use" rule and never use `var` again.

And then fire that person. (Or rather, see if big commits introduce more bugs than short ones, or whatever else you might find.)

Here's a some for asp.net, http://stackoverflow.com/questions/4419253/seeking-a-code-an...

For .NET you have Code Contracts[2], this requires you to annotate pre- and postconditions for your functions but after that it's for example impossible to pass a possibly null into a function without compile time error. It can even do range-check of numerics after calculations etc.

[1]http://www.icosaedro.it/phplint/

[2]http://research.microsoft.com/en-us/projects/contracts/

Googling "$LANGUAGE lint" will probably find what you need.

Well, I wouldn't say nobody. I, for one, play/watch Quakeworld and Quakelive on a regular basis.

Do you have actual sales numbers to back that up or is this based on anecdotal evidence, or is it just because you didn't like the games as much :)? Publicly available internet numbers show that Doom 3 sold 3.5 million, and even Rage sold 2.31 million, making Doom 3 extremely profitable and probably putting Rage a little higher than break even given the long development time and much larger team size.

Two of the Quake games are listed on wikipedia's top selling PC games and sold 1.7 and 1 million. (http://en.wikipedia.org/wiki/List_of_best-selling_PC_video_g...)

Although none of these numbers are 100% accurate unless they are wildly off I wouldn't call any of id's games except Rage unsuccessful.

Okay, yes, Blizzard games are nice and stable and polished too. But have you been reading the Starcraft development articles recently? They've been talking about how much of a nightmare the entire experience was, with major bugs until just before launch. From what I gather, one of the senior developers was running around fixing bugs across the entire code base, and personally reviewed and fixed up an astonishingly large percentage of Starcraft code.

One can wonder: if they were using static analysis, could they have shipped the same game earlier?

Quake was/is one of the best games of it's genre, ever. It sold 1.7 million copies, as a shareware release.

Doom 3 sold 3.5 million copies. It wasn't a commercial success?

Not to mention the games that were built on top of id technology, games like Half Life, Call of Duty, et al