Ask HN: A better metric than lines of code
Anybody disagree and think the opposite is true; that a simpler syntax usually means more thought?
Anybody here ever heard of somebody doing something like that?
Anybody disagree and think the opposite is true; that a simpler syntax usually means more thought?
Anybody here ever heard of somebody doing something like that?
Having written a lot of Python it is now frustrating to write C++, mostly because of the verbosity. Instead of thinking about the problem I'm trying to solve, I'm thinking about how to deal with the language or necessary data structures. Python just gets out of your way and lets you solve the problem at hand quickly and concisely.
I don't think more thought goes into a line of Python code, and if it does, it's productive thought. It's not the "what's the 6th argument to this function?" or "is the type of the 3rd argument to this function really just a typedef for a float?" type of my-language-requires-me-to-jump-through-hoops thought.
To me, all that verbosity means more time thinking about the code and less time thinking about the problem.
In your example, the C program would appear to have been a more productive use of time, but in reality, the Python program was - even as it was much smaller.
http://shootout.alioth.debian.org/
I feel relatively strongly that LoC is almost entirely worthless as a metric. Even compressed source code size makes me uneasy. Shameless self promotion: I wrote a blog post which appeared on HN just the other day about better ways of comparing programming languages.
http://news.ycombinator.com/item?id=766841
In short, I disagree that there is almost any relationship between mental effort and verbosity.
That's just one argument. Similarly, if someone was going out into the fields with a shovel, it would be a bit tragic if he ended up digging it all by hand when there was a perfectly good tractor nearby.
So go ahead and write hobbyist software in whatever you like, but if it's going to cost other people time and money, then it is a worthwhile discussion.
By doing that over time programming languages evolve. If we didn't compare programming languages you'd be programming ALGOL right now and the whole web things (much less the internet) would have never happened.
For example - http://shootout.alioth.debian.org/u32/benchmark.php?test=spe...
The most meaningful test of the length of a program is not lines or
characters but the size of the codetree-- the tree you'd need to represent
the source. The Arc example has a codetree of 23 nodes: 15 leaves/tokens
+ 8 interior nodes. How long is it in your favorite language?
So instead of measuring lines of code, you measure the size of the AST. Works well for Lisp, at least; not sure how well it generalizes to other languages.I can solve a problem in 1 line of buggy code. You have to write 1000 lines, but it is mathematically certain that yours works. Whose is the better implementation? Mine might be O(n^n) in the worst case, while yours is O(1).
See, I'm talking about specific programs, but these sorts of comparisons extend fairly naturally to programming languages too.
That you can contrive counter-examples does not invalidate the metric for use in the normal cases.
A small line count / compressed size / AST node count is probably a good indicator of this, but I'm not convinced it's exactly the same.
I much rather would like to see metrics which actually involve the complexity of the code involved, that is, a measure for 'what the user has to keep in his head'. The cyclomatic complexity (~#of paths through a method) is a good example of a step in the right direction in my opinion. I just don't know how one would formalize the complexity of a framework, for example, where one has to remember 'ok, I need to implement this, this and that method in my class and these contracts have to hold' and such :)
Some people use LoC without comments or blank lines.
In general these measures can be useful indicators but have to be taken with a grain of salt.
Also take a look at this Martin Fowler's article http://martinfowler.com/bliki/CannotMeasureProductivity.html
If this is true, then the implication is that when you can implement the same functionality in less code, it will also probably have less bugs. So, if we look at opportunities for bugs, I think lines of code is relevant.
For example, let's say I was writing in language X and the average error rate was 1 error per 10 lines. I write 100 lines of code (because I'm very thrifty with my code!) and so my program has 10 bugs.
Now, I write in language Y where the average error rate is 1 error per 20 lines. I write my program in 150 lines, because Y is a bit more verbose.
I'm sure you can do the math. Would you seriously try to tell me that using lines of code is a relevant metric for making comparisons between programming languages?
I just made up my estimates arbitrarily. I think the difference in error rates between a language like PHP versus a language like ML would be even more dramatic.
I agree the metric is coarse, but I think it's a decent stand-in for things that matter. More lines of code tend to mean more concepts unrelated to the problem you're solving, more assumptions and a greater cognitive load. All of these introduce more opportunities for error. Consider the problem of concatenating a sequence of text, and putting commas inbetween each distinct entry. In Python:
concat = ','.join(texts)
In C++, this would be string concat;
for (list<string>::const_iterator i = texts.begin(); i < texts.begin(); ++i) {
concat += *i;
}
There are more opportunities for me to make mistakes in the C++ code. I've introduced more concepts (using iterators to access a sequence, using iterators to access a value, building a string) which increase my cognitive load. Hence, I'm more likely to have more bugs in this code, despite solving the same problem.I don't even want to write the C code to do this right now, because I would have to deal with the following concepts: allocating sufficient memory for the C-style strings and ensuring I'm not accidentally overflowing that memory; determining what kind of sequence texts is (native array or my own linked list?); determining how to iterate over that sequence, and how to "add" strings.
I'm assuming that in a fixed amount of program text, the number of bugs across all languages is relatively constant. More text means more concepts and more opportunities to be wrong.
I think your point is that for a given problem, different programming languages will yield solutions with variable number of bugs. In a language that is simpler to express the solution, it's more likely to be correct.
These two points are compatible. I'm fixing the amount of program text, you're fixing the problem.
Module LoC | Error Rate per 1k LoC
63 | 1.5
100 | 1.4
158 | 0.9
251 | 0.5
398 | 1.1
630 | 1.9
1000 | 1.3
>1000 | 1.4
This appears to show no such linear relationship between error rate and LoC within a single language, with a fixed amount of program text.This report appears to suggest an observed average error rate of 18 defects per 1000 lines of code:
http://www.lanl.gov/projects/CartaBlanca/webdocs/PhippsPaper...
and the same programmer generated an average of 6 defects per 1000 lines of code in Java.
This is just a random sampling of error rates I could find quickly with a google search and a paper I already had on my desk, however the fact that three samples from three projects (two of which were from the same programmer) suggests that there is likely to be variation both in terms of the (fixed) amount of program text, and the problem between languages.
Although the equivalent of the Python code would be: QString str = texts.join(",");
Base on the truism that every line of code has a potential bug, even if the line is cruft, then the length of the codebase is proportional to the potential bugs.
It's a simplification, obviously, but LOC is useful in this respect.
The main problem with LOC IMHO is people try and use it for the wrong purpose.
"Measuring programming progress by lines of code is like measuring aircraft building progress by weight." - Bill Gates