What Makes Code Hard to Understand?
arxiv.org
arxiv.org
There is a lot of research done in this and other areas of Software Engineering. Problem is that it's usually locked away behind a huge paywall. If you're not affiliated with some university, you essentially have no access to it.
This may be true for research in other domains but as a researcher in Computer Science I can assure you that the number of academic papers locked behind a paywall is negligible in CS. Only once was did I find a paper not available and I promptly received a copy by sending an e-mail to the authors of the paper.
Like I said earlier areas like computational Medicine, Biology and other such areas run into pay walls. CS research areas like theory, data mining, NLP etc. will rarely run into such problems.
It surprises me how little we really know about the human aspects of our own field. The practical/pragmatic side of understanding how we really program, what's hard, where we stumble, what we do easily, etc is something that we're trying to address with Light Table. LT happens to be the perfect playground to do this sort of experimentation, so at the very least we'll be making it easier for researchers already in the space. But as we get further along and have a few more hands to spare, we'll start picking off research questions ourselves.
The more the merrier, right? :)
As someone who uses Visual Studio everyday, I am not surprised by this at all. Don't get me wrong, Visual Studio is one of the best IDE's out there, but sometimes it's inspite of itself.
Like removing functionality from the Text Explorer in VS 2012, which they have finally been patching back in. Dumb, dumb, dumb. If they HAD end-to-end user testing, they never would've remove the functionality in the first place.
I could easily redesign the horizontal whitespace test to show that horizontal whitespace matters.
Compare: spot the error
class FruitCollection
{
pubic:
FruitCollection()
: num_apple(0),
num_apricot(0),
num_avocado(0),
num_banana(0),
num_breadfruit(0),
num_bilberry(0),
num_blackberry(0),
num_blackcurrant(0),
num_blueberry(0),
num_currant(0),
num_cherry(0),
num_cherimoya(0),
num_clementine(0),
num_cloudberry(0),
num_coconut(0),
num_damson(0),
num_date(0),
num_dragonfruit(0),
num_durian(0),
num_elderberry(0),
num_feijoa(0),
num_fig(0),
num_gooseberry(0),
num_grape(0),
num_grapefruit(0),
num_guava(0),
num_huckleberry(0),
num_honeydew(0)
num_jackfruit(0),
num_jettamelon(0),
num_jambul(0),
num_jujube(0),
num_kiwifruit(0),
num_kumquat(0),
num_legume(0),
num_lemon(0),
num_lime(0),
num_loquat(0),
num_lychee(0),
num_mandarine(0),
num_mango(0),
num_melon(0)
{
};
};
class FruitCollection
{
public:
FruitCollection()
: num_apple(0)
, num_apricot(0)
, num_avocado(0)
, num_banana(0)
, num_breadfruit(0)
, num_bilberry(0)
, num_blackberry(0)
, num_blackcurrant(0)
, num_blueberry(0)
, num_currant(0)
, num_cherry(0)
, num_cherimoya(0)
, num_clementine(0)
, num_cloudberry(0)
, num_coconut(0)
, num_damson(0)
, num_date(0)
, num_dragonfruit(0)
, num_durian(0)
, num_elderberry(0)
, num_feijoa(0)
, num_fig(0)
, num_gooseberry(0)
, num_grape(0)
, num_grapefruit(0)
, num_guava(0)
, num_huckleberry(0)
, num_honeydew(0)
num_jackfruit(0)
, num_jettamelon(0)
, num_jambul(0)
, num_jujube(0)
, num_kiwifruit(0)
, num_kumquat(0)
, num_legume(0)
, num_lemon(0)
, num_lime(0)
, num_loquat(0)
, num_lychee(0)
, num_mandarine(0)
, num_mango(0)
, num_melon(0)
{
};
};
a = Vector(1, 2, 3, 4)
b = Vector(3, 4, 5, 6)
c = mul(a, b);
a = Vector(1, 2, 3, 4)
b = Vector(3, 4, 5, 6)
c = mul(a.x, a.y, a.z, a.w, b.x, b.y, b.z, b.w);
In all seriousness, you make some good points, but I don't think they invalidate the findings so much as they demonstrate avenues for further inquiry.
The following code is probably my favorite, "What will this program output?" example. This is taken from the Quake III Arena code in "q_math.c" [1].
Note line 561 [2]. Non-obfuscated code, and one is left wondering... just what, exactly, is going on at that line?
I understand that the point of the paper is to analyze code without having comments to help, but it serves as a reminder to me that commenting is important in helping not only other developers understand the code, but to help myself when revisiting code particulars that may have faded from memory.
I find this to be a great example when I hear, "I don't comment; the code itself is self-documenting."
552 float Q_rsqrt( float number )
553 {
554 long i;
555 float x2, y;
556 const float threehalfs = 1.5F;
557
558 x2 = number * 0.5F;
559 y = number;
560 i = * ( long * ) &y; // evil floating point bit level hacking
561 i = 0x5f3759df - ( i >> 1 ); // what the fuck?
562 y = * ( float * ) &i;
563 y = y * ( threehalfs - ( x2 * y * y ) ); // 1st iteration
564 // y = y * ( threehalfs - ( x2 * y * y ) ); // 2nd iteration, this can be removed
565
566 #ifndef Q3_VM
567 #ifdef __linux__
568 assert( !isnan(y) ); // bk010122 - FPE?
569 #endif
570 #endif
571 return y;
572 }
Still, I find that with 90% of the code I write for work, it's not so difficult to come up with variable and method names that make it pretty clear what's going on. I take the approach that I'll add comments anywhere I think that naming might not be enough, but that very rarely happens.
Granted, I'm not writing low-level, math heavy code. I think it really just depends.
This can even be expanded to more complex single-purpose functions that are not exposed directly to any external APIs. For instance, if you have a functioned called restart_thread() which first tries to stop a thread, then stats a new one then you might be safe.
However, as soon as you leave the realm of the obvious, comments are a must. Clearly in situations where you're trying something clever, like that Q_rsqrt function, missing comments will boggle all but the most specialized professionals.
Unfortunately people often forget about another important type of comments; those that explain what function this piece of code serves in the program. I've lost count of the number of "Context"s or "Interface"s or "get_data"s I've seen when trying to read someone's code. In those situations a single line like "This context links the rendering pipeline to the physics simulation" would go a very long way.
Instead when I see this sort of code it will either not have any comments at all, or it will be a dissertation that all the theoretical uses of the piece of code (Omitting what it's actually used for in the current context).
Why not rewrite your code so there is a rendering_pipepline variable and a physics_simulation variable and a function called link? Then the comment is redundant.
That's what refactoring for self-documentedness is to me.
Personally, I like having some sort of structure in memory that allows me to access and query contextual information. It's amazingly useful if you are trying to link a variety of different elements. In fact it's practically unavoidable if you are writing highly dynamic code. Trying to manage this sort of system with a few variables and functions would become a nightmare.
In fact, those dynamic situations are where comments describing layout are absolutely critical. In these situations you may be using your class as a generalization, and tracking the full extent of this sort of interaction could take a ridiculous amount of time.
static uint32_t toRepresentation(float x) {
uint32_t rep;
memcpy(&rep, &x, sizeof x);
return rep;
}
static float fromRepresentation(uint32_t rep) {
float x;
memcpy(&x, &rep, sizeof x);
return x;
}
static float rsqrt_linearApproximation(float x) {
uint32_t xrep = toRepresentation(x);
uint32_t yrep = 0x5f3759df - xrep/2;
return fromRepresentation(yrep);
}
static float rsqrt_newtonRaphsonStep(float x, float y) {
return y*(1.5f - (0.5f*x)*y*y);
}
float Q_rsqrt(float x) {
float y = rsqrt_linearApproximation(x);
y = rsqrt_newtonRaphsonStep(x, y);
y = rsqrt_newtonRaphsonStep(x, y);
return y;
}
All that said, I tend to write a lot of comments. Much of the code I write professionally is math library code, and it’s not uncommon for me to have a ratio of several paragraphs of error analysis or correctness proofs to a few lines of code.
This is a floating point number and some deep juju bit twiddling is happening with it, it's not just being divided by 2. The exponent is being effectively divided by 2 and the fractional part of the mantissa is also being effectively divided by 2. Except in the case where the representation of the exponent ended in a 1, in which case that 1 is shifted down into the fractional part of the mantissa. Exactly why this is ok and doesn't hurt the approximation is a subject that could fill a report.
The constant simply accounts for the bias in the floating point representation and balances the error over the domain of interest. Again, not magic.
There is no "bit-twiddling" at all.
You're using bit operations on a data type with a non-trivial bit structure. That's pretty much the definition of bit-twiddling. The fact that blunt bit operations on such a finely-defined structure can correspond to useful mathematical operations is non-obvious.
It’s using integer operations on an encoding that has a meaningful (approximate) interpretation as an integer.
Emm, by definition it does.
At one point I was browsing through a crypto-library (I forget which one). Almost all of the crypto functions used short variable names, and had only a one line comment. That comment is a citation for the paper that the algorithm came from. Ignoring the fact that the code authors were not the source of the crypto (so putting the explanation in the source may be plagourism or such), source code is not a good format for this type of thing.
I made this mistake too. In fact, even after reading that most people wrote [8] instead of [8,9,0], it took me a solid minute or so to figure out how anyone could possibly think the correct answer was [8,9,0].
This is something I've noticed happens very often while debugging. I make certain assumptions about the code and even when staring it right in the face it's hard to see where my assumptions differ from the code as written.
My assumption in this case was "why would you calculate the between lists if you weren't going to use them right away".
EDIT:
I also made the mistake, but in their defence the between lists are used straight away: They're printed. But the code seems to do three unrelated things in a row, so I'd expect an implementation like their one to have three separate function calls if there's no dependency between them.
def between( a, low, high ):
r = []
for x in a:
if low < x < high:
r.append( x )
return r
def between( a, low, high ):
return [ x for x in a if low < x < high ]
def between(numbers, low, high):
winners = []
for num in numbers:
if (low < num) and (num < high):
winners.append(num)
return winners
But isn't it obviously returning a list? It's a list comprehension!
I disagree, as someone with low experience with python, I found their example much more understandable than both of your examples.
Even if I'm wrong, how often is it an expert, that is not yourself, left to maintain your old code?
Sorry but descriptive names are clarifying what the code is doing, or are not descriptive.
Generally speaking this is true. However, when you are debugging, this no longer is true. When variables say what they are supposed to be, then it is more difficult to see that they are something else.
If you want, you can subdivide this further and more clearly define the concepts used in the between function. But we don't generally choose to do that if the result is already a scant 5 lines.
Point: you should also take into account good modular design, something you have control over.
The very plurality of `numbers` conveys so much more meaning than `a` in the method signature. Is that a Python idiom, to use `a` to mean array as a parameter? What happens if you don't use `a` until line 23 of the function, far from its original declaration?
Being someone who likes reading random code in random languages, I find single or 2 letter variable names a horrible curse to the uninitiated of a problem domain. It means you have an extra step figuring out a program, what the abbreviations even mean in that particular domain.
The only exception being when it's the standard way of using the syntax of a language (i in a for loop for example).
Obviously these code examples are trivial, but when you start getting a slightly longer function, it starts becoming hard to remember what's what half-way down a function or where they even came from if they're just virtually meaningless single or dual letters.
Then you have horribly violated the previous poster's note that the more local a variable is, the shorter its name should be.
I find single or 2 letter variable names a horrible curse to the uninitiated of a problem domain. It means you have an extra step figuring out a program, what the abbreviations even mean in that particular domain.
It is a trade-off. If you master that domain, you'll be using that all the time, and you'll get the savings all the time.
You need to amortize the cost of memorizing the notation over the expected usage of that notation. Programmers, by profession, have to use a lot of notations from a lot of fields, and therefore benefit from using notations that require a minimum of memorization. But that is not always the right trade-off.
As I pointed out in https://news.ycombinator.com/item?id=5157539, The rule is not that you need long or short variables. You need meaningful ones. A short variable name is inherently ambiguous, which can lead to confusion and mistakes. Thinking through anything with a long-variable name abuses your working memory, limiting how complex your thoughts can be.
2 * cows ^ 2 + 3 * cows = 48
instead of 2 * x ^ 2 + 3 * x = 48. In the history of the mathematics there was a time when x didn't exist, just "cows" - you weren't able to write an expression involving "the one unknown, no matter what it's supposed to represent." That was very long ago, still a lot of programmers today don't recognize the power of abstraction.
For example, consider the parents original code: def between( a, low, high ): r = [] for x in a: if low < x < high: r.append( x ) return r Even as someone who 'likes' small variable names, the r make that difficult because of the all distance between its definition and use.
Compare that to this haskel implentation
between a low high = filter f a
where f x = x>low && x < high
(Actually, I would probably use 'as' instead of 'a')
x = [..]
y = [..]
between = lambda ls, low, high: (n for n in ls if low < n < high)
x_between = between(x, 2, 10)
y_between = between(y, -2, 9)
xy_common = set(x).intersection(y)
Personally I find declarations about properties, relationships, and categories of things easier to understand than descriptions of the tedious processes that calculate them. The former give me the ability to reason about the program elements in a logical way. The latter requires me to become a stack machine and execute the program in my head... a process that is error-prone and full of false assumptions.
eg: how many people believe that "all arrays are just pointers," in C? When is an array not like a pointer?
Research like this is good. Are there any studies that go beyond trying to trick programmers with trivial programs in the Algol family and look at the difference in performance between categorically different styles? I mean languages that are declarative vs imperative vs functional vs concatenative in nature. I think that would be very interesting to read.
updated for clarity
Ok I know this wasn't your point but I got stuck on this and am just curious to know: what you were thinking of here? One thing that occurs to me is that in a recursive function you will run out of stack space a lot faster if you are using arrays rather than allocating from the heap. I don't think that is what you were referring to though, hence the question.
One area where I think there is a high dissonance is in how C uses the same syntax for defining arrays, indexing into arrays, and referencing pointer offsets. Other examples are the array decomposition rules, array function parameters, etc. Even experienced programmers get tripped up by them:
int a[10] = { 7 };
int* p = a;
assert(a[0] == p[0]);
assert(sizeof(a) == sizeof(p)); // FAILS1: "between". Experienced programmers were more likely to incorrectly assume that the results of earlier calculations would be used in a later calculation.
2: "counting". All participants, regardless of experience, were more likely to assume that a statement separated by vertical whitespace was outside of a loop, when it was actually still inside the loop. (Note that the programming language is Python, which doesn't have a loop termination token.)
3: "funcall". Whitespace between operators (e.g., 3+4+5 vs. 3 + 4 + 5) had no effect.
4: "initvar". No interesting result. (This one seems to have been mis-designed.)
5: "order". Respondents were slower when functions were defined in a different order than they were used. Experienced programmers didn't slow down as much.
6: "overload". Experienced programmers were more likely to be slower when faced with a "+" operator used for both string concatenation ("5" + "6") and addition (5 + 6), rather than used for just concatenation.
7: "partition". There was a result, but I don't think we can draw meaningful conclusions from it.
8: "rectangle". Calculating the area of a rectangle using a function that took tuples (e.g., "area(xy_1, xy_2)") took longer for programmers to understand than a function that took scalars (e.g., "area(x1, y1, x2, y2)"). Calculating the area using a class (e.g., "rect = Rectangle(0, 0, 10, 10); print rect.area()" took the same amount of time as using scalars, despite being a longer program.
9: "scope". No conclusive result.
10: "whitespace". Using horizontal whitespace to line up similar statements had no effect on response time. (There was another result relating to order of operations that deserves further study, but it wasn't thoroughly tested here, so I don't think it's conclusive.)
Note that all programs were exceedingly simple (the longest was 24 lines), so be cautious applying these conclusions to real-world work.
Why can't/don't we have more tools along the lines of App Inventor [1] that provide a visual structure for programming?
The vast majority of time I've spent writing anything has been spent struggling with specific syntax, not any particular function of the code.
Coding feels like trying to manipulate a clear model that exists in the machine by spelling my intentions out one letter at a time while peering through a straw.
While a representation that uses graphical cues in lieu of textual ones may be easier to learn and understand initially, the disadvantage of not being able to use the thousands of utilities for searching, refactoring examining code are then less useful, if not entirely useless.
P.S. In a way, idiomatic structure to code puts a graphical representation on the textual form. Programmers learn to recognize this. Python goes as far as to enforce it.
Yes, I completely agree.
To go off on a bit of a tangent, I wonder if it's just a matter of the right technology and implementation? Is there an analog to the rise of touch interfaces waiting to happen in the way we record, manipulate and relay information?
>While a representation that uses graphical cues in lieu of textual ones may be easier to learn and understand initially, the disadvantage of not being able to use the thousands of utilities for searching, refactoring examining code are then less useful, if not entirely useless.
That's a good point. I was thinking of this purely from the standpoint of creating as an individual, not maintaining or sharing.
>P.S. In a way, idiomatic structure to code puts a graphical representation on the textual form. Programmers learn to recognize this. Python goes as far as to enforce it.
Certainly, learning that word (idiomatic) was a boon to my understanding of code.
Someday, I do think there will be tools that let non-programmers create things that only programmers can create today, but by that time, I think there will also be _other_ tools that let the "real" programmers still do the most powerful stuff. Maybe no typing though.
It's like smileys. In the beginning it may be nice to have a pop-up "insert smiley" list, but in the end it's easier just to type >:-O or whatever you're trying to convey than find it in a list.
Admittedly the more verbose programming languages do end up being fairly IDE-dependent, so they get a lot closer to the visual programming you talk about.
See also: http://www.catb.org/esr/writings/unix-koans/gui-programmer.h...
DISCLAIMER: I haven't used AppInventor
It's ironic that the authors of this statement can't be bothered to typeset their opening quote marks in the right direction-- a typical noobie mistake in TeX.
The second question is: ''How are programmers
[0] http://www.maths.tcd.ie/~dwilkins/LaTeXPrimer/QuotDash.html
Unless I'm losing my mind, the correct answer for the code as stated is 6 * 5 * 4=120, not 60. It would be 60 if f (x) = x + 4.
Thanks to you and a few others, I'm making corrections and will be uploading a fixed version of the paper soon.
this statement feels tautological, I don't think there's a single correct set of expectations
My experience bears this out. In most of my professional life, I've abided by coding standards, even if I consider the standards imperfect. That's because of the very real improvements to the team's productivity when code does what it looks like it does.
I guess the biggest hurdle is that people essentially think "functionally" not procedurally
Because we fundamentally think x = x2 in terms of "the value is being doubled" without thinking of all the steps needed and everything that can go wrong there.
The article is certainly interesting, from what I can quickly glance, naming is very important
With the same logic, C is functional because you don't care about the bit fiddling in x++...
Yes, x = x*2 is not (y = x*2 would be better)