Comment by eyelidlessness | Hacker News Reader

Comment by eyelidlessness | Hacker News Reader

Parent Full thread

I don’t think the intent of this tool is to generate comments which you’d then embed into the code it describes. I think it’s meant to explain, in plain language, what the actual behavior is (for whatever confidence level you might assign to “actual” and “is”).

To your point about the utility of code comments describing the behavior this way, I agree it’s probably much more valuable for beginners. In fact when I’ve mentored early programmers, I sometimes ask them to write out essentially prose like this in comments before writing a single line of executable code.

Now, I’m far from a beginner. I’ve been considered a senior engineer long enough that friends discourage me from disclosing the amount of time, for fear of age discrimination. I can absolutely see the potential of this tool as part of my IDE. I’m on vacation now, but when I return to work I plan to take it for a spin as an aid for refactoring areas of code which clearly work as intended (well, for the most part) but the actual behavior and intent is much less clear.

Here’s why I think it’ll be valuable for refactoring: it can help limit the amount of mental context switching necessary to build a mental model of what the code does. I often find myself trying to produce prose much like this for my own reference, but I end up losing context as fast as I acquire it as I follow references into their respective rabbit holes. Having the tool do that for me can help me stay in a single area of focus. It could also be a useful reference for adding and improving type definitions, maybe even regression tests.

The best part is that it doesn’t, from what I’ve seen, do anything besides populate ephemeral annotations. It doesn’t try to write code or automate anything other than producing a narrative. Like at least one other commenter, I’m skeptical about the reliability of that. But unlike that commenter, I’m willing to take the risk… probably because I’ve learned to be skeptical of my own reliability performing the same task. If my instinct is right that I can use this tool the way I hope, I’ll still scrutinize it for accuracy. But that’s potentially much better than having only one imperfect, meat-based computer doing the work.

To be clear, I'm not really arguing that the intent is to embed comments. But I'm just going by the "validateSignature" example they give in the video. All that does is generate a 1-1 mapping of each code line to an English comment. TBH I would expect a senior developer to just be faster reading the code.

Perhaps there are some other examples that explain more of a "structural understanding" of the code, but I'm skeptical without more evidence.

> TBH I would expect a senior developer to just be faster reading the code.

TBH, I would also expect this tool to be a lot more affordable than a senior developer for language X.

Plus, chances are good that your senior developer for language X will barely scrape by as an extremely junior developer for language Y that you also use in some capacity, and that there is a problem which involves stuff written in both X and Y. Also, many languages have horrid gotchas, where innocuous looking code does something quite unexpected. Like the C++ classic log->debug("Timeout: %d", config["timeout"]). If some tool where to actually tell you "Writes a debug log with the current value of timeout, if it exists, or otherwise sets it to 0", well that would be pretty useful for someone who has only superficial knowledge of C++.

> TBH I would expect a senior developer to just be faster reading the code.

That specific code? Sure. Real world code that’s gotten more convoluted over a decade of maintenance?

I have a dozen pages of notes I’ve taken on the apparent behavior of a single function, all of its call sites, all of the known possible kinds of states it might encounter, all of the categories of implications those states might carry and the kinds of downstream effects its return value might have. It’s not even a particularly large function as far as those go. The notes aren’t even complete, if I had to guess they’re 1/3 there. To be complete they’ll span not just several modules but cross package and language boundaries and even repo migrations. Not because the function itself is that complex (although it’s much more complex than I’d prefer), but because the universe of its inputs and usage is enormous and the history which produced it is long and just as convoluted. And, importantly, because the space of very similar behaviors and functions I’ve discovered has grown each time I peel a layer of the onion off.

Now when I go back to work, given I get to continue untangling this thing, having a tool which helps even explain this universe would be a godsend for actually acting on it in a reasonably safe way. Especially if it does so with consistent language. I could literally shove it into a database and query it, or do all kinds of other analysis. I can’t do that if I’m spending all of my energy trying to just describe the thing in my own words with incomplete understanding. I mean, I can. The documentation I’m describing literally began with me wrapping up the previous day’s notes with “humans built this, a human can understand it, model the damn thing”. But on what timeline? And is it a good use of my time to do it when a machine could probably do it more quickly and hopefully just as reliably at scale?

I’d be happy to share the evidence if my instinct proves out.

I'm sympathetic but it seems like you are wishing for something that this toll can't do.

Really? The thing it does in the video is almost identical to something I’ve spent days doing manually. Even if it's a small part of the task, it does it in about a second. Why do you think it doesn’t do what I think it does?

The problem with complex code is usually branches / conditionality.

Also complex indexing or expressions.

This stuff very rapidly stops being expressible in narrative form because the details can't be "compressed".

The explanation starts to look like legalese and you realise the code itself is the more compact and legible expression.

Ideally an autocommenter will be able to take a hand matmult and write "compute the surface normal" (this is an example where apparently high local detail can map to good conceptual compression) but actually that is the best possible case.

When it comes to important but somewhat arbitrary stuff like shipping or tax rules, english will not speed comprehension and is less good than a tageted dsl or well thought out tables.

The core problem is that "sometimes there is no simple explanation".

That’s all fine and good but the code exists already and even if I think its domain is an exceptionally good candidate for a DSL I can’t snap my fingers and make it morph into what I wish it was. And I can’t get there gradually without understanding what it is.

I’m well acquainted with its complexity but I need to articulate it coherently to justify changing any of it, not just to myself but to my team and stake holders. I’m not looking for a simple explanation. I’m looking forward to having tools help me navigate a very complicated explanation.

I also don’t understand the motivation here or upthread to splain the problem I’m solving away as some nihilistic unsolvable thing when I’m saying I see real prospects of this helping me. I mean, you’re welcome to whatever nihilism you see fit in your course of action but I personally don’t benefit from being told things I find potentially useful are not useful for me, actually.

More power to you if it's helpful to you.

So a better "output" might be a kind of chatbot that will answer questions about that particular piece of code. Basically it understands every detail including branches and you can ask whether a specific assumption is correct.

I don't think we're there yet (because complicated side effects etc. are a thing) but I would rather have such a solution than some text to read, after all I'd be using such a tool for code I cannot comprehend just by reading it.

It's possible that if you changed from translating the code to prose by hand and translating it with the tool, you would find that the "value" of that translation would be reduced. Basically, I think reading the code and mentally analyzing it enough to write to prose is doing most of the work, and when you read the prose, it's only a reminder for the deeper analysis you've done in your head.

It’s possible that you’re being a well meaning jerk. I said why I’d value it. Why not just let me value it?

Yeah, I find the signature example kind of unimpressive because it can just pick up the meaning of the code from the function name and variables, which an experienced human can do just as easily. If it could get "validates a signature with secret key" when all the variable names are obfuscated it might be useful in some code bases.

Why could you not create a comment block above the code section in question and fold it. All IDE's nowadays support folding, including comment block folding. To see what a section of code does, you unfold the comment block above it, read it, then fold it back.

But if it can be generated on the fly, why persist it to the code at all?

I guess if the AI is good enough, your point makes some sense. It's a good programming habit to doc your code as you write it. If you do this in a structured way and code folding is used the comments should get out of the way from the main code.

To save computational resources could be an answer.

Reply on news.ycombinator.com