Hacker News Reader

Comment by seunosewa | Hacker News Reader

I use a different LLM family to review commits and write detailed descriptions. If a commit was written with Fable/Opus, I use Sol/Astra to write a well reasoned commit message. If the message doesn't match my intent, then that triggers a manual review.

View on HN

If the second LLM is just describing the content of the commit doesn't that defeat the purpose of the description, to capture the context that doesn't make it to the code?

The second LLM is prompted to actually research the code, not just the diff, with fresh eyes to figure out what it does and why, before writing the commit message.

But the code doesn't always have the answer to "why". At best that means the commit-writer has 'guessed' at why.

This would have an even greater loss of the “why” context the author is describing in the piece.

The human knows the "why", so they can correct the commit message if it is wrong or incomplete. Most of the time, though, they don't have to do that.

This makes no sense to me. The code is already self-documenting if written well, and all you need is a one line commit message to summarize that.

Doesn't the original conversation at least retain the context about why the change was made? A different LLM literally has no way to tell why you made this change besides guessing from the codebase and git history.

the mechanism is self documenting; context is not unless you pollute all your files with an ADR's worth of alternatives.

If a change makes sense, a different frontier model can usually figure out why it was made from the code alone. I take that as a signal that that the commit is good.

I believe they can do this due to having millions of public pull requests and github issues in their training data.

There's pretty much always several plausible reasons for why a change was made. What's interesting is knowing exactly which one, especially when it later turns out to be wrong!

What a change does is self documenting. Why it was made and why that specific approach was chosen could be due to many things, such as:

* business objectives,

* the result of experimentation,

* the result of an offline conversation

etc.

This cannot be captured in code alone. This is why we write commit messages.

The blog post is advocating against this.

If you ask an LLM to write a message that explains "the why", it'll make up a why.

So we can read it, and correct it in the occasional event that its wrong.

Reply on news.ycombinator.com