Hacker News Reader

Comment by diggan | Hacker News Reader

> how to easily get regression tests from REPL driven development

Usually it's a matter of renaming "comment" to "deftest", and move some things around. Not sure how/why it would be different when using a LLM compared to doing it manually.

View on HN

There is often a misunderstanding that “REPL driven development” means typing things into the REPL prompt interactively and accumulating state that’s not easy to reproduce. Writing code down in comment blocks as you go is the solution to that, as you mentioned. With that technique I have no fear of restarting my REPL and getting back to where I was.

That's true. To be fair, if you come from a language like Ruby, JavaScript or Python, a REPL is basically a standalone experience, so I understand why people initially think it's like that, the name is more or less the same :) "Interactive Development" is maybe a better overall term that can lead to people thinking "Oh, what is that?" rather than "I've tried that already, what's the point?"

I like that. So you're saying that the switch between REPL and writing a test is much more seamless in a clojure system. That makes a lot of sense to me.

At a basic level any Lisp-like with an editor integrated REPL lets you put your cursor over the `greet` invocation below and evaluate it, and have the results shown in the manner of your choice (IDE window, inline temporary comment, etc).

This is all in the file as saved to your version control:

  (defn greet [name] 
    (-> (str "Hello, " name "!")
         println))

  (comment
    (greet "worldsayshi"))

It's 90% of the way to being a test already, just from the typical workflow.

But if you're maintaining a bunch of code blocks by comment toggling, why not just turn it into tests from the get go? I suppose there's a somewhat fine line between TDD and REPL given the right tooling for it.

Different purposes. I usually start out with comment blocks, for experimenting/prototyping, once happy, cement whatever assertions I did there, but inside deftests. Not everything is moved to a test though, some things are just "helpers" sort of, and those tend to remain inside comment blocks instead of being converted to deftest or similar.

I use the REPL as lightweight form of testing while I'm in the flow state, which is complementary to TDD. If there's too much code in the comment block, that's a good signal to move it into a formal test like you're saying, or at least move the code into a reusable function so the comment block can be shorter and focused on usage examples instead of implementation details.

Exactly. I start in the repl, but when it becomes a few expressions deep, I transfer it to the editor and turn it into a proper defun. Start 'sending' the defun to the repl and test it out in repl. Rince repeat.

> I start in the repl [...] I transfer it to the editor

That's exactly the kind of misunderstanding parent was talking about :)

Why start somewhere else than your editor, if it's already hooked up to the REPL? When I launch my REPL, I don't think it even accepts stdin, because there is no reason it has to.

There is no hard and fast rule (for me).

It is easy to start creating objects in the repl (say if you are testing an API) and work with those created objects in the repl, one step at a time. You are able to observe the behaviour of the objects every step of the way. This gives you a much better idea when you are writing the functions (usually copy-pasting from repl) in the editor.

I am sure it can be done from the editor itself. (say using the 'comment' block in clojure). It is just matter of preference.

> LLM compared to doing it manually.

When testing against an LLM you probably want to mock its responses for tests. Which is quite doable by copy-pasting its responses. I'm just curious what the "best" workflow is.

If I'd be using Clojure for this, it would look something like this:

1. Write a function that returns a LLM response

2. Play around with this function in a `comment` block, highlighting what I'm testing

3. If I find a response I'd like to "cement"/save and use for testing, Conjure (what I use to connect to the REPL) has a command that is basically "evaluate what's highlighted in the editor with whatever that returns when evaluated", so that basically copy-pastes the return value into the editor

4. Move it into a test somewhere

Not sure there is a better or faster workflow, from deciding "Hmm I want to test against this in the future" to "make test checks this" takes something like 2 minutes with a workflow like this.

Reply on news.ycombinator.com