Comment by simonw | Hacker News Reader

Comment by simonw | Hacker News Reader

Parent Full thread

Quoting my article:

> This is a useful starting point for a project like this—in figuring out the compile step the coding agent gets seeded with a little bit of relevant information about the project, and if it can compile that means it can later partially test the code it is writing while it works.

"Figure out how to build this" is a shortcut for getting a coding agent primed for future work. If you look at the transcript you can see what it did: https://gistpreview.github.io/?ddabbff092bdd658e06d8a2e8f142...

   Ran ls
    └ AGENTS.md
      CHANGELOG.md
        +13 lines
      scripts
      sdk
  
   Ran cat README.md
    └ <p align="center"><code>npm i -g @openai/codex</code><br />or <code>brew
      install --cask codex</code></p>
  
        +105 lines
  
      This repository is licensed under the [Apache-2.0 License](LICENSE).
  
   Ran rg -n "rust" -n docs/install.md
    └ 23:curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y
      25:rustup component add rustfmt
      26:rustup component add clippy
  
   Ran sed -n '1,200p' docs/install.md
    └ ## Install & build
  
        +36 lines
      cargo test
      ```
  
   Im gathering the build steps for the Rust tool, focusing on running cargo
    build inside the codex-rs directory. I'll check dependencies and confirm
    the instructions before actually building, aiming to provide a clear summary
    afterward.
  
   Ran cargo build
    └     Updating git repository `https://github.com/nornagon/crossterm`
          Updating git repository `https://github.com/nornagon/ratatui`
        +825 lines
         Compiling codex-cli v0.0.0 (/Users/simon/Dropbox/dev/codex/codex-rs/cli)
          Finished `dev` profile [unoptimized + debuginfo] target(s) in 1m 27s
  
   Ran ls target/debug | head
    └ apply_patch
      apply_patch.d
        +6 lines
      codex-execpolicy

That's a decent starting point on seeding the context with information that's relevant to making and then testing the modifications I'm about to ask for.

What useful context is in there? How to call “cargo build”? It already knows that.

The README files that it looked at, and the directory structure it explored.

It now knows what the project is, what dependencies it uses, how it's laid out and the set of binaries that it generates.

Even more importantly: it knows that the project can be built without errors. If it tries a build later and sees an error it will know that the error was caused by code it had modified.

You could just tell it to check out readme, but I suspect it would have checked it out anyway or figured out the type of project and how it is structured as a first step of any other command you give it as without it it is impossible to add or update the project.

The most important thing is to have it successfully build the software, to prove to both me and itself that a clean compile is possible before making any further changes.

Suggestion: make a “check.sh” script that builds everything, lints everything, and runs all (fast) tests. Add a directive in the agent system prompt to call it before & after doing anything. If it fails, it will investigate why.

In situations like this is better to ask the agent to write a short document about how to run the project. Then you read it and delete useless parts. Then you ask the agent to follow that document and improve it until the software builds. By the final step, you get a personalized README.md for your needs.

For a Rust developer, neglecting their ability to debug cargo build issues puts their career at risk. For someone like that, letting AI handle it would be a really shortsighted move.

But Simon isn’t a Rust developer - he’s a motivated individual with a side project. He can now speedrun the part he’s not interested in. That doesn’t affect anyone else’s decisions, you can still choose to learn the details. Ability to skip it if you wish, is a huge win for everyone.

>> He can now speedrun the part he’s not interested in

In this case its more like slowrunning. Building rust project is 1 command and chatgpt will tell you this command in 5 seconds.

Running an agent for that is 1000x more inefficient.

At this point its not optimizing or speeding things up but running agent for the sake of running agent.

The best thing about having an agent figure this out is you don't even need to be at your computer while it works. I was cooking dinner.

You’re not properly accounting for the risk of getting blocked on one of these 5 second tasks. Do an expected value calculation and things look very different.

Across a day of doing these little “run one command” tasks, even getting blocked by one could waste an hour. That makes the expected value calculation of each single task tilt much more in favor of a hands off approach.

Secondly, you’re not valuing the ability to take yourself out of the loop - especially when the task to be done by AI isn’t on the critical path, so it doesn’t matter if it takes 5 minutes or 5 milliseconds. Let AI run a few short commands while you go do something else that’ll definitely take longer than the difference - maybe a code review - and you’ve doubled your parallelism.

These examples are situational and up to the individual to choose how they operate, and they don’t affect you or your decisions.

> He can now speedrun the part he’s not interested in.

The reductio that people tend to be concerned about is, what if someone is not interested in any aspect of software development, and just wants to earn money by doing it? The belief is that the consequences then start becoming more problematic.

Those people are their own worst enemies.

Some people will always look for ways to "cheat". I don't want to hold back everyone else just because a few people will harm themselves by using this stuff as a replacement for learning and developing themselves.

Do you genuinely believe that this only applies to "a few people"?

This new post gets at the issue: https://news.ycombinator.com/item?id=45868271

I don't understand the argument that post is making.

I agree that people using LLMs in a lazy way that has negative consequences - like posting slop on social media - is bad.

What's not clearly to me is the scale of the problem. Is it 1/100 people who do this, or is it more like 1/4?

Just a few people behaving badly on social media can be viewed by thousands or even millions more.

Does that mean we should discard the entire technology, or should we focus on teaching people how to use it more positively, or should we regulate its use?

Reply on news.ycombinator.com