mfalcon

I'd like to get the reasoning trace by stage in order to verify the AI work. I'd like the AI to explain me why it thinks I'm lacking knowledge on some specific topic or sub-topic.

To finally trust the AI I'd like for it to find (if the content has already some human created exercises) or generate some question/exercise that confirms their diagnosis.

You can't simplify learning that way, learning is expanding your mental horizon, by learning something you grow and can take bigger things.

The LLMs are a way to get quick feedback and getting it promotes quicker learning (if the feedback is good). Learning is difficult, sometimes I get blocked on something and before llms that could prevent me from learning some material, nowadays I have another tool to help me unblock and learn faster.

I think that could be called a shortcut, not in learning per se, but in the process of getting to learn.

Do you trust everything you read or how do you verify that the LLM is not hallucinating and giving you erroneus content?

SQL seems to be a great fit because of how easy is to verify the output.

Are you building something in this space? Or are you giving the resources directly to a llm chat interface along with some prompt?

I think SWEs will still be needed but to do higher level work. The thing is, there will be less SWEs per company.

Obviously I dont know ehat will happen, but my guess is that there will be more small companies highly specialized on specific niches.

If I switch to business mode, I think this is the best time to start a company because you have more leverage in your hands and also because the job market is and is going to be disrupted, at least for a while.

Agree, and for the delivery-riders side, you and some vlose people can start making the deliveries if possible.

I don't know what requires more skills: creating Bitcoin or avoiding getting identified after all these years.

"Just when I thought I was out, they pull me back in"

You have to evaluate the llm responses. https://aunhumano.com/index.php/2025/09/03/on-evaluating-age...

In my opinion, conferences are mostly high noise low signal. My boss went to an "AI conference" last month and all the learnings he shared were things I was advocating already:

- use claude code

- evals are critical

- don't use AI everywhere

I think the conferences are more "work vacations" than work. Maybe they're useful if you think about them this way.

The truth is that no one really knows.

My approach is to "ride the wave", don't resist, just try to use it as an advantage and if not possible, adapt and look where I can keep adding value to the world.

Currently I'm trying to get my own company started because that way I can iterate the product faster using AI and if it goes well I can slowly migrate from coder to manager. I've come to terms that my days as a pure coder are coming to an end but I've been coding for at least 15 years.

Wake me up at Opus 12

Would be nice to use as an interface to interact with Claude Code.

Nice, I'll take a look. I was thinking about building a benchmark similar to the one you described, but first focusing on the negotiation between the store and the product suppliers.

Does your software also handle this type of task?

I was eagerly waiting for a chapter on semantic similarity as I was using Universal Sentence Encoder for paraphrase detection, then LLMs showed up before that chapter :).

I had been working on NLP, NLU mostly, some years before LLMs. I've tried the universal sentence encoder alongside many ML "techniques" in order to understand user intentions and extract entities from text.

The first time I tried chatgpt that was the thing that surprised me most, the way it understood my queries.

I think that the spotlight is on the "generative" side of this technology and we're not giving the query understanding the deserved credit. I'm also not sure we're fully taking advantage of this funcionality.

You can evaluate with your programming language of choice.

Good idea for a follow up post :)

Yes, and these problems are more present in the first iterations, when you are still trying to get a good enough agent behaviour.

I'm still thinking about good ways to mitigate this issue, will share.

Hey fellow hners, OP here. Been working on agents for a while so I started sharing some things.

The idea is to keep updating this post with a few more approaches I'd been using.

I know that the "understanding" is a stretch, but I refer to the Understanding of the NLU that wasn't really understanding either.

I think that the natural language understanding capability of current LLMs is undervalued.

To understand what the user meant before LLM's we had to train several NLP+ML models in order to get something going but in my experience we'll never get close to what LLM's do now.

I remember the first time I tried ChatGPT and I was surprised by how well it understood every input.

I guess it depends on how you use the LLMs. We implemented some workflows where the LLMs were used only for dialogue understanding, then the system response was generated by classic backend code.

That's the way I'd used it, I've built a document with all the requirements and then gave it to CC. But it was not a final document, I had to go back and make some changes after experimenting with the code CC built.

We're using gitlab issue tracker and it's not even close.

I know you tried to hack it the same way as I did.

I think that we will be reviewers too, we have to know if the AI generated artifact does what we want.

The AI doesn't suffer, doesn't struggle, doesn't love, doesn't question itself.

The AI is not like us. The AI only can produce something "nice" with no backstory.