frankc

I see this complaint frequently about losing track of what the agents are doing, and I agree you do need to understand your system. But there seems to be this baked in assumption that if you lose track, you now need to manually wade through this massive mess to untangle it and maybe that is impossible. I don't agree.

If you don't understand the codebase, ask the agent to explain it to you. I'm not not kidding. Modern frontier models are fantastic as this - even more so than actually writing the code. It can tell you in words. It can generate architectural diagrams and sequence diagrams. It can write tests and scripts that prove it's assumptions. It can happily refactor so that the system design is aligned with your preferences.

Once you accept this, you can stop worrying so much about it and instead focusing on building the architectures and tools that lets the agents succeed better and faster - so called closed loops or agents prompting agents. Build systems that are more easily verifiable and deterministic so the agent can write very powerful property based tests. Focus more on what and why you are building, how to make sure all external properties are verifiable and leave the internals to the agents. The code is not really for us anymore.

It could easily be the other way around - religious addiction for people can't let go of the code.

I agree but I think its also more nuanced. I am also happy to use AI to operate at a higher level faster. I think that is somewhere in between. For instance, maybe I have some specific ideas and architecture I want to try for building a durable workflow engine. I dont just say 'make me a durable workout engine'. I'm very intentional about what it's doing at a system level but I am happy to cede the low level implementation details. If things work out, refactoring those details to my liking is also easy.

My main thought is would this allow me to speed up prompt process for large MoE models? That is the real bottleneck for m3ultra. The tokens per second is pretty good.

I think it's more granular than this, though. I also like to "make computer do thing" and have enjoyed using AI. But I also like building systems, optimizing systems. I find AI is a great partner in that. I can churn out prototypes more quickly, iterate on them more quickly etc. That also applies intra-system level. I might have a theory about how a different data structure or caching layer will affect application performance. It's now so much faster to test those kind of theories, and actually building good scaffolding around them to test them scientifically.

Yes, sometimes I can also ask AI to evaluate things at the system level and it often has surprisingly good insights, but that is usually a collaboration where our powers combined comes up with a better solution. I enjoy that process, too.

I do sympathize with the people "in mourning". I feel like this is really about how your identify is tied up in what you do. I have generally identified as a command line wizard. The xkcd of the guy flying in with "perl" very much speaks to me. But AI absolutely crushes at this. It's not that useful a skill anymore. Now I identify more as a local AI expert instead :D

I think we really need to have a serious think of what is "good quality" in the age of coding agents. A lot of the effort we put into maintaining quality has to do with maintainability, readability etc. But is it relevant if the code isn't for humans? What is good for a human is not what is good for an AI necessarily (not to say there is no overlap). I think there are clearly measurable things we can agree still apply around bugs, security etc, but I think there are also going to be some things we need to just let go of.

I recently built something in the same universe - using ffmpeg to receive streams from obs to capture audio and video - don't want to get into details beyond except to say it involved a fairly involved pipeline of ray actors and a significant admin interface with nicegui. I had no problem doing this with claude. You need to give it access to look up how do things, like context7. If you are doing something very specific, you need to have a session that does research to build a skill so it doesn't need to redo that research every time. And yes, you do need to tell it the architecture and be fairly detailed with something like how you want rbac.

Using these tools takes quite a bit of effort but even after doing all those steps to use the tool well, I still got this project done in a few days when it otherwise would have taken me 1-2 months and likely simply would never happened at all.

You need to be in plan mode. Not only can it not change code, its interaction with you is quite different. It will surface issues and ask you for choices.

One of the ways the chinese companies are keeping up is by training the models on the outputs of the American fronteir models. I'm not saying they don't innovate in other ways, but this is part of how they caught up quickly. However, it pretty much means they are always going to lag.

I think this is a pretty solid analogy but I look at the metaphor this way - people used to get strong naturally because they had to do physical labor. Because we invented things like the forklift we had to invent things like weightlifting to get strong instead. You can still get strong, you just need to be more deliberate about it. It doesn't mean shouldn't also use a forklift, which is its own distinct skill you also need to learn.

It's not a perfect analogy though because in this case it's more like automated driving - you should still learn to drive because the autodriver isn't perfect and you need to be ready to take the wheel, but that means deliberate, separate practice at learning to drive.

if you are doing equity statarb, its all low touch strategies, so yes the quants 'trade' in that they can write the strategies. The traders in that environment are more like support, usually.

I think this makes sense but I wonder if firecracker would work better than vagrant for this? I haven't used it before, though. I guess it might if you are trying to run gas town level orchestration.

Personally, I have been using beads for a few days on a couple of projects. I also like https://github.com/Dicklesworthstone/beads_viewer which is a nice tui for beads (with some additional workflow i haven't tried). I have found its been useful for longer, multi-session implementations. Its easier to get back into the work. I wouldn't go so far as to it couldn't do the work without it, but so far it seems smoother. These things are hard to measure. I think the it's really not that different than how an engineering team would use jira but more hierarchical, which helps preserve context, and with prebuilt instructions for how the agent should use it.

The skills that matter most to me are the ones I create myself (with the skill creator skill) that are very specific and proprietary. For instance, a skill on how to write a service in my back-testing framework.

I do also like to make skills on things that are more niche tools, like marimo (a very nice jupyter replacement). The model probably does known some stuff about it, but not enough, and the agent could find enough online or in context7, but it will waste a lot of time and context in figuring it out every time. So instead I will have a deep thinking agent do all that research up front and build a skill for it, and I might customize it to be more specific to my environment, but it's mostly the condensed research of the agent so that I don't need to redo that every time.

I think that is both pretty true but massively underrated in how much faster you can solve the problems you know how to solve. I do also help it finds helps me more quickly learn how to solve new problems, but I must still must learn how to solve these new problems I have it solve those new problems or things go off the rails.

I actually think the social media factor is the biggest reason...we can now compare ourselves to a much, much large circle which makes our relative standing seem much worse. I think relative standing affects our happiness much more than absolute. From a mate competition standpoint, that actually makes logical sense.

We are all going to need to have personal passwords/safe words we don't reveal to untrusted parties for authentication. Or maybe personal retinal scanners? I think personal auth might be an interesting startup to get ahead of this.

So far I am in the skeptic camp on this. I don't see it adding a lot of value to my current claude code workflow which already includes specialized agents and a custom mcp to search indexed mkdocs sites that effectively cover the kinds of things I would include in these skills file. Maybe it winds up being a simpler, more organized way to do some of this, but I am not particularly excited right now.

I also think "skills" is a bad name. I guess its a reference to the fact that it can run scripts you provide, but the announcement really seems to be more about the hierarchical docs. It's really more like a selective context loading system than a "skill".

The thing is that I don't use AI to replace things I can do deterministically with code. I use it to replace things I cannot do deterministically with code - often something I would have a person do. People are also fallible and can't be completely trusted to do the thing exactly right. I think it works very well for things that have a human in the loop, like codeing agents where someone needs to review changes. For instance, I put an agent in a tool for generating aws access policies from english descriptions or answering questions about current access (where they agent has access to tools to see current users, buckets policies etc). I don't trust the agent to do it exactly right so it just proposes the policies and I have to accept or modify them before they are applied, but its still better than writing them myself. And it's better than having a web interface do it because that is lacking context.

I think it's a good example of the kind of internal tools the article is talking about. I would not have spent the time to build this without claude making it much faster to build stand-alone projects and I would not have the agent to do the english -> policy output with LLMs.

Demand is a function of price. At any given price there is a quantity demanded. To know the price you also need to know the supply function. There is a demand for socks but if no one is will to supply socks for less a million dollars, the quantity demanded of socks at that price could be 0.

I also feel like what gets lost in this is not everything you are building is a bite size feature in large existing project. Sometimes you are adding an entire subsystem that is large to something relatively greenfield. if you broke that down into features, you will need 20 PRs and if you wait for review, or even don't wait but have to circle back to integrate lots of requested changes, what might be a couple of weeks of work turns into 2 to 3 months of work. That just does not work unless you are in a massive enterprise that is ok with moving like molasses. Do you wind up with something not as high quality? Probably. But that is just the trade-off with shipping faster.

I never tell anyone they have to use AI tools. You do you. In a few years we will see who is better off.

I don't know exactly where AI is going to go, but the fact that I keep seeing this one study of programmer productivity, with like 16 people with limited experience with cursor,uncritically quoted over and over assures me that the anti-AI fervor is at least as big of a bubble as the AI bubble itself.

Clarifying can help but ultimately it was trained on older versions. When you are working with a changing api, it's really important that the llm can see examples of the new api and new api docs. Adding context7 as a tool is hugely helpful here. Include in your rules or prompt to consult context7 for docs. https://github.com/upstash/context7

Somewhat, though CQRS might advocate for separate read and write models. It might be something like the writer publishes an event that is consumed by something that updates the read model and queries go directly to the read model. Whether or not that is overengineering depends on the problem and load.

I think that is a good question and we don't really know yet. I think we are going to have to overhaul a lot of how we educate people. Even if all AI progress stops today, there is still a massive shift in how many professions operate that is incoming.

I think it's doing the same for me but tbh, I am ok with that and not trying to fix it. I do not want to go back to the world before claude could knock out all of the tedious parts of programming.

Perhaps, but I don't think we should optimize for scenario of going back before these tools existed. Of course you need the medical equivalent of BCP, but it's understood that BCP doesn't imply you must maintain the same capacities, just that you can function until you get your systems back online.

To continue to torture analogies, and be borderline flippant, almost no one can work an abacus like old the masters. And I don't think it's worth worrying about. There is an opportunity cost in maintaining those abilities.

But it sounds like with AI doctors did better overall, or that is how I read the first couple of lines. If that is true, I don't really see a problem here. Compilers have eroded my ability to write assembly, that is true. If compilers went away, I would get back up to speed in a few weeks.

I just don't agree with this. I am generally telling the model how to do the work according to an architecture I specify using technology I understand. The hardest part for me in reviewing someone else's code is understanding their overall solution and how everything fits together as it's not likely to be exactly the way I would have structured the code or solved the problem. However, with an LLM it generally isn't since we have pre-agreed upon a solution path. If that is not what is happening than likely you are letting the model get too far ahead.

There are other times when I am building a stand-alone tool and am fine wiht whatever it wants to do because it's not something I plan to maintain and its functional correctness is self-evident. In that case I don't even review what it's doing unless it's stuck. This is more actual vibe code. This isn't something I would do for something I am integrating into a larger system but will for something like a cli tool that I use to enhance my workflow.