I am exploring ways to document the design for the agent to read and update. What makes it difficult is the lack of structure. Spec writing is not my core skill. Schemas and APIs are easier, there are declarative ways to document them. Runtime concepts and workflows have less structure and writing prose seems so unstructured for my taste. Formal languages are too rigid. But I could not find a better way.
I totally agree. I loved coding because of its closed feedback loop. Since last November, I also delegated it mostly to agents. Now I concentrate more on the design part, which is not the same. However, you move with the times and hope something else will become exciting. I do not know a more worthwhile and satisfying way than computing to spend my work hours.
Could you briefly describe your workflow for doing that or give a pointer to a blog you wrote/like that aligns with the process? Thanks in any case, happy designing ;-)
Please see Ben Evans’ podcast on a good take on this. Coding is just one of the task you do in your job, it is not the job or at least it probably is not. You do not get paid to code, you get paid to make a set of decisions that create value to the company. If this is automated then yes sadly your salary is not justified.
I wonder what will be the effect of this on open source software. I do not mean the technical aspects, easy of coding, documenting, explosion of PRs, .. Social aspects scare me a bit more. If everyone is building their own version, will people stick around to contribute to the same project for years. I guess the dedicated ones will do so maybe it is a good thing, a filtering out of the disinterested.
Debating how not to use AI will not get anyone anywhere since negative framing almost never works with humans (it also does not work with llms). Let’s concentrate on how to build closed loop systems that verify the llm output, how to manage context, and how to build failsafes around agentic systems and then and only then we might start to make progress.
Not all parts of the code is equal in this respect. Those parts pertaining to the user visible portion (API of a library, command args of a CLI, UI of a GUI/TUI app, endpoints in a web service, etc.) are closely related to the spec. The rest is more fluid as long as it does not change user visible behavior. The choices still affect maintenance and debugging costs, so there is some pressure to not YOLO these portions. I think the most difficult design decisions relate to how to separate the two and how to ensure a smooth evolution of both user facing and programmer facing design decisions.
What is different now is that maintainability and debugging design decisions were made w.r.t. human coders or teams in the past which is not necessarily the case anymore. Should we just specify the API and let agents figure the rest or do we still want to control the rest to ensure maintenance and security? A year ago I definitely thought so. Now it is more murky as the agents are faster browsers of codebases and can explore runtime effects faster than I can type and parse output. Strongest empirical observations depend on the runtime behavior so they have an edge there.
I am rereading the Asimov robot novels. A decrease in human to human interaction is a major side effect that he has foreseen. Decreasing interaction and collaboration are some of the core themes.
I think this question is one of the more concrete and practical ways to attack the problem of understanding transformers. Empirically the current architecture is the best to converge training by gradient descent dynamics. Potentially, a different form might be possible and even beneficial once the core learning task is completed. Also the requirements of iterated and continuous learning might lead to a completely different approach.
Thanks for posting a through and accurate summary of the historical picture. I think it is important to know the past trajectory to extrapolate to the future correctly.
For a bit more context: Before 2012 most approaches were based on hand crafted features + SVMs that achieved state of the art performance on academic competitions such as Pascal VOC and neural nets were not competitive on the surface. Around 2010 Fei Fei Li of Stanford University collected a comparatively large dataset and launched the ImageNet competition. AlexNet cut the error rate by half in 2012 leading to major labs to switch to deeper neural nets. The success seems to be a combination of large enough dataset + GPUs to make training time reasonable. The architecture is a scaled version of ConvNets of Yan Lecun tying to the bitter lesson that scaling is more important than complexity.
They say that they did test but the coverage was not enough to pick it up, at least for the prompt change:
“ After multiple weeks of internal testing and no regressions in the set of evaluations we ran, we felt confident about the change and shipped it alongside Opus 4.7 on April 16.
As part of this investigation, we ran more ablations (removing lines from the system prompt to understand the impact of each line) using a broader set of evaluations. One of these evaluations showed a 3% drop for both Opus 4.6 and 4.7. We immediately reverted the prompt as part of the April 20 release.”
Considering the number and scope of users they serve, I can sympathize with the difficulty. However, they should reimburse affected users at least partially instead of just announcing “our bad, sorry “. That would reduce the frustration.
I attended the related session at Next’26 yesterday. From my understanding it is a new backend and they will release the torch tpu source on github in one or two months. It will not support all ops initially but they are moving fast. Still for a while torchax is mature enough to run torch models on tpus by translating to jax.
My ancient boxed copy of Visual Basic for DOS 1.0 that supported mouse clicks on TUI buttons would have found your viewpoint quite offensive if it had any AI in it ;-) Oh boy, good old days.
Similar trend in open text-to-image models: Flux.1 was 12B but now we have 6B models with much better quality. Qwen Image goes from 20B to 7B while merging the edit line and improving quality. Now that the cost of spot H200s at 140GB came down to A100 levels, you can finally try larger scale finetuning/distillation/rl with these models. Very promising direction for open tools and models if the trend continues.
I guess, the sense of accomplishment is very person dependent. I enjoy programming a lot, but it is easy to find people who would challenge themselves to scale the said website to a million users/X view per day. I don't know the why, probably there is no fixed meaning to existence and nature likes diversity.
For me, the fun in programming also depends a lot on the task. Recently, I wanted to have Python configuration classes that can serialize to yaml, but I also wanted to automatically create an ArgumentParser that fills some of the fields. `hydra` from meta does that but I wanted something simpler. I asked an agent for a design but I did not like the convoluted parsing logic it created. I finally designed something by hand by abusing the metadata fields of the dataclass.field calls. It was deeply satisfying to get it to work the way I wanted.
But after that, do I really want to create every config class and fill every field by myself for the several scripts/classes that I planned to use? Once the initial template was there, I was happy to just guide the agent to fill in the boilerplate.
I agree that we should keep the fun in programming/art, but how we do that depends on the what, the who, and the when.
That is likely. Another factor that came into my mind is the gpu using less power due to simpler computations. You can store less data for grayscale, so you need to go over less pixel data to do effects etc. Whether accessibility controls achieve this or not would be implementation dependent I guess.
The bitter lesson here is that if you want to control a business you can not avoid or outsource marketing. It is a huge part of any trade and you have to bear the marketing cost.
I totally understand the desire to avoid it and concentrate on the craft and to create. I tried and failed at it numerous times. I decided that I will not start a business if I do not have any partners who understand and are willing to engage in sales and marketing.
If you dig ml/vision papers from old, you will see that formulation-wise they actually did, but they lacked the data, compute, and the mechanistic machinery provided by the transformer architecture. The wheels of progress are slow and requires many rotations to finally reach somewhere.
I think this is what is blunted by mass education and most textbooks. We need to discover it again if we want to enjoy our profession with all the signals flowing from social media about all the great things other people are achieving. Staying stupid and hungry really helps.
I think this is more about mechanistic understanding vs fundamental insight kind of situation. The linear algebra picture is currently very mechanistic since it only tells us what the computations are. There are research groups trying to go beyond that but the insight from these efforts are currently very limited.
However, the probabilistic view is very much clearer. You can have many explorable insights, both potentially true and false, by jıst understanding the loss functions, what the model is sampling from, what is the marginal or conditional distributions are and so on. Generative AI models are beautiful at that level. It is truly mind blowing that in 2025, we are able to sample from the megapixel image distributions conditioned on the NLP text prompts.
I have deep respect for cuda and Nvidia engineering. However, the arguments above seem to totally ignore Google Search indexing and query software stack. They are the king of distributed software and also hardware that scales. That is way TPUs are a thing now and they can compute with Nvidia where AMD failed. Distributed software is the bread and butter of Google with their multi-decade investment from day zero out of necessity. When you have to update an index of an evolving set of billions of documents daily and do that online while keeping subsecond query capability across the globe, that should teach you a few things about deep software stacks.
That is insightful. Courage to take risks means higher standard deviation in outcomes, more visible successes, but also more hard failures. Risk averse cultures have more stable outcomes, no big successes, but also less financially crippling failures. A personal or social safety net may or may not make you risk averse. Taking semi-calculated risks seems like a skill that needs to be learned for successful entrepreneurship.
The computations in transformers are actually generalized tensor tensor contractions implemented as matrix multiplications. Their efficient implementation in gpu hardware involves many algebraic gems and is a work of art. You can have a taste of the complexity involved in their design in this Youtube video: https://www.youtube.com/live/ufa4pmBOBT8
As a 15+ years emacs user the only item on my wishlist is client-server remote editing mode similar to that of vs code. Then I can go back to using emacs on cloud VMs. Does anyone know a solution to this that works as good as VS Code even when your latency is high? Hopefully, I will be pissed off with all the weird configuration flags of VS Code enough to write one myself ;-) To be fair its python integration is quite good at least for the usual stuff.
In some commercial contexts with the savings from that 20%, you can buy a lot freedom and then with the freedom you bought you can make more free things :)
Google Deepmind is the closest lab to that idea because Google is the only entity that is big enough to get close to the scale of AT&T. I was skeptical that the Deepmind and Google Brain merge would be successful but it seems to have worked surprisingly well. They are killing it with LLMs and image editing models. They are also backing the fastest growing cloud business in the world and collecting Nobel prizes along the way.
Sadly this naturally happens in any field that ends up expanding due to its success. Suddenly the number of new practitioners outnumbers the number of competent educators. I think it is a fundamental human resources problem with no easy fix. Maybe llms will help with this, but they seem to reinforce the convergence to the mean in many cases as those to be educated is not in a position to ask the deeper questions.
I totally agree with your assessment. I just wanted to highlight that “implementation agnostic” is both a blessing and a curse. You can always apply the principles of diataxis but it provides near zero guidance on how to actually build the documentation for a specific project. This does not reduce its conceptual value, but I wish there was another framework with a complementary practical value.