Comment by musebox35 | Hacker News Reader

Comment by musebox35 | Hacker News Reader

Parent Full thread

This is one of the dilemmas that I am trying to wrap my head around. I love optimizing software pipelines, which often boils down to figuring out the operational constraints that the compiler and the generic libraries can’t assume. Then I exploit these to squeeze out performance. But in a world I can start from scratch and code a domain specific solution from line zero in a matter of hours/days, I do not need general libraries as much as I used to. On one hand the code won’t be as well tested as a good general library. On the other hand, it also won’t have a plethora of library bugs that are there because the code is generic and opaque. One counter argument is that things are never static and you can’t have specific code for too long. A counter to that is that you can then change the code to be specific to the new reality at very low cost. This is the mental loop I ride constantly. Disclaimer: My circumstances are definitely not general, I am not writing code that is truly large scale.

Many open-source communities are wrestling with similar questions. In many ways the generic frameworks end up being reference implementations for the specific solutions to copy and hill-climb towards, and it's as you say unclear what the point is to maintain generic things beyond that.

What I feel is lacking with the solutions that have for example agent-generated and tuned GPU kernels is that the use-cases for them are unclear. If you are a researcher on second-order optimizers, you probably want to be able to handle variable input shapes to experiment, you also might want something readable to understand intermediate steps and perhaps build on that. If you are a neolab running massive training runs for 80% of your VC funding, you need to know that every line in your training code is bitwise identical to the theory/reference because a divergent run from some LLM-generated numerical bug will set you back or bankrupt you, so you can't just plop in a random kernel even if it promises good performance.

So who is the agentically-looped end result for? Except for Openai and Anthropic of course who sold the tools.

You already answered your own question. Testing and trust have always been the most important part.

This has nothing to do with LLMs. There have always been plenty of solutions that are more capable, but untrusted.

Your real question seems to be whether you can prioritize better. What are your project goals? If you have no say in or insight into those goals, you have an even bigger problem. What are you even working on?

For sure, you don't want to be maintaining your dependencies. LLMs make it trivial to rack up insane amounts of technical debt. Why is that appealing to anyone? How is it meaningfully different from the idiots wanting to fork everything on github and copypasta their way to startup success over a decade ago?

Reply on news.ycombinator.com