In my experience LLMs excel at SQL.
7,194 karma · joined August 17, 2016
[ my public key: https://keybase.io/efitz; my proof: https://keybase.io/efitz/sigs/hazfgppUsyRbFXxDYXd8v6fzMdb7IsC2BmEHVftCiBw ]
I am not sure token efficiency is an interesting problem in the long term, though.
And in the short term I wonder if prompts could be pre-compiled to “compressed tokens”; the idea would be to use a smaller number of tokens to represent a frequently needed concept; kind of like LZ compression. Or maybe token compression becomes a feature of future models optimized for specific tasks.
I was wondering last year if it would be worthwhile trying to create a language that was especially LLM-friendly, eg that embedded more context in the language structure. The idea is to make more of the program and the thinking behind it, explicit to the LLM but in a programming language style to eliminate the ambiguity of natural language (one could just use comments).
Then it occurred to me that with current LLM training methodology that there’s a chicken-and-egg problem; it doesn’t start to show rewards until there is a critical mass of good code in the language for LLMs to train on.
I think that you need one extra bit, that can contextually encode "rook has moved" or "en passant available".
Which ones?
btw young idealistic me voted for Perot in 1992
The product today doesn’t feel like the product I worked on; I feel no connection to it, and every time I think that there’s nothing they could do to make me dislike it any more, Microsoft has another “hey, hold my beer and watch this” moment.
The problem with the old Microsoft this that it was run by engineers who didn’t understand user experience. The problem now is that it’s run by MBAs who dont care about users, just licenses.
We used a streaming compression format that was originally designed for IBM tape drives.
It was fast as hell and worked really well, and was gentle on CPU and it was easy to control memory usage.
In the early 2000s on a modest 2-proc AMD64 machine we ran out of fast Ethernet way before we felt CPU pressure.
We got hit by the SOAP mafia during Longhorn; we couldn’t convince the web services to adopt it; instead they made us enshittify our “2 bytes length, 2 bytes msgtype, structs-on-the-wire” speed demon with their XML crap.
I don’t regularly use en-dashes, cause I don’t know how to make them.
Much better to start with an API. Then you can have the server abstract the store and the operations - use git or whatever - but you can change the store later without disrupting your clients.
I could write about this for hours, but instead I’ll just discuss two concepts that you need in modern logging: vertical correlation and horizontal correlation.
Within a system, requests tend to go “up” and “down” stacks of software. It is very useful in these scenarios to have “vertical correlation” fields shared between adjacent layers, so that activity in one layer can be unambiguously attributed to activity in the adjacent layers. But sharing such a correlation value requires passing the value between layers, which might be a breaking api change. Occasionally it’s possible to construct a correlation value at each adjacent layer by transforming existing parameters in exactly the same way on the calling side and called side.
Additionally, software on one system converses with software on other systems; in those cases you need to have pairwise correlation values between adjacent peer layers. Again, same limitations apply to carrying such a correlation value via the API or protocol.
Really foresighted devs can anticipate these requirements and generate unique transaction ids that can be shared between machines and up and down the stack.
A large part of the magic of art is the human choices that go into it.
As a matter of fact, I am (in the computer security vertical) - look for an announcement on Hacker News at the beginning of the year. I suspect that others are too, but there's always a discoverability problem for niche tools in verticals that one doesn't participate in, e.g. I know nothing about software for dentists but I know that at least one exists, and that there are probably a lot of dentists who use it but resent the fees, features or support, and there are probably some dentists who could manage an agentic coding project.
There have ALWAYS been niche OSS projects, and agentic coding will make them better and more prolific.
There are people like me who are passionate about a space and have the skills to manage an agentic coding project and the domain knowledge to design the software that they want, but not the skills and time necessary to have built the software in the absence of agentic AI. Last year I would never have started my 100k+ LoC project. This year I am proposing colleagues at two Fortune 50 companies to adopt it (at zero financial benefit to me). I am doing this from love of the problem space and desire to improve software security across the industry.
Many SaaS products I am interested in have very little “moat”. I am interested in them not because I can’t build them, but because my limited engineering time is better spent building business specific stuff.
Many products with product management teams spend a lot of their effort building functionality either to delight their highest paying customers, or features that are expected to be high-revenue.
I’m never going to be your highest paying customers, so I’m never going to get custom work from you (primarily orienting workflows to existing workflows inside your customers).
What everyone wants when they buys SaaS is to get value from it immediately without having to change our internal processes, broken as they are. But your model of feature prioritization is antithetical to this; you don’t want to build or support the 5-10 integration points I want; because that would allow me to build my own customizations without paying for your upsells.
You aren’t at immediate risk from agentic Ai from losing your big customers. But Agentic AI is enabling me and thousands of others to build hobby projects that deliver part of your core value but with limitless integration. I expect that you’ll see bleeding from the smallish customers way before you see hits from your whales.
However in a couple of years there will be OSS alternatives to what you do, and they will only become more appealing, rapidly.
As a side note it’s not just license pricing that will drive customers to agentically-coded solutions; it’s licensing terms. Nowadays whenever I evaluate SaaS or open source, if it’s not fully published on GitHub and Apache or MIT licensed, then I seriously consider just coding up an alternative - I’ve done this several times now. It’s never been easier.
https://efitz-thoughts.blogspot.com/2025/08/the-effect-of-ll...
I am not having some existential crisis, but if we get to a point where X% of humans cannot outperform “AI” on any task that humans deem “useful”, for some nontrivial value of X, then many assumptions that culture has inculcated into me about humanity are no longer valid.
What is the role of humans then?
Can it be said that humans “think” if they can’t think a thought that a non thinking AI cannot also think?
What would make me even more happy is if we linked our foreign policy, especially our trade and aid policies, to align with our Constitution.
Other governments can do what they want, but we should prefer to interact with governments that share our values, and we should not reward or prefer governments that don’t.
Many people today find LLMs useful, as evidenced by willingness to pay to use them. That alone IMO largely demolishes the “wasted” argument; markets will find the right price vs cost vs value trade-off when necessary.
The author presupposes that the only goal of all the LLM investment that has been done is to achieve AGI.
Certainly the big players are seeking AGI but they’re also businesses trying to make money, or at least not lose huge amounts of money. So we see a lot of market experiments trying to figure out where and how LLMs can provide value for paying customers.
They want you to pay them money and leave them alone. They literally don’t want to talk to you; the cost of the CS rep means that a call from you likely offsets most of all of what you pay them in a month.
The last thing they are going to do is hire more expensive people who can do more than just read a script “did you unplug it and plug it back in?”
Besides, they have very sensitive algorithms that don’t let things get bad enough that they risk losing customers. The algorithms understand their monopoly in your area (granted by the local government that you can vote out), and so “bad enough” is pretty bad compared to a place with real competition.
It sucks, but there it is.
Btw I had a similar situation with an ISP once; I literally sent them a traceroute showing a routing loop in their infra; they just don’t know how to deal with that kind of thing.
I don’t know why one would classify an accountant as a middleman.
I will say that other frontier models are starting to surprise me with their reasoning/understanding- I really have a hard time making (or believing) the argument that they are just predicting the next word.
I’ve been using Claude Code heavily since April; Sonnet 4.5 frequently surprises me.
Two days ago I told the AI to read all the documentation from my 5 projects related to a tool I’m building, and create a wiki, focused on audience and task.
I'm hand reviewing the 50 wiki pages it created, but overall it did a great job.
I got frustrated about one issue: I have a github issue to create a way to integrate with issue trackers (like Jira), but it's TODO, and the AI featured on the home page that we had issue tracker integration. It created a page for it and everything; I figured it was hallucinating.
I went to edit the page and replace it with placeholder text and was shocked that the LLM had (unprompted) figured out how to use existing features to integrate with issue trackers, and wrote sample code for GitHub, Jira and Slack (notifications). That truly surprised me.
On the other hand, it opens new worlds for farming…