> These are not going to be problems tomorrow because the technology will shift. As it happened many time in the span of the last 2 years.
What technology shifts have happened for LLMs in the last 2 years?
What technology shifts have happened for LLMs in the last 2 years?
Now, I'm open to the idea that I am just using it wrong, but I have seen several reports around the web that the most that people got in tool calling accuracy is 80%, which is unusable for any production system, also for info retrieval I have seen it lose coherence the more data is available overall.
Is there a model that actually achieved 100% tool calling accuracy?
So far I built systems for that myself, surrounding the LLM, and only like this it worked well in production.
When we were at 4,000 and 16,000 context windows, a lot of effort was spent on nailing down text splitting, chunking, and reduction.
For all intents and purposes, the size of current context windows obviates all of that work.
What else changed?
- Multimodal LLMs - Text extraction from PDFs was a major issue for rag/document intelligence. A lot of time was wasted trying to figure out custom text extraction strategies for documents. Now, you can just feed the image of a PDF page into an LLM and get back a better transcription.
- Reduced emphasis on vector search. People have found that for most purposes, having an agent grep your documents is cheaper and better than using a more complex rag pipeline. Boris Cherny created a stir when he talked about claude code doing it that way[0]
Large context windows can make some problems easier or go away for sure. But you may still have the same issue of getting the right information to the model. If your data is much larger than e.g. 256k tokens you still need to filter it. Either way, it can still be beneficial (cost, performance, etc.) to filter out most of the irrelevant information.
>Reduced emphasis on vector search. People have found that for most purposes, having an agent grep your documents is cheaper and better than using a more complex rag pipeline
This has been obvious from the beginning for anyone familiar with information retrieval (R in RAG). It's very common that search queries are looking for exact matches, not just anything with similar meaning. Your linked example is code search. Exact matches/regex type of searches are generally what you are looking for there.
These last few years, I've noticed that the tone around AI on HN changes quite a bit by waking time zone.
EU waking hours have comments that seem disconnected from genAI. And, while the US hours show a lot of resistance, it's more fear than a feeling that the tools are worthless.
It's really puzzling to me. This is the first time I noticed such a disconnect in the community about what the reality of things are.
To answer your question personally, genAI has changed the way I code drastically about every 6 months in the last two years. The subtle capability differences change what sorts of problems I can offload. The tasks I can trust them with get larger and larger.
It started with better autocomplete, and now, well, agents are writing new features as I write this comment.
I wouldn't want to have to review the output of an agent going wild for an hour.
Now that's the idea anyway. Of course they all will lie to each other and there's hallucinations every step of the way. If you want to see a great example look at the documentation for the TEMU marketplace API. The whole API system, docs, examples etc appears to be vibe coded and lots of nonsensical formatting, methods that don't work and parameters in example that just say "test" or "parameters", but they are presented as working examples with actual response examples (like a normal API) but it largely appears to just be made up!
It’s the worst kind of disposable software.
Both sides have valid observations in their experiences and circumstances. And perhaps this is simply another engineering "it depends" phenomenon.
My skepticism and intuition that AI innovations are not exponential, but sigmoid are not because I don't understand what gradient-descent, transformers, RAG, CoT, or multi-head attention are. My statement of faith is: the ROI economics are going to catch up with the exuberance way before AGI/ASI is achieved; sure, you're getting improving agents for now, but that's not going to justify the 12- or 13-digit USD investments. The music will stop, and improvements slow to a drip
Edit: I think at it's root, the argument is between folk who think AI will follow the same curve as past technological trends, and those who believe "It's different this time".
I did neither of these two things... :) I personally could not care about
- (over)hype
- 12/13/14/15 ... digit USD investment
- exponential vs. sigmoid
There are basically two groups of industry folk:
1. those that see technology as absolutely transformational and are already doing amazeballs shit with it
2. those that argue how it is bad/not-exponential/ROI/...
If I was a professional (I am) I would do everything in my power to learn everything there is to learn (and then more) and join the Group #1. But it is easier to be in Group #2 as being in Group #1 requires time and effort and frustrations and throwing laptop out the window and ... :)
>> ...there are people that are professionals and are willing to put the time in to learn and then there’s vast majority of others who don’t...
tl;dr version: having negative view of the industry is decoupled from one's familiarity with, and usage of the tools, or the willingness to learn.
I hack for a living. I could hardly give two hoots about “false promises” or “hucksters” or some “impeding economic reckoning…” I made a general comment that a whole lot of people simple discount technology on technical grounds (favorite here on HN)…
I suppose this is the crux of our misunderstanding: I deeply care about the long-term health and future of the field that gave me a hobby that continues to scratch a mental itch with fractal complexity/details, a career, and more money than I ever imagined.
> or some “impeding economic reckoning…”
I'm not going to guess if you missed the last couple of economic downturns or rode them out, but an economic reckoning may directly impact your ability to hack for a living, that's the thing you prize.
I’d make the distinction between these systems and what they’re used for. The systems themselves are amazing. What people do with them is pretty mundane so far. Doing the same work somewhat faster is nice, and it’s amazing that computers can do it, but the result is just a little more of the same output.
Your whole point isn't supported by anything but ... a guess?
If given the chance to work with an AI who hallucinates sometimes or a human who makes logical leaps like this
I think I know what I'd pick.
Seriously, just what even? "I can imagine a scenario where AI was involved, therefore I will treat my imagination as evidence."
Remember, a logistic curve is an exponential (so, roughly, a process whose outputs feed its growth, the classic example being population growth, where more population makes more population) with a carrying capacity (the classic example is again population, where you need to eat to be able to reproduce).
Singularity 2026 is open and honest, wearing its heart on its sleeve. It's a much more respectable wrong position.
Just in my office, I have seen “small tools” like Charles Proxy almost entirely disappear. Everyone writes/shares their AI-generated solutions now rather than asking cyber to approve a 3rd party envfile values autoloader to be whitelisted across the entire organization.
I do lower level operating systems work. My bread and butter is bit-packing shenanigans, atomics, large-scale system performance, occasionally assembly language. It’s pretty bad at those things. It comes up with code that looks like what you’d expect, but doesn’t actually work.
It’s good for searching code big codebases. “I’m crashing over here because this pointer has the low bit set, what would do that?” It’s not consistent, but it’s easy to check what it finds and it saves time overall. It can be good for making tests, especially when given an example to work from. And it’s really good for helper scripts. But so far, production code is a no-go for me.
But all the times I tried using LLMs to help me coding, the best it performs is when I give it a sample code (more or less isolated) and ask it for a certain modification that I want.
More often than not, it does make seemingly random mistakes and I have to be looking at the details to see if there’s something I didn’t catch, so the smallest scope there better.
If I ask for something more complex or more broad, it’s almost certain it will make many things completely wrong.
At some point, it’s such a hard work to detail exactly what you want with all context that it’s better to just do it yourself, cause you’re writing a wall of text to have a one time thing.
But anyway, I guess I remain waiting. Waiting until FreeBSD catches up with Linux, because it should be easy, right? The code is there in the Linux kernel, just tell an agent to port it to FreeBSD.
I’m waiting for the explosion of open source software that aren’t bloated and that can run optimized, because I guess agents should be able to optimize code? I’m waiting for my operating system to get better over time instead of worse.
Instead I noticed the last move from WhatsApp was to kill the desktop app to keep a single web wrapper. I guess maintaining different codebases didn’t get cheaper with the rise of LLMs? Who knows. Now Windows releases updates that break localhost. Ever since the rise of LLMs I haven’t seen software release features any faster, or any Cambrian explosion of open source software copying old commercial leaders.
The usefulness of your comment, on the other hand, is beyond any discussion.
"Anyone who disagrees with me is dishonest" is some kindergarten level logic.
Ridiculous statement. Is Google also not good for humanity as a whole? Is Internet not good for humanity as a whole? Wikipedia?
It seems pretty clear to me that culture, politics and relationships are all objectively worse.
Even remote work, I am not completely sure I am happier than when I use to go to the office. I know I am certainly not walking as much as I did when I would go to the office.
Amazon is vastly more efficient than any kind of shopping in the pre-internet days but I can remember shopping being far more fun. Going to a store and finding an item I didn't know I wanted because I didn't know it existed. That experience doesn't exist for me any longer.
Information retrieval has been made vastly more efficient so I instead of spending huge amounts of time at the library, I get that all back in free time. What I would have spent my free time doing though before the internet has largely disappeared.
I think we want to take the internet for granted because the idea that the internet is a long term, giant mistake is unthinkable to the point of almost having a blasphemous quality.
Childhood? Wealth inequality?
It is hard to see how AI as an extension of the internet makes any of this better.
It's a defensible claim I think. Things that people want are not always good for humanity as a whole, therefore things can be useful and also not good for humanity as a whole.
Whether that's true or not, it is a different claim which doesn't fit the way I responded. It does fit the way Libidinalecon responded.
I don't think it's because the audience is different but because the moderators are asleep when Europeans are up. There are certain topics which don't really survive on the frontpage when moderators are active.
This would mean it is because the audience is different.
This fru-fru about how "we all play a part" is only serving to obscure the reality.
There's dang who I've seen edit headlines to match the site rules. Then there's the army of users upvoting and flagging stories, voting (up and down) and flagging comments. If you have some data to backup your sentiments, please do share it - we'd certainly like to evaluate it.
My email exchanges with Dang, as part of the moderation that happens around here, have all been positive
1. I've been moderated, got a slowdown timeout for a while
2. I've emailed about specific accounts, (some egregious stuff you've probably never seen)
3. Dang once emailed me to ask why I flagged a story that was near the top, but getting heavily flagged by many users. He sought understanding before making moderation choices
I will defend HN moderation people & policies 'til the cows come home. There is nothing close to what we have here on HN, which is largely about us being involved in the process and HN having a unique UX and size
Emphasis mine. The question is does the paid moderation team disappear unfavorable posts and comments, or are they merely downranked and marked dead (which can still be seen by turning on showdead in your profile).
When paul graham was more active and respected here, I spoke negatively about how revered he was. I was upvoted.
I also think VC-backed companies are not good for society. And have expressed as much. To positive response here.
We shouldn't shit on one of the few bastions of the internet we have left.
I regret my negativity around pg - he was right about a lot and seems to be a good guy.
The by far more common action is for the mods to restore a story which has been flagged to oblivion by a subset of the HN community, where it then lands on the front page because it already has sufficient pointage
What I'm pointing out is just that moderation isn't the same at different times of the day and that this sometimes can explain what content you see during EU and US waking hours. If you're active during EU daytime hours and US morning hours, you can see the pattern yourself. Tools like hnrankings [1] make it easy to watch how many top-10 stories fall off the front page at different times of day over a few days.
This is what you said. There has only been one until this year, so now we have two.
The moderation patterns you see are the community and certainly have significant time factors that play into that. The idea that someone is going into the system and making manual changes to remove content is the conspiracy theory
On the application layer, connecting with sandboxes/VM's is one of the biggest shifts. (Cloudfares codemode etc). Giving an llm a sandbox unlocks on the fly computation, calculations, RPA, anything really.
MCP's, or rather standardized function calling is another one.
Also, local llm's are becoming almost viable because of better and better distillation, relying on quick web search for facts etc.
Let's not kid ourselves as to what it actually is.
I think the next period of high and rapid growth will be in media (image, video, sound, 3D), not text.
It's much harder to adapt LLMs to solving business use cases with text. Each problem is niche, you have to custom tailor the solution, and the tooling is crude.
The media use cases, by contrast, are low hanging fruit and result in 10,000x speedups and cost reductions almost immediately. The models are pure magic.
I think more companies would be wise to ignore text for now and focus on visual domain problems.
Nano Banana has so much more utility than agents. And there are so many low hanging fruit ways to make lots of money.
Don't sleep on image and video. That's where the growth salient is.
I am so far removed from multimedia spaces that I truly can't imagine a universe where this could be true. Agents have done incredible things for me and Nano Banana has been a cool gimmick for making memes.
Anyone have a use case for media models that'll expand my mind here?
As someone in the film space, here's just one example: we are getting extremely close to being able to make films with only AI tools.
Nano Banana makes it easy to create character and location consistent shots that adhere to film language and the rules of storytelling. This still isn't "one shot", and considerable effort still needs to be put in by humans. Not unlike AI assistance in IDEs requiring a human engineer pilot.
We're entering the era of two person film studios. You'll undoubtedly start seeing AI short films next year. I had one art school professor tell me that film seems like it's turning into animation, and that "photorealism" is just style transfer or an aesthetic choice.
The film space is hardly the only space where these models have utility. There are so many domains. News, shopping, gaming, social media, phone and teleconference, music, game NPCs, GIS, design, marketing, sales, pitching, fashion, sports, all of entertainment, consumer, CAD, navigation, industrial design, even crazy stuff like VTubing, improv, and LARPing. So much of what we do as humans is non-text based. We haven't had effective automation for any of this until this point.
This is a huge percentage of the economy. This is actually the beating heart of it all.
The two are kind of orthogonal concepts.
AI still can’t reliably write text on background details. It can’t get shadows right. If you ask it to shoot things from a head on perspective, for example a bookshelf, it fails to keep proportions accurate enough. The bookshelf will not have parallel shelves. The books won’t have text. If in a library, the labels will not be in Dewey decimal order.
It still lacks a huge amount of understanding about how the world works necessary to make a film. It has its uses, but pretending like it can make a whole movie is laughable.
I'd say the image and video tools are much further along and much more useful than AI code gen (not to dunk on code autocomplete). They save so much time and are quite incredible at what they can do.
In terms of cinema tech, it took us arguably until the early 1940s to achieve "deep focus in artificial light". About 50 years!
The last couple of years of development in generative video looks, to me, like the tech is improving more quickly than the tech it is mimicking did. This seems unsurprising - one was definitely a hardware problem, and the other is most likely a mixture of hardware and software problems.
Your complaints (or analogous technical complaints) would have been acceptable issues - things one had to work around - for a good deal of cinema history.
We've already reached people complaining about "these book spines are illegible", which feels very close to "it's difficult to shoot in focus, indoors". Will that take four or five decades to achieve, based on the last 3 - 5 years of development?
The tech certainly isn't there yet, nor am I pretending like it is, and nor was the comment you replied to. To call it close is not laughable, though, in the historical context.
The much more interesting question is: At what point is there an audience for the output? That's the one that will actually matter - not whether it's possible to replicate Citizen Kane.