AI founders will learn the bitter lesson
lukaspetersson.com
lukaspetersson.com
No matter how good the AI gets, it can't answer about what it doesn't know. It can't perform a process for which it doesn't know the steps or the rules.
No LLM is going to know enough about some new drug in a pharma's pipeline, for example, because it doesn't know about the internal resources spread across multiple systems in an enterprise. (And if you've ever done a systems integration in any sufficiently large enterprise, you know that this is a "people problem" and usually not a technical problem).
I think the startups that succeed will understand that it all comes down to classic ETL: identify the source data, understand how to navigate systems integration, pre-process and organize the knowledge, train or fine-tune a model or have the right retrieval model to provide the context.
There's fundamentally no other way. AI is not magic; it can't know about trial ID 1354.006 except for what it was trained on and what it can search for. Even coding assistants like Cursor are really solving a problem of ETL/context and will always be. The code generation is the smaller part; getting it right requires providing the appropriate context.
Seems to apply to AI as well.
Cursor can do at least the following "actions":
* code generation
* file creation / deletion
* run terminal commands
* answer questions about a code base
I totally agree with you on ETL (it's a huge part of our product https://www.definite.app/), but the actions an agent takes are just as tricky to get right.
Before I give Cursor, I often doubt it's going to be able to pull it off and I constantly impressed by how deep it can go to complete a complex task.
So Isn’t AI useless without the tools to manipulate?
--
[0] - https://aider.chat/
https://techcrunch.com/2024/12/19/in-just-4-months-ai-coding...
(The author used to post a lot of insightful comments here about LLMs and other generative models, too.)
Paul (Aider author) is a lot better at writing useful stuff than me though! (like the amazing blog posts)
Can you clarify what you mean? If "most people here haven't heard of it" then it's probably not a household name.
>>>SEARCH
}
>>>REPLACE
}, {'more': 'data'}
To add insult to injury, the IntelliJ plugin threw spurious errors. I ended up uninstalling it and marking my calendar to try again in 6 months.
Yet some people say Cursor is great. Is it something about my project? I can't imagine how it deals with a codebase that is many millions of tokens. Or is it something about me? I'm asking hard questions because I don't need to ask the easy ones.
What are people who think Cursor is great doing differently?
I don't think your poor experience is because of you, it's because of your codebase. Cursor works worse (in my experience) on larger codebases and seems particularly good at JS (e.g. React, node, etc.).
Cursor excels at things like small NextJS apps. It will easily work across multiple files and complete tasks that would take me ~30 minutes in 30 seconds.
Trying again in 6 months is a good move. As models get larger context windows and Cursor improves (e.g. better RAG) you should have a better experience.
I've been using it recently with @nanostores/react and @nanostores/router.
It constantly wants to use router methods from react-router and not nanostores so I am constantly correcting it.
This is despite using the rules for AI config (https://docs.cursor.com/context/rules-for-ai). It continually makes the same mistakes and requires the same correction likely because of the dominance of react-router in the model's training. That tells me that the prompt it's using isn't smart enough to know "use @nanostores/router because I didn't find react-router".
I think for Cursor to really nail it, the base prompt needs to have more context that it derived from the codebase. It should know that because I'm referencing @nanostores/router, to include an instruction to always use @nanostores/router.
It might help if you included API reference/docs for @nanostores/router in the context, but I'm not sure even that would fix it completely.
I've never thrown 1.5M LoC at it, but for smaller code bases (10k LoC) it is amazing at giving summaries and finding where in the code something is being done.
You can pretty easily disprove this theory by noting the number of extremely well-known, veteran software developers who claim that Cursor (or other kinds of LLM-based coding assistants) are working for them.
Very doubtful any one of them can be bought off, but certainly not all of them.
Anyone that's done any amount of systems integration in enterprises knows this.
"Let me talk to Lars; he should know because his team owns that system."
"We don't have any documentation on this, but Mette should know about it because she led the project."1. Agentic search can allow the model to identify what context is needed and retrieve the needed information (internally or externally through APIs or search)
2. I received an offer from OpenAI to give me free credits if I shared my API data with it, in other words, it is paying for industry specific data as they are probably fine tuning niche models.
There could be some exceptions to UI/UX going down specific verticals but eventually these fine tuning sector specific instances value will erode over time but this will likely occupy a niche since enterprise wants maximum configuration and more out of box solutions are oriented around SMEs.
And AI is too fundamental of a technology that a "loss leader biggest wallet wins" strategy, used by the likes of Uber, will work.
API access can be restricted. Big part of why Twitter got authwalled was so that AI models can't train from it. Stack overflow added a no AI models clause to their free data dump releases (supposed to be CC licensed), they want to be paid if you use their data for AI models.
1. Existing legacy players with massive data lock-ins like ERP providers and Google/Microsoft.
2. Massive consolidation within AI platforms rather than massive fragmentation if these legacy players do get disrupted or opportunities that do pop up.
In other words - the usual suspects will continue to win because they have the data and lock in. Any marginal value in having a specialized model, agent workflow, or special training data, ect. will not be significant enough to switch to a niche app.
It is indeed unfortunate and niches will definitely exist. What I am referring to is primarily in enterprise.
> Agentic search
The AI and LLM are just the the "bake" button. If you want anything good, you still have to prep and put good ingredients in.
Just like you can’t google for a movie if you don’t know the genre, any scenes, or any actors in it, and AI can’t build its own context if it didn’t have good enough context already.
IMO that’s the point most agent frameworks miss. Piling on more LLM calls doesn’t fix the fundamental limitations.
TL;DR an LLM can’t magically make good context for itself.
I think you’re spot on with your second point. The big differentiators for big AI models will be data that’s not easy to google for and/or proprietary data.
Lucky they got all their data before people started caring.
ChatGPT was able to answer "What was the video game with cards where you play against a bear guy, a magic guy and a set of robots?" (it's Inscryption). This is one area where LLMs work.
Let’s do another experiment. Do the same for the game I’m thinking of right now.
There were characters in it and one of them had a blue shirt, but that’s all I can remember.
That’s true. It’s why these things aren’t useless.
I’m saying that LLMs arent able to make context for themselves. It’s why these agentic startups are doomed to morph into a more sensible product like search, document QA, or automated browser testing.
If all that matters is what you can put into context, then AI really isn't a product in most cases. The people selling models are actually just selling compute, so that space will be owned by the big clouds. The people selling applications are actually just packaging data, so that space will be owned by the people who already have big data in their segment: the big players in each industry. All competitors at this point know how important data is, and they're not going to sell it to a startup when they could package it up themselves. And most companies will prefer to just use features provided by the B2B companies they already trust, not trust a brand new company with all the same data.
I fully expect that almost all of the AI wins will take the form of features embedded in existing products that already have the data (like GitHub with Copilot), not brand new startups who have to try to convince companies to give them all their data for the first time.
Startups can outcompete the Foundational Model companies by concentrating on creating a very domain specific model, and providing support and services that comes out of having expertise in that specific domain.
This is why OpenAI chose to co-invest in Cybersecurity startups with Menlo Ventures in 2022 instead of building their own dedicated cybersecurity vertical, because a partnership driven growth model nets the most profit with the least resources expended when trying to expand your TAM into a new and very competitive market like Cybersecurity.
This is the same reason why hyperscalers like Microsoft, Amazon, and Google themselves have ownership stakes in the foundational model companies like Anthropic, OpenAI, etc because at Hyperscalers size and revenue, Foundational Models are just a feature (an important feature, but a feature nontheless).
Foundational Models are a good first start, but are not 100% perfect in a number of fields and usecases. Ime, tooling built with these models are often used to cut down on headcount by 30-50% for the team using it to solve a specific problem. And this is why domain specific startups still thrive - sales, support, services, etc will still need to be tailored for buyers.
I don’t disagree with this from an economics perspective (it’s expensive running an FM to handle domain specific queries). But the most accurate domain knowledge always tends to involve internal data. And then it becomes the issue raised above: a people problem involving internal knowledge and data management.
Incumbent hyperscalers and vendors like MS, Amazon, etc (and even third party data managers like snowflake) tend to have more leverage when it becomes this type of data problem.
Well-put because the business is focused and to-the-point from the beginning.
For those applications where this gets you in the door to the domain, or gets you in sooner, this can be a competitive advantage. I think Lukas is pointing out the longer-term limitations of the approach though. I thought this would extend from 1980s electronics myself.
You could edit this however:
>Startups can [prosper] by concentrating on creating a very domain specific model, and providing support and services that comes out of having expertise in that specific domain.
And it may hold true anyway and you may have a lifetime of work ahead of you whether or not the more-generalized capabilities catch up or not. You don't always have to actually be competitive with capitalized corporations in the market if you are adding real value to begin with, and the sky can still be the limit.
>the most accurate domain knowledge always tends to involve internal data.
>Incumbent hyperscalers . . . tend to have more leverage when it becomes this type of data
That can help as a benchmark to gauge when a person or small team actually can occasionally outperform a billion-dollar corporation in some way or another.
I'm no Mr. Burns, but to this I have slowly said to myself "ex-cel-lent" similarly for decades.
It's good to watch AI approaches come and go and even better to be adaptable over time.
That said, the easiest proof-of-value for foundation models in security today is automating the SOC function by auto-generating playbooks, stitching context from various signal sources, and being able to auto-summary an attack.
This reduces the need for hiring a junior SOC Analyst, and is a workflow that has already been adopted (or is in the process of being adopted) by plenty of F500s.
At the end of the day, foundational models cannot reason yet, and that kind of capability is still far away.
This is not the same as having unique access to domain-specific data, which becomes more valuable as you run it through more powerful domain-agnostic models. It sounds like this latter point is the one you say has value for startups to tackle
Exactly!
Yes, fully agreed. Anything AI is discovering in your dataset could have been found by humans, and it could have been done by a more efficient program. But that would require humans to carefully study it and write the program. AI lets you skip the novel analysis of the data and writing custom programs by using a generalizable program that solves those steps for you by expending far more compute.
I see it as, AI could remove the most basic obstacle preventing us from applying compute to vast swathes of problems- and that’s the need to write a unique program for the problem at hand.
> AI will not play out the way the Web did (upstarts unseat giants)
I recently spoke to a doctor that wanted to do a startup one part of which is an AI agent that can provide consumers second opinions for medical questions. For this to be safe, it will require access to not only patient data, but possibly front line information from content origins like UpToDate because that content is a necessity to provide grounded answers for information that's not in the training set and not publicly available via search.
The obvious winner is UpToDate who owns that data and the pipeline for originating more content. If you want to build the best AI agent for medical analysis, you need to work with UpToDate.
> ...not brand new startups who have to try to convince companies to give them all their data for the first time.
Someone correct me if I'm wrong, but didn't smartphones go the "upstarts unseat giants" way? Apple wasn't a phone-maker, and became huge in the phone-market after their launch. Google also wasn't a phone-maker, yet took over the market slowly but surely with their Android purchase.
I barely see any Motorola, Blackberry, Nokia or Sony Ericsson phones anymore, yet those were the giants at one time. Now it's all iOS/Android, two "upstarts" initially.
They weren't upstarts, they were giants who moved into a new (but tightly related) space and pushed out other companies that were in spaces that at first seemed closely related but actually were more different than first appeared.
Android and iOS won because smartphones were actually mobile computers with a cellular chip, not phones with fancy software. Seen that way Apple was obviously not an upstart, they were a giant that grew even further.
Google is perhaps somewhat more surprising since they didn't do hardware at all before, but they did have Chrome, giving them a major in on the web platform side, and were also able to leverage their enormous search revenue. Neither resource is available to an upstart/startup.
(and the parent poster was competing with both :) )
* Google: $16.5B revenue, $4.2B net income [0]
* Apple: $24B revenue, $3.5B net income [1]
* Motorola: $36B, -$49M net income [2]
* BlackBerry (RIM): $2B, $382M net income [3]
* Nokia: €51B, €7.9B net income [4]
* Sony: $48B, $799M net income (converted from yen at 158 yen/$) [5]
These numbers are absolutely in the same ballpark as the supposedly-larger players which they beat out, and in terms of profit substantially higher than all but Nokia.
[0] https://abc.xyz/assets/investor/static/pdf/2007_google_annua...
[1] https://d18rn0p25nwr6d.cloudfront.net/CIK-0000320193/4913a18...
[2] https://www.motorolasolutions.com/content/dam/msi/docs/en-xw...
[3] https://www.annualreports.com/HostedData/AnnualReportArchive...
[4] https://www.nokia.com/system/files/files/request-nokia-in-20...
[5] https://www.sony.com/en/SonyInfo/IR/library/ar/SonyAR07-E.pd...
I think "upstarts" is being used uphthread to mean "startups" and "giants" is being used in a general, not market-specific, sense; that is, it isn't referring to entities that are mere new entrants in a particular market but still potentially quite large and established firms displacing incumbents in the particular market, but new, small-starting firms taking over a newly-opened market segment, beating out the large, established firms (from other markets) that are also trying to compete in it.
Except they won't package it themselves because they are inept and inert. They still won't sell it to startups though.
In the end, models are fundamentally a commodity. Data is all that matters, and in the not too distant future you won't gain anything at all by sending your data to OpenAI versus just using the tooling provided by your existing vendors.
i feel that the current artifact is already quite close to something that can operate in a competent manner if the downstream RL matches the task of interest well enough
RAG is just a single-purpose instance of the more general process of tool calling, so, that's not surprising.
This is exactly the motivation behind https://github.com/OpenAdaptAI/OpenAdapt: so that users can demonstrate their desktop workflows to AI models step by step (without worrying about their data being used by a corporation).
Providing all context will be little more than copying and pasting everything, or just letting the agent do its thing.
Super careful or complicated setups to filter and manage context probably won't be needed.
Is there a trick that bypasses this scaling constraint while strictly preserving the attention quality? I suspect that most such tricks lead to performance loss while deep in the context.
I suspect the sustainable value will be in providing context that isn't easily accessible as a copy and paste from your hard drive. Whatever that looks like.
Even if this is not solved, there is so much economic benefit, tens of TBs of VRAM will become feasible.
Yall really are just assuming this technology will stay still and not extrapolating from trends. A model that can get 25% on frontiermath is probably soon going to be able to navigate your company slack, that is not a more difficult problem than expert-level math proof development.
And that's why the teams that really want to unlock AI will understand that the core problem is really systems integration and ETL; the AI needs to be aware of the entire corpus of relevant information through some mechanism (tool use, search, RAG, graph RAG, etc.) and the startups that win are the ones that are going to do that well.
You can't solve this problem with more compute nor better models.
I've said it elsewhere in this discussion, but the LLM is just a magical oven that's still reliant on good ingredients being prepped and put into the oven before hitting the "bake" button if you want amazing dishes to pop out. If you just want Stouffer's Mac & Cheese, it's already good enough for that.
The process that feeds RAG is all about how you extract, transform, and load source data into the RAG database. Good RAG is the output of good ETL.
Is this another way of saying "content is king"?
We been using this pattern forever with traditional APIs but the huge hurdle is that the information in any system you integrate with is often both complex and messy. LLMs handle the hard work of handling ambiguity and variations.
HN, i find, also has a tendency to fall prey to the bitter lesson.
Startups should really try to get such a moat. Chapter 2 will cover this.
Context aside, you have the generation aspect of it, which can be very important (models trained to output good SQL, or good legal contracts, etc). You have the UI, which is possibly the most important element of a good AI product (think the difference between an IDE and Copilot - very very different UX/UI for the same underlying model).
Context is incredibly important, and I agree that people are downplaying some aspects of ETL here (though this isn't standard ETL in some cases). But it's not even close to being everything.
If you believe, as I do, that these things are a lot further off than some people assume, I think there's plenty of time to build a successful business solving domain-specific workflows in the meantime, and eventually adapting the product as more general technology becomes available.
Let's say 25 years ago you had the idea to build a product that can now be solved more generally with LLMs–let's say a really effective spam filter. Even knowing what you know now, would it have been right at the time to say, "Nah, don't build that business, it will eventually be solved with some new technology?"
The original "Bitter Lesson" article referenced in the OP is about developing new AI. In that domain, its point makes sense. But for the reasons you describe, it hardly applies at all to applications of AI. I suppose it might apply to some, but they're exceptions.
I think it will be less than 5 years.
You seem to be assuming that the rapid progress in AI will suddenly stop.
I think if you look at the history of compute, that is ridiculous. Making the models bigger or work more is making them smarter.
Even if there is no progress in scaling memristors or any exotic new paradigm, high speed memory organized to localize data in frequently used neural circuits and photonic interconnects surely have multiple orders of magnitude of scaling gains in the next several years.
I often suspect that the tech industry's perception of reality is skewed by Moore's Law. Moore's Law is, quibbles aside, basically real, and has of course had a dramatic impact on the tech industry. But there is a tendency to assume that that sort of scaling is _natural_, and the norm, and should just be expected in _everything_. And, er, that is not the case. Moore's Law is _weird_.
> I think if you look at the history of compute, that is ridiculous. Making the models bigger or work more is making them smarter.
It's better to talk about actual numbers to characterise progress and measure scaling:
" By scaling I usually mean the specific empirical curve from the 2020 OAI paper. To stay on this curve requires large increases in training data of equivalent quality to what was used to derive the scaling relationships. "[^2]
"I predicted last summer: 70% chance we fall off the LLM scaling curve because of data limits, in the next step beyond GPT4.
[…]
I would say the most plausible reason is because in order to get, say, another 10x in training data, people have started to resort either to synthetic data, so training data that's actually made up by models, or to lower quality data."[^0]
“There were extraordinary returns over the last three or four years as the Scaling Laws were getting going,” Dr. Hassabis said. “But we are no longer getting the same progress.”[^1]
---
[^0]: https://x.com/hsu_steve/status/1868027803868045529
Quantify this please? And make a firm prediction with approximate numbers/costs attached?
We will likely see between 3 and 10000 times improvement in efficiency or IQ or speed of LLM reasoning in the next 5 years.
Then do you actually know what you're talking about or are you handwaving? I'm not trying to be offensive but business plans can't be made based on a lack of predictions.
> We will likely see between 3 and 10000 times improvement in efficiency or IQ or speed of LLM reasoning in the next 5 years
That variance is too large to take you seriously, unfortunately. That's unfortunate because I was really hoping you had an actionable insight for this discussion. :(
If I, for instance, tell my wife I can improve our income by 3x or 1000x but I don't really know, there's no planning that can be done and I'll probably have to sleep on the couch until I figure out what the hell I'm doing.
They can. It's called "taking a risk". Which is what startups are about, right?
It's hard to give a specific prediction here (I'm leaning towards 10x-1000x in the next 5 years), but there's also no good reason to believe progress will stop, because a) there's many low and mid-hanging fruits to pick, as outlined by GP, and b) because it never did so far, so why would it stop now specifically?
Some things that are technologically possible are not economically viable. AI is a marvel but I'm not convinced it will actually plug into economic gains that justify the enormous investment in compute.
Spoken like a young man. I salute you. However, on your journey remember that risk of ruin is what you want to minimize relative to your estimated rewards. That is, not all risks can be afforded. I happen to have a limited budget, perhaps you don't and costs in terms of money and time don't matter for you.
Ruin can set you back years or decades or permanently and then you find yourself on a ycombinator thread hopelessly trying to find someone who can meaningfully quantity and forecast future medium term AI progress so that you can hire them to help your ongoing project. Alas all you get is the comments' section. :-)
> but there's also no good reason to believe progress will stop, because a) there's many low and mid-hanging fruits to pick, as outlined by GP, and b) because it never did so far, so why would it stop now specifically?
Specifically, due to lack of data. Please refer to the earlier comment[^0]: deep learning requires vast amounts of data. Current models have already been trained on the entire internet and corpus of published human knowledge. Models are now being trained on synthetic data and we're running out of that too. This data bottleneck has been widely reported and documented.
---
For most people, even a mere 3x in the next 5 years is huge, it's 25% per year growth.
3x in 5 years is a reasonable low-ball for hardware improvements alone. Caveat: top-end silicon is now being treated as a strategic asset, so there may be wars over it, driving up prices and/or limiting progress, even on the 5-year horizon.
I'm unclear why your metaphor would have you sleeping on the sofa: If tonight you produce a business idea for which you can be 2σ-confident that it will give you an income 5 years from now in the range [3…1000]x, you can likely get a loan for a substantially bigger house tomorrow than you were able to get yesterday; in the UK that's a change slightly larger than going from the median average full-time salary to the standard member of parliament salary.
(The reason behind this, observed lowering of compute costs, has been used even decades ago to delay investment in compute until the compute was cheaper).
The arguments I've seen elsewhere for order-of-10,000x* cost improvements (which is a proxy for efficiency and speed if not IQ) is based on various different observations cost reductions** since ChatGPT came out — personally, I doubt that the high end of that would come to pass, my guess is those all represent low-hanging fruit that can't be picked twice, but even then I would still expect there to be some opportunity for further gains.
* The original statement had one more digit in it than yours, but this doesn't make much difference to the argument either way
** e.g. https://www.wing.vc/content/plummeting-cost-ai-intelligence
And you seem to assume that it will just continue for 5 years. We've already seen the plateau start. OpenAI has tacitly acknowledged that they don't know how to make a next generation model, and have been working on stepwise iteration for almost 2 years now.
Why should we project the rapid growth of 2021–2023 5 years into the future? It seems far more reasonable to project the growth of 2023–2025, which has been fast but not earth-shattering, and then also factor in the second derivative we've seen in that time and assume that it will actually continue to slow from here.
I just looked on midjourney reddit to make sure I wasn't missing some new great model.
Instead what I notice is the small variations on the themes I have already seen a thousand times a year ago now. Midjourney is so limited in what it can actually produce.
I am really worried that all this is much closer to a parlor trick than AGI. "simple trick or demonstration that is used especially to entertain or amuse guests"
It all feels more and more like that to me than any kind of progress towards general intelligence.
Can you provide a source for this? I'm not super plugged into the space.
[0] https://www.wsj.com/tech/ai/openai-gpt5-orion-delays-639e769...
[1] https://techcrunch.com/2024/11/20/ai-scaling-laws-are-showin...
At one time I would have said you should be able to have an efficient office operation using regular typewriters, copiers, filing cabinets, fax machines, etc.
And then you get Office 97, zip through everything and never worry about office work again.
I was pretty extreme having a paperless office when my only product is paperwork, but I got there. And I started my office with typewriters, nice ones too.
Before long Google gets going. Wow. No-ads information superhighway, if this holds it can only get better. And that's without broadband.
But that's besides the point.
Now it might make sense for you to at least be able to run an efficient office on the equivalent of Office 97 to begin with. Then throw in the AI or let it take over and see what you get in terms of output, and in comparison. Microsoft is probably already doing this in an advanced way. I think a factor that can vary over orders of magnitude is how does the machine leverage the abilities and/or tasks of the nominal human "attendant"?
One type of situation would be where a less-capable AI could augment a defined worker more effectively than even a fully automated alternative utilizing 10x more capable AI. There's always some attendant somewhere so you don't get a zero in this equation no matter how close you come.
Could be financial effectiveness or something else, the dividing line could be a moving target for a while.
You could even go full paleo and train the AI on the typewriters and stuff just to see what happens ;)
But would you really be able to get the most out of it without the momentum of many decades of continuous improvement before capturing it at the peak of its abilities?
Mostly people either get bogged down into deep philosophical debates or simply start listing things that AI can and cannot do (and why they believe why that is the case). Some of those things are codified in benchmarks. And of course the list of stuff that AIs can't do is getting stuff removed from it on a regular basis at an accelerating rate. That acceleration is the problem. People don't deal well with adapting to exponentially changing trends.
At some arbitrary point when that list has a certain length, we may or may not have AGI. It really depends on your point of view. But of course, most people score poorly on the same benchmarks we use for testing AIs. There are some specific groups of things where they still do better. But also a lot of AI researchers working on those things.
I resemble that remark ;)
>that can now be solved more generally with LLMs
Nope, sorry, not yet.
>"Nah, don't build that business, it will eventually be solved with some new technology?"
Actually I did listen to people like that to an extent, and started my business with the express intent of continuing to develop new technologies which would be adjacent to AI when it matured. Just better than I could at my employer where it was already in progress. It took a couple years before I was financially stable enough to consider layering in a neural network, but that was 30 years ago now :\
Wasn't possible to benefit with Windows 95 type of hardware, oh well, didn't expect a miracle anyway.
Heck, it's now been a full 45 years since I first dabbled in a bit of the ML with more kilobytes of desktop memory than most people had ever seen. I figured all that memory should be used for something, like memorizing, why not? Seemed logical. Didn't take long to figure out how much megabytes would help, but they didn't exist yet. And it became apparent that you could only go so far without a specialized computer chip of some kind to replace or augment a microprocessor CPU. What kind, I really had no idea :)
I didn't say they resembled 25-year-old ideas that much anyway ;)
>We've had a lot of progress over the last 25 years; much of it in the last two.
I guess it's understandable this has been making my popcorn more enjoyable than ever ;)
Consider OpenAI's products as an example. GPT-3 (2020) was a massive step up in reasoning ability from GPT-2 (2019). GPT-3.5 (2022) was another massive step up. GPT-4 (2023) was a big step up, but not quite as big. GPT-4o (2024) was marginally better at reasoning, but mostly an improvement with respect to non-core functionality like images and audio. o1 (2024) is apparently somewhat better at reasoning at the cost of being much slower. But when I tried it on some puzzle-type problems I thought would be on the hard side for GPT-4o, it gave me (confidently) wrong answers every time. 'Orion' was supposed to be released as GPT-5, but was reportedly cancelled for not being good enough. o3 (2025?) did really well on one benchmark at the cost of $10k in compute, or even better at the cost of >$1m – not terribly impressive. We'll see how much better it is than o1 in practical scenarios.
To me that looks like progress is decelerating. Admittedly, OpenAI's releases have gotten more frequent and that has made the differences between each release seem less impressive. But things are decelerating even on a time basis. Where is GPT-5?
It isn't a specific model for any of those problems, but a "general" intelligence.
Of course, it's not perfect, and it's obviously not sentient or conscious, etc. - but maybe general intelligence doesn't require or imply that at all?
In other words, just AI, not AGI.
I'd like to believe this, but I'm not a mind reader and I feel like the last decade has eroded a lot of my trust in the ability of adults to know when they're wrong. I still have hope for children, at least.
If a general AI model is a "drop-in remote worker", then UX matters not at all, of course. I would interact with such a system in the same way I would one of my colleagues and I would also give a high level of trust to such a system.
If the system still requires human supervision or works to augment a human worker's work (rather than replace it), then a specific tailored user interface can be very valuable, even if the product is mostly just a wrapper of an off-the-shelf model.
After all, many SaaS products could be built on top of a general CRM or ERP, yet we often find a vertical-focused UX has a lot to offer. You can see this in the AI space with a product like Julius.
The article seems to assume that most of the value brought by AI startups right now is adding domain-specific reliability, but I think there's plenty of room to build great experiences atop general models that will bring enduring value.
If and when we reach AGI (the drop-in remote worker referenced in the article), then I personally don't see how the vast majorities of companies - software and others - are relevant at all. That just seems like a different discussion, not one of business strategy.
I suggest that before we satisfy _everyone_'s definition of AGI, more and more people may decide we are there as their own job is automated.
The UX at that point, maybe in 5 or 10 or X years, might be a 3d avatar that pops up in your room via mixed reality glasses, talks to you, and then just fires off instructions to a small army of agents on your behalf.
Nvidia actually demoed something a little bit like that a few days ago. Except it lives on your computer screen and probably can't manage a lot of complex tasks on it's own. Yet.
Or maybe at some point it doesn't need sub agents and can just accomplish all of the tasks on its own. Based on the bitter lesson, specialized agents are probably going to have a limited lifetime as well.
But I think it's worth having the AGI discussion as part of this because it will be incremental.
Personally, I feel we must be pretty close to AGI because Claude can do a lot of my programming for me. I still have to make important suggestions, and routinely for obvious things, but it is much better at me at filling in all the details and has much broader knowledge.
And the models do keep getting more robust, so I seriously doubt that humans will be better programmers overall for much longer.
I suspect that we will still be filling out forms, because that’s a better UI for a routine business transaction. It’s easier to know what the bank needs from you if it’s laid out explicitly, and you can also review the information you gave them to make sure it’s correct.
AI could still be helpful for finding the right forms, auto-filling some fields, answering any questions you might have, and checking for common errors, but that’s only a mild improvement from what a good website already does.
And yes, it’s also helpful for the programmers writing the forms. But the bank still needs people to make sure that any new forms implement their consumer interactions correctly, that the AI assist has the right information to answer any questions, and that it’s all legal.
At the end of the day, someone has to be checking the work. This is true of humans and of any potential AI agent, and the UX of that is a big deal. I can get on a call and talk through the code another engineer on my team wrote and make sure I understand it and that it's doing the right thing before we accept it. I'm sure at some point I could do that with an LLM, but the worry is that the LLM has no innate loyalty or sense of its own accuracy or honesty.
I can mostly trust that my human coworker isn't bullshitting me and any mistakes are honest mistakes that we'll learn from together for the future. That we're both in the same boat where if we write or approve malicious or flagrantly defective code, our job is on the line. An AI agent that's written bad or vulnerable code won't know it, will completely seriously assert that it did exactly what it was told, doesn't care if it gets fired, and may say completely untrue things in an attempt to justify itself.
Any AI "remote worker" is a totally different trust and interaction model. There's no real way to treat it like you would another human engineer because it has, essentially, no incentive structure at all. It doesn't care if the code works. It doesn't care if the team meets its goals. It doesn't care if I get fired. I'm not working with a peer, I'm working with an industrial machine that maybe makes my job easier.
The article sits on an assumption that if we just wait long enough, the unreliability of deep learning approaches to AI will just fade away and we'll have a full-on "drop-in remote worker". Is that a sound assumption?
Then, LLMs entered the stage. Suddenly, we became able to both derive vastly better output from the data we got, and also offer our users easier ways to describe what they were looking for, find good keywords automatically, and actually deliver helpful results!
This was only possible because AI augments our product well and really provides a benefit in that niche, something that would just not have been possible otherwise. If you plan on founding a company around AI, the best advice I can give you is to choose a problem that similarly benefits from AI, but does exist without it.
how did the LLM help with that challenge?
The best strategy from your experience, is to jump on a problem as soon there is opportunity to solve it and generate lots of business value within the next 6 months. The trick is finding that subproblem that is worth a lot right now and could not be resolved 6 months ago. A couple of AI-sales startups "succeeded" quite well doing that (e.g. 11x), now they are in a good position to build from there (whether they will succeed in building a unicorn, that's another question, it just looks like they are in a good position now).
> 70% of Y Combinator’s Winter 2024 batch are AI startups. This is compared to -57% of YC Summer 2023 companies and ~32% from the Winter batch one year ago (YC W23).
The thinking is, the models will get better which will improve our product, but in reality, like the article states, the generalized models get better so your value add diminished as there's no need to fine tune.
On the other hand the crypto fund made a killing off of "me too" block chain technology before it got hammered again. So who knows about 2-5 year term but 10 year almost certainly won't have these billion dollar companies that are wrappers around LLMs
https://x.com/natashamalpani/status/1772609994610835505?mx=2
LLMs are a platform.
Bill Gates definition of a platform was “A platform is when the economic value of everybody that uses it exceeds the value of the company that creates it.”
Startups that use cloud servers still write the software that goes on those servers, which is 90% of the value creation.
Almost all of them are using LLMs along with “tools” and RAG.
If you rely on an OpenAI LLM for your business, they can basically do whatever they want to you. Oh, prices went up 10x? What are you gonna do, train your own AI?
And if you think it’s hard to move to another LLM you haven’t done a major implementation using an LLM and used LangChain (I have). It abstracts a lot of the work and people can choose which LLM they want to use.
You don’t train your LLM. You use your LLM along with RAG.
https://simonwillison.net/2024/Dec/31/llms-in-2024/#llm-pric...
My second controversial point regarding AI research and startups: doing research sucks. It's risky business. You are not guaranteed success. If you make it, your competitors will be hot on your tail and you will have to keep improving all the time. I personally would rather leave the model building to someone else and focus more on building products with the available models. There are exceptions like finetuning for your specific product or training bespoke models for very specific tasks at hand.
I'll add even further. The transformers and etc that we are using today are not good either.
That's evidenced by the enormous amount of memory they need to do any task. We have just taken the one approach that was working a bit better for sensorial tasks and pattern matching, and went all in, adding hardware after hardware so we could brute-force some cognitive tasks out of it.
If we do the same to other ML architectures, I don't think they would stay much behind. And maybe some would get even better results.
I think it is true in an AI research context, but an unstated assumption is that you have complete data, E2E training, and the particular evaluated solution is not real-world unbounded.
It assumes infinite data, and it assumes the ability to falsify the resulting model output. Most valuable, 'real world' applications of AI when trying to implement in practice have an issue with one or both of those. So in other words: where a fully unsupervised AI pathway is viable due to the structure of the problem, absolutely.
I'm not convinced in the universality of this. Doesn't mean the core point of this essay on the futility of startups basing their business around one of the off the shelf LLMs isn't valid - I think for many they risk being generalized away.
I don't see how the bitter lesson could not be true for the current crop of LLM's. They seem to have memorised just about everything mankind has written down, and squished it into something of the order of 1TB. You can't do that without a lot of memory to recognise the common patterns and eliminate them. The underlying mechanism is nothing like the zlib's deflate but when it comes to memory you have to throw at it they are the same in this respect. The bigger the compression window the better deflate does. When you are trying to recognise all the pattens in everything humans have written down to a deep level (such as discovering the mathematical theorems are generally applicable), the memory window and/or compute you have to use must be correspondingly huge.
That was also true to a lesser extent when Deep Mind taught an AI to play pong in 2013. They had 1M of pixels arriving 24 times a second, and it had to learn to pick out balls, bats and balls in that sea of data. It's clearly going to require a lot of memory and compute to do that. Those resources simply weren't available on a researchers budget much before 2013.
Since 2013, we've asked our AI's to ingest larger and larger datasets using the much same techniques used in 2013 (but known long before that) and been enchanted with the results. The "bitter lesson" predicts you need correspondingly more compute and memory to compress those datasets. Is it really a lesson, or engineering rule of thumb that only became apparent when we had enough compute to do anything useful with AI?
I'm not sure this rule of thumb has much applicability outside of this "lets compress enormous amounts of data, looking for deep structure" realm. That's because if we look at neural networks in animals, most are quite small. A mosquito manages to find us for protein, find the right plant sap for food, find a mate, find water with enough algae for it's eggs, using data from vision, temperature sensors, and smell, and uses that to activate wings, legs and god knows what else. It does all that with 100,000 neurons. That's not what a naive reading of "the bitter lesson" tells you it should take.
Granted it may take an AI of enormous proportions to discover how to do it with 100,000 neurons. Nature did it by iteratively generating trillions upon trillions of these 100,000 neurons networks over millennia, and used a genetic algorithm to select the best at each step. If we have to do it that way it will be a very bitter lesson. The 10 fold increases in compute every few years that made us aware of the bitter lesson is ending. If the prediction of the bitter lesson is that we have rely on it continuing to build our mosquito emulation, then it's predicting it will take us centuries to build all the sorts of robots we need to do all the jobs we have.
But that's looking unlikely. We have an example. On one hand we have Tesla FSD, using throwing more and more resources an conventional AI training in the way the bitter lesson says you must do in order to progress. On the other we have Waymo using a more traditional approach. It's pretty clear which approach is failing and the other is working - and it's not going the way the bitter lesson says it should.
As I understand the article, it is going the way the bitter lesson predicts it would - the initial "more traditional" approach generates almost-workable solutions in the near term while the "bitter lesson" approach is unreliable in the near term.
Unless you think that FSD is already in the "far" term (i.e. already at the endgame), this is exactly what the article predicts happens in the near term.
For the slightly more complex stuff, sure at some point some general AI will probably be able to do it. But with two big caveats, the first being: when? and the second being: for how much?.
In theory every deep and wide enough neural network should be able to be trained to do object detection in images, yet no one is doing that. Technologies specifically designed to process images, like CNN's, reign supreme. Likewise for architectures of LLM's.
At some point your specialization might become obsolete, but that point might be a decade or more from now. Until then, specializations will have large economic and performance advantages making the advancements in AI today available to the industry of tomorrow.
I think it's the role of the VC to determine not if there's an AI breakthrough behind a startups technology, but if there's a market disruption and if that market disruption can be leveraged to establish a dominant company. Similar to how Google leveraged a small and easily replicable algorithmic advantage into becoming one of the most valuable companies on earth.
I haven't found it to work particularly well for some more domain-specific things I tried, but it was surprisingly good for an LLM.
The word "eventually" is doing a lot of work here. Yes, it's true in the abstract, but over what time horizon? We have to build products to solve today's problems with today's technology, not wait for the generalized model that can do everything but may be decades away.
I think with a lot of AI folk in San Francisco, this is a tacit assumption when having these sorts of conversations.
Let's imagine for a moment that this is even achieved. Then, there is still complex engineering required in the world: to maintain and continually improve the AI engines and their interfaces. Unless you want to say that, past some point, the AI will be self-improving without any human input whatsoever. Unless the AI can read our minds, I'm not sure it can continue to serve human interests without human input.
But, never mind, we will never get there. At this very moment, tech is capable of so much more, but most sites I visit have bad UI, are bloated downloading and executing massive amounts of JS, riddled with annoying ads that serve no real useful purpose to society, and riddled with bugs. Even as an engineer, I really struggle to find any good no-code tools to create anything truly sophisticated without digging into hard-core code. Heck, they are now talking about adding more HTTP methods to HTML forms.
Single app/feature startups will lose (true long before AI). A few will grow large enough to entrench distribution and offer a suite of services, creating defensibility against competitors
The distributors (eg. a SaaS startup that rapidly landed/expanded) will continue to find bleeding edge ways to offer a 6-12mo advantage against foundation models and incumbents
GitLab is a great example of this model. The equivalent bitter lesson of the web is that every cutting edge proprietary technology will eventually be offered free open source. However, there is a commercial advantage to purchasing the bleeding edge features with a strong SLA and customer service
The mistake is to think technology is a business. Business has always been about business. Good technology reduces the cost of sale (CAC) and cost of goods sold (COGS) to create a 85-90% margin. Good technology does not create a moat
Resilient businesses do not rely on singular technology advantages. They invest heavily in long term R&D to stay ahead of EACH wave. Resting on one's laurels after catching a single wave, or sitting out of the competition because there will be bigger waves later, are both surefire ways to lose the competition
So, what we're doing on the whole seems to be a lot of coding and decoding, hoping that the data used in training can be adequately mapped to the problem domain realities. That would mean that the model you end up with is somehow a valid representation of some form of knowledge about the problem domain. Trouble is, more text won't yield higher and higher resolution of some representation of the problem domain. After some point, you start to introduce noise.
But humans DO need ~16 years of growth and development "to learn how to drive competently" and then will also know how to ride a bycycle, mow grass, build shelves, cook pizza, use a smart phone, ...! There's a lesson in that somewhere ....
The amount of knowledge/behavior that is innate - encoded in our DNA - is pretty minimal, consisting of things like opposite sex sexual attraction, fear of heights, fear of snakes, disgust at rotting smell, etc - basic survival stuff.
Its also not given or constant, modern humans are super young in the evolutionary scale and very different even if very similar from other animals. Funny enough we might have cracked the difference (language processing) but are still very far in the rest for AGI.
I agree with that. I.e., for
"drive competently."
the way humans learn it is more general, i.e., also brings the list I gave with bycycles, pizza, grass mowing, etc.
Also
"drive competently."
the way humans do it requires a lot of judgment, maturity which is not very well defined!
Guys, I don't understand how to make progress on AI. E.g., just asked X > Grok "similitude", and it gave a good answer, maybe like a long dictionary answer. But it looked like Grok had some some impressive plagiarism -- I can't tell the difference.
> He points out that throughout AI’s history, researchers have repeatedly tried to improve systems by building in human domain knowledge. The “bitter” part of the title comes from what happens next: systems that simply use more computing power end up outperforming these carefully crafted solutions. We’ve seen this pattern in speech recognition, computer chess, and computer vision. If Sutton wrote his essay today, he’d likely add generative AI to that list. And he warns us: this pattern isn’t finished playing out.
According to Chomsky (recalled in relatively recent years for him) this is why he didn’t want to work on the AI/linguistics intersection when someone asked him in the mid 50’s. Because he thought that successful approaches would just use machine learning and have nothing to do with linguistics (that he cares about).
Building a quality product (AI or otherwise), involves design and data at all levels: UX, on-boarding, marketing, etc. The companies learning important verticals and getting in the door with customers will have a pretty huge advantage as models get better. Both in terms of install base, and knowing what customers need. Really great products don’t simply do what a customer asks, but are built by taking to a ton of customers over and over, and solving their problems better than any one of them can articulate.
It’s true we will need less and less custom software for problems. But it isn’t realistic to say the software wrapper effort is going to zero when models improve.
Plus: a lot of software effort is needed for getting the data AI needs. This is going to be a huge area - think Google maps with satellites, camera cars, network effect products (ratings), data collection (maps, traffic), etc.
But excel math is easy to outperform, ChatGPT etc wont be.
It can be simultaneously true that a tech is doomed to obsolescence from over specialization, and that it does incredibly useful things in the mean time.
Will someone truly discover a way to start from fundamentals and do better not just crunch zillions of petabytes faster and faster.
Or is that completely wrong? Happy to stand corrected
It sure seems like we have a lot of inherent knowledge, but by the book we're little more than the product of an instruction set that is our DNA and there's no explanation that includes inherited knowledge in that DNA.
I think the harder thing we need to acknowledge is that we understand much less than we think we know. Concepts like inherent, or collective, knowledge would all roll down hill from that.
But at some point I hope there's breakthrough from any entirely different paradigm.
We have ideas about how it works, sure. We know a bit about how the basics of the brain works and we know some correlations of what areas of the brain electrically light up in various conditions, but that's about the extent of it.
Its hard to really create an artificial version of something without pretty well understanding the real thing. In the meantime, brute forcing it is probably the best (and most common) approach.
Another big push back is copyrighted content. Without proper revenue model how to pay for that?
That will also restrict what can be "learned". Already there's lawsuit, allegations of using pirated books etc
Copyright issues here feel very similar to claims against Microsoft in the 80s and 90s.
OpenAI defines it based on the economic value of output relative to humans. Historically it had a much less financially arrived definition and general expectation.
I agree this approach is only viable one but I do hope the other way is also being tried. Who knows there may be a breakthrough. Like they try to create life from basic chemicals present in nature.
Regarding brute force, of course not. We are 2 years into chatgpt people are still thinking its just ngram statistical models smh. Dont listen to bad youtubers.... Not even ppl like 3b1b, sabine, thor, or primeagen. Why, when you can take it from so many ppl actually working in ai instead.
Anyway, yes chatgpt is huge. But if you use ngram statistical models like its 1980, you'd need the whole universe as a server and it still wont be as good. Big!=infinitely big. Its not 'brute force'.
Maybe its not what you meant. But i heard this a lot. Anyway, sorry for venting!
comming back to research one of the frontier model's deepseek was able to come close to SOTA with a relatively small budget because of one of their mixture of experts approach.
From that point of view, AI is more of the same. I've done a few things with the OpenAI apis. Easy work. There's not much to it. Scarily simple actually. With the right tools and frameworks we are talking a few lines of code mostly. The rest is just the usual window dressing you need to turn that into an app or service. And LLMs can generate a lot of that these days.
The worry for VC funded companies in this space is that a lot of stuff is becoming a commodity. For example, the llama and phi models are pretty decent. And you can run them yourself. Claude and OpenAI are a bit better and larger so you can't run them yourself. But increasingly those cheap models that you can run yourself are actually good enough for a lot of things. Model quality is a really hard to defend moat long term. Mostly the advantage is temporary. And most use cases don't actually need a best in class LLM.
So, I'm not a believer in the classic winner takes all approach here where one company turns into this trillion dollar behemoth and the rest of the industry pays the tax to that one company in perpetuity. I don't see that happening. The reality already is that the richest company in this space is selling hardware, not models. Nvidia has a nice (temporary) moat. The point of selling hardware is that you want many customers. Not just a few. And training requires more hardware than inference. So, Nvidia is rich because there are a lot of companies busy training models.
I agree with this sentiment. There are a lot of frontier model players that are very competent (OpenAI, Anthropic, Google, Amazon, DeepSeek, xAI) and I'm sure more will come onboard as we find ways to make models smaller and smaller.
The mental framework I try to use is that AI is this weird technology that is an enabler of a lot of downstream technology, with the best economic analogy being electricity. It'll change our society in very radical ways, but it's unclear who's going to make money off of it. In the electricity era Westinghouse and GE emerged as the behemoths because of their ability to manufacture massive turbines (which are the equivalent of today's NVIDIA and perhaps Google).
And even gpt-4o needs constant attention to ensure it doesn’t analyze input in a haphazard way.
I made a simple AI app for myself where I ran a bunch of recipes I’ve saved over the years through batch analysis. It tagged recipes with “kosher salt” as kosher lol. I had to baby sit the prompt for this simple pet project analysis for like 4 hours until I felt confident enough. And even then I was at maybe 60%.
Now imagine for a business application. This kind of incorrect analysis would be detrimental.
In specialized domains you can’t just rely on OpenAI finding all the training data required to render your experts obsolete.
If you can build a data flywheel you can fine-tune models and build that lead into a moat; whoever has the best training set will have the best product. In the short term you might start fine-tuning on OpenAI but once you’ve established your dataset you can potentially gain independence by moving onto OSS models.
If you are building a software assistant, sure, this is clearly something that OpenAI will get better at very quickly, and Altman has commented as much that many companies are building things which are certain to get steamrolled by core product development. But I think there is a very interesting strategic question around areas like law, medicine, and probably more so niche technical areas, where OpenAI will/can not absorb knowledge as quickly.
I think the bitter lesson is true for AI researchers, but OP overstates it's relevance to actual products. For example the best chess engines are very much still very much specific to chess. They incorporate more neural networks now, but they are still quite specific.
Though personally, I think we’re missing whatever architecture / mathematical breakthrough will make online learning (or even offline incremental, I.e. dreams) work.
At that point we could give the AI a robot body and train it of lived experience.
We don't need to ingest the whole internet. I'd wager that upwards of 75% of the internet is spam, which would be useless for LLM training purposes. By the way, spam and useless information on the internet is only going to get worse, largely thanks to LLMs.
Only a subset of the internet contains "useful" information, an even a smaller subset contains information which is "clean enough" to be used for training purposes, and an even smaller subset can be legally scraped and used for training purposes.
It's highly likely that we've reached "peak training data" a long time ago, for many areas of knowledge and activities which are available on the internet.
At this scale you can't overfit. The model might not improve in a meaningful way, but it can't overfit because the amount of data is much much larger that the size of the model.
That said, as the state of the art open models show, the way to get better models is not "use more data", but use "more high quality data" (e.g. look at the graphs here comparing the datasets: https://huggingface.co/datasets/HuggingFaceFW/fineweb-edu). I don't know how much high quality data we can extract from the Internet (most of the data on the Internet is, as you'd guess, garbage, which is why aggressively filtering it can improve performance so much), but I'd wager we're still nowhere near running out.
There's also a ton of data that's not available on the public Internet in an easily scrapable form, or even at all; e.g. Anna's Archive apparently contains almost 1PB of data, and even that is (by their estimate) only ~5% of the world's books.
It seems unlikely at this point, but it it might be the case that new methods, models and/or algorithms rise due to the trend of having high expectations/investments on this area
But yes, AI business stuff will be Swallowed by those companies as much as the companies that unwillingly are providing content to those companies.
The Bitter Lesson is an elegant asymptotic result. But from a business perspective it pays to distinguish problems that general deep learning approaches will disrupt in 1 year vs. 5 years vs. 30 years.
Being able to learn from others' mistakes is generally considered to be a sign of aptitude.
Why not pick databases wrapped in a UI as an analogy? That analogy would go the other way - despite the fact it seems simple and sensible from an integration perspective to have a single database with all the UI a business would need, we have a bajillion specialist SAAS products which then jump through hoops to integrate with each other? Why? Because the workflows are so damned specific to the tasks at hand, some "generic" solution doesn't work.
Computer power has been increasing all the time, but that hasn't kept people from experimenting with the limited power of their time (which ultimately led to better solutions) rather than waiting for the more powerful machines.
Don't focus on the method of solving the problem until you have identified the problem and all the current solutions and avoidances.
E.g. you can build a drawing program where you can tell it in natural language to apply certain effects and make some changes. But another company might be working on a OS assistant that can help the user in any program currently running on the computer.
Or you can build a robot that can pick bolts from an assembly line, but at some point there will be a company building a cheap robot that can do anything from folding t-shirts to knitting a sweater.
The best application developers deeply understand the interface design space, the quirks and limitations of the underlying technology, their users’ mental models, and how to bring all of this into alignment to deliver a worthwhile product. I see no evidence that LLMs are on the precipice of acquiring (much less mastering) these capabilities.
I don't see the general-ness of AI overcoming the challenges there...cause there is a lot of non-obvious messiness there.
On production side of "AI" (I don't love the term being thrown around this loosely as true AI should include planning, etc not just inference) the only question is how well do you solve the one problem that's in front of you. In most business usecases today that problem is narrow.
LLMs drive a minuscule (but growing) amount of value today. Recommender systems drive huge amount of value. Recommender systems are very specialized.
The nuance is that even though AI researchers have learned this lesson, they are still building purpose built AIs because it's not possible to build an AI that can learn to do anything (i.e. AGI). Therefore, building on top of an existing AI model to meet a vertical market demand is not that crazy.
It's the same risk when building on any software platform. That is the provider of the platform may add the feature/app you are building as a free addition and your business becomes obsolete.
The previous administration had Harris as the "AI czar". Which means that nothing was expected to happen.
The following administration has Sacks as the "Crypto & AI czar". I'm not aware that Sacks has any particular competence in the AI area, but he has connections. So government money is likely to flow, presumably to "defense" startups.
This All-In podcast has paid off: From peddling gut bacteria supplements and wrong explanations on how moderator rods work in nuclear power plants to major influence.
These AI wrappers run ship engines on a kayak. For example, most coding companion's ignore my code base the moment I close other files. Somehow their context is minimal. Such systems can be improved dramatically just by changing the software around the LLM.
But I get it, you have to be quick and don't waste time on MVPs. If you get critical mass, you can add that later... hopefully
Even if it is true that the next generational change in AI is based on computational improvements, how can it be true that it's hopeless to build products by specializing this generation of tech?
Moreover if I specialize gen 1, and gen 2 is similar enough, can't specialization maintain an edge?
There seems to be a timescale mismatch.
Thus starting with engineering effort to fit some of the AI limitations make sense but realize many of those will be temporary and will be replaced in the future.
But there is always a new tech or framework or something that emerges after you start and adopting it will improve your product measurably.
Please correct me if I'm wrong, but to my understanding, every single one of these "AI" products is based on a model, i.e. it's just a normal program with a very big configuration file. The only thing "AI" about it is the machine learning performed when training the model. At the generation stage, there is nothing "AI" about it anymore. The AI is done already. What runs is a normal program.
I know it's used in computer vision, recommendation algorithms, and generative software like ChatGPT, Stable Diffusion, etc. I don't know if there is anything besides these.
From what I know, the biggest problems with AI is: 1) the program pretends to be intelligent, when it isn't, e.g. by using natural language. And 2) the program doesn't give the user enough control, they only get one text box prompt and that's it, no forms, no dialogs, so the product has to somehow generate the right answer from the most uncontrolled piece of data imaginable.
These two things combined not only limit the products potential by giving it unreasonable expectations but also makes it feel a bit of a scam: the product is a program, but what sells is its presentation: it has to present itself as being MORE than just a program by hiding as much of its internal workings from the consumers and covering its warts with additional layers of logic.
I don't know if the author would consider natural language to be a "hardcoding" create to temporarily solve a problem that should be solved by using more compute (AGI), but to me it feels like it is.
The best application of AI is probably going to be using AI internally to solve existing problems in a given domain your business is familiar with rather than try to come up with some AI solution that you have to sell other people.
1) Have AI turn natural language interactions into programs
2) Use test-driven development on the domain data
There is a third thing that’s far more crucial but if you want to find out what it is, contact me. I’m building a general-purpose solution for a few domains.
Unlike the other two platforms I built (Web 2.0 [1] and Web 3.0 [2]) I am not planning to open-source this project. So I don’t want to discuss more. I will however say that open source is a huge game-changer — because organizations / customers want to be able to be in control and you don’t want to be reliant on some third-party platform for an API provider. We all know how well that one turned out for startups building on Web 2.0 platforms.
Anyway, I don't follow what you're premise is about the relationship between "open source being a game changer," but then "customers want control," and then "startups are not successful on Web 2.0 platforms". Maybe you're trying to cram too much insight into one sentence and I'm not seeing what the thesis is
That’s why I built these open source solutions, so our customers can know that it will never change, THEY choose the hosting, THEY own the data, they can even hire a different certified developer or agency to add features or maintain it. They have all the choice, and they have a direct relationship with all their members.
> "Flexible is better. Why? Because specific has been consistently worse"
I mean, I don't deny the historical trend. But really, why is it so? It makes sense to follow the trend, but knowing more about the reason why would be cool.
Also, I feel that the human cognitive aspects of "engineering some shit" are being ignored.
People are engineering solutions not only to be efficient, but to get to specific vantage points in which they can see further. They do it so they can see what the "next gen flexible stuff" looks like before others.
Finally, it assumes the option to scale computation is always available and ignores the diminishing returns of trying to scale vanguard technology.
The scale requirements for AI stuff are getting silly real fast due to this unshakable belief in infinite scaling. To me, they're already too silly. Maybe we need to cool down, engineer some stuff and figure out where the comfortable treshold lies.
I don't think the bitter lesson is (or should be?) that you don't need to engineer biases in because with enough compute you could learn them instead ... The real lesson is that finding the right (minimal) set of biases is hard, and you should therefore always learn them instead if possible. Systems that learn are liable to outperform those that don't because by letting the data speak for itself, they are learning the right biases.
I think this is why historically the bitter lesson has been correct - because people have built too much into AI systems, got it wrong, and made them limited and brittle as a result.
The same thing is continuing to happen today with pre-trained LLMs (a fixed set of patterns!) and bespoke test-time reasoning approaches (of which at most one, likely none, are optimal). Of course it's understandable how we got here, even if a bit like the drunk looking for his lost car keys under the street lamp rather than in the dark spot he lost them. Continuous incremental learning (no training vs inference time distinction) is an unsolved problem, and the basis of intelligence/reasoning is not widely understood.
I'm not familiar with this concept of a minimal set of biases.
The way I see it, it's a series of loosely community-defined tresholds. If there's something like a theory that defines those biases in a formal way, I would very much like to read it.
My point is that the more biases you build in, especially with a poorly understood goal like AGI, the more chance you have of either getting it wrong, or simply of over-constraining the model. In general less is more - you want minimal necessary set of biases to learn effectively, without adding others that it could better (providing greater generality) learn for itself.
I also believe the original bitter lesson essay is correct. General methods on more powerful compute will supersede anything specific built today.
However I believe the lesson in this blog post is an incorrect conclusion. It is roughly analogous to "all people die, so do not bother having children because they will die one day." It is true that what you build today will be replaced in the future. That is OK. Don't get caught out when it happens. But building specific systems today is the only way to make progress with products.
Waymo started some 12 years ago as Google Chauffeur, built on all sorts of image/object recognition systems that I am sure have fallen to the bitter lesson. Should Google have refused to start a self-driving car project 12 years ago, because generalized transformers would replace a lot of their work in the future? No. You have to build products with what you've got, where you are. Adopt better tools when they come along.
Remember, most of these startups are grifters. Only few of them really believe in their products.
That's a success in anyone's book.
The article gave the TL;DR as below, for those who skip to the comments:
Historically, general approaches always win in AI.
Founders in AI application space now repeat the mistakes AI researchers made in the past.
Better AI models will enable general purpose AI applications. At the same time, the added value of the software around the AI model will diminish.Most of the things you build will end up in the trash can anyway. But what matters is what you get while doing them, like domain knowledge, improved processes, actual customers..
If you are ok to scrap your code in order to integrate a better model then fine. Since your model is better and does not need all the software around it, it will take 10x less time to re-do what you have, leaving you more time for things that also matter, like sales.