Vulcan Materials Company has produced some of the strongest and most resilient economic gains for investors at around 25% gross margins for 50 years.
Vulcan’s business is crushing rocks, and then driving those rocks to where people need crushed rocks.
I mention it because Vulcan isn’t a sophisticated business at its core, but its economic returns are exceptional, because they do their unsophisticated work exceptionally well, and exceptionally efficiently.
Across the economy, most economic gains are created by companies like Vulcan, who do boring repetitive work exceptionally well and efficiently.
I expect this to hold true into the era of AI. Most stuff probably doesn’t need an exceptional model, and paying for an exceptional model to do unsophisticated work will leave your business vulnerable to competitors who take time to find the most efficient model for the task, and undercut you.
Conversely, AI models feel pretty different. People will swap models at the drop of a hat and while not perhaps fully fungible, the cost of switching can be as low as one engineers it to be. In this case, the service does really seem to feel more like a commodity.
Not sure where this leads or how this ends. Perhaps those two will swap poles for me (IE as robotic automation improves, there are more commoditized Vulcans, and as metered intelligence improves, they become less fungible). I just can't quite see how yet. Just food for thought.
If you need image recognition, and a 30B model saturates the use case with 100% accuracy, you absolutely wouldn't continue to use the next frontier model as they come out.
And I'd argue most economically meaningful tasks will be saturated by cheaper models than those requiring frontier.
Think about what today's models can do with pretty close to 100% accuracy, and then consider that they will be orders of magnitudes cheaper over the years.
5.6 Sol can already obviate tons of labor, and why would you pay 2x or more for no meaningful gain?
The relative gap between frontier and non frontier also continues to shrink, so it's not like you take a meaningful performance loss by rewinding to models from 3-6 months ago. And soon that gap will expand to 12-24 months.
I get the impression the majority of people on here only think about coding, which net net will be a tiny volume of overall AI use in the end.
Disagree that the frontier model is where the economic gains will be realized.
The smaller the relative gap between frontier and non-frontier/open weights, the less pricing power.
This gap has shown only to shrink over time, not expand.
Businesses will pay more for frontier, but not meaningfully more to justify the economics. It's always going to be a low margin business, perhaps outside of cyber security, warfare/intelligence and perhaps drug discovery.
Though the expensive and time consuming part of drugs is doing the trials and getting approval, not coming up with ideas
Or to put it another way, there's enough natural variation in real-world bottlenecks that no pharma company can assume they'll beat competitors to market by using a smarter model.
A really smart model could significantly improve the pharma business if it could identify promising approaches to cancer treatment that are less likely to fail in clinical trials, but I don't think that the frontier labs have data to make that work yet. Much of the biomedical literature is poorly reproducible ("replication crisis") and much of the drug-development-specific data is proprietary, never published in the first place.
I do have hopes that general laboratory automation will go faster with LLM assistance, even if all the LLM does is write Python glue scripts to enable custom workflows and instrument integrations.
Where the interesting work will be is at the median point where cheap models do almost all of it but need to hand off some parts to the SOTA/more expensive models. Seems like there's money to be made by maximizing low end use while maintaining quality.
Investors are largely treating these as future monopolies though.
We can already do so much with existing models. Harness improvements are probably more meaningful at this point.
e.g. say most image recognition can get saturated by a model of size xB parameters, so your tool for that can handoff to a smaller model. Document text extraction can use a model of size yB parameters. A model of size zB for summarizing text.
We are starting to get to a point where you can reasonably scope out an upper bound of required size/effort for many common tasks, and if you string these together, the frontier will largely act as an intelligent invoker of more efficient models.
Up until now there have been meaningful gains to each of those types of workstreams by using newer models, but that is starting to no longer be the case.
Yes, I do believe token consumption will rise exponentially from here in the near term. But cost of switching is low, and substantial profitability will be difficult.
Silly nitpick: the reason we don’t do daily cancer scans isn’t the cost, it’s the false positive rate. Invasive procedures like biopsies come with complications like infections that happen at a higher rate and do more damage than the cancer that doesn’t even exist. This dilemma is pervasive in medicine, because our tests aren’t perfect but the thing they’re testing for is rare.
This isn’t something you can solve with more scans because the tests test for data that is indistinguishable. They look the same on a scan, there’s an overlap in the assay with some random protein with the same binding sites that is only present in 1% of the population, the coding gene in one person gets repeated in a noncoding region in another, and so on. The “more data” that works is a doctor applying professional judgement (which they’re also famously bad at because biology is a fickle mistress).
The frequent "muh false positives" comment we hear from doctors appears to be a lack of imagination?
If it looks like a duck, it might be a duck - or a painting of one. If it looks like a duck, swims like a duck, and quacks like a duck? The joint duck estimation is much more confident now. There might be a few more observational tests one should administer before committing to a duckhood decision, but each tests pins down variables and rejects confounders. Uncertainties are cut down, and we get closer to crossing the threshold between "duck-informative" and "duck-actionable".
Thus, it's often worth it to improve observability. If you managed to make a certain test more reliable, or cheaper to administer, or reduced the chance of adverse effects? Or, in other words, improved SNR, reduced costs, and reduced costs? You can get more information for your buck. Paired with good knowledge: you can make better decisions more easily.
The fact that the thought of "having more information might be bad actually" even occurs in the field of medicine shows just how far it is from being optimal. Having more information isn't always beneficial - some information is genuinely redundant. Some information is not worth the effort of gathering and integrating it. But if you get more information and it results in worse outcomes? You're doing something wrong.
And that's not even really touching societal/network intelligence. A single human isn't that smart and can't accomplish that much. Hence we form families, and companies, and societies, and governments. What does a society of AIs look like?
And a society of humans does not increase our overall intelligence. It allows all of humanity to access the accomplishments of our most intelligent members throughout history (which is huge, don't get me wrong!) but it seems almost self-evident that our civilization as a whole is not smarter than Newton/Einstein/von Neumann/etc.
If I dump Einstein on an island at 5 you don't get a theory of relativity. The data provided by a civilization is required, hence why people 10,000 years ago didn't go to the stars. It is unlikely they were significantly less intelligent than us. They just didn't have the language of data we do now.
The thing about playing strength in games like go is that it doesn't map at all linearly to results. It could very well be the case that the bots are only a few points of handicap away from perfect play, but thousands of points of ELO.
I wrote about all of this on Stack Exchange, along with some napkin math, a couple years back: https://boardgames.stackexchange.com/a/61058
Anyway, this is a poor argument for a possible "limit to intelligence" because of course the game is finite (even if the search space is extraordinarily large). Of course you can't beat the "hand of god", by definition. Of course you can create games and puzzles for which "hand of god" is a coherent concept. It shouldn't be too hard to specify games where it isn't, though (perhaps something involving real numbers).
https://analyticsindiamag.com/ai-features/almost-nobody-is-u...