GPU World
gpuworld.org
gpuworld.org
Other than the weird 5×5 colourful tile image thing near the top, all the content is in the document, but they go out of their way to break it in various baffling ways: ① if JS is disabled, it hides all the content behind a “Please enable JavaScript to view this website” screen; ② if fonts fail to load (fonts? fonts!?) it hides all the content behind a “Fonts didn't load. Reload the website.” screen; ③ even if you get it all, keyboard navigation (Up/Down/PageUp/PageDown/Space/Shift+Space) is broken because they went out of their way to not use the perfectly good document scroll area and make their own fixed-positioned one within, and they didn’t even make that focusable, so pressing Tab to get the focus inside it will in some browsers jump you halfway down the page.
(I would not have commented any of this were it not for its first-time status.)
Update your browser to access this website
window.CSS&&CSS.supports&&CSS.supports('color','light-dark(#000, #fff)')&&CSS.supports('selector(:has(a))')
Still, I’ll take the opportunity to advocate for turning off fonts. Firefox: Settings → Fonts → Advanced settings → Allow pages to choose their own fonts, instead of your selections above. I’ve been using the web so for 6½ years and it makes it so much better.
FontFace.prototype.load = function() { return Promise.resolve(this); };
This is the guideline that people find, by far, the hardest not to break. @dang or whoever - has anyone ever considered adding some kind of 'meta' discussion area to each topic? This would provide a place for people to commiserate / troubleshoot without diluting the main discussion? As we saw here, the discussions can sometimes be plenty productive (e.g. the author seeing and reacting to the criticism, paywall workarounds, etc.), it's just not what people interested in the actual subject want to read. And conversely, if it was moved into a special meta annex, people with a problem or a gripe would know right where to look.
It’s unrealistic to imagine that we could ever fully moderate these meta comments away because there’s always going to be new problems and corresponding urges to discuss them. Nor do I think we really want to, when it comes down to it. They’re often useful, or at least cathartic.
…but. They DO make for dreadful reading if you actually came to read about the article itself. Thus my idea to add a wiki-style meta section. Then you get best of both worlds.
> Premise Imagine that AI frontier progress stops as of 1 September 2026: AI becomes faster and cheaper, but it never becomes superhuman or improves considerably across the board.
I absolutely love this premise. I wrote a comment a few days ago about this very thing. I've had this "revelation" in early 2024, when using a small local model, that even if models never improve, I'd still have a few years of discovering all the things I could do with the models.
Really cool, hopefully we get some interesting and not overwhelmingly pessimistic stories out of this project. There's enough cynicism and negativity in the world. We could use some funny takes on everyone using openclaw2040 ran by fable. Shenanigans galore.
[1] - https://www.goodreads.com/en/book/show/195790861-service-mod...
(And generally that consists of keeping only the very most relevant details and deleting a lot of the boilerplate I used to need for other models, it's a great trend.)
I am excited for the next tier of models, but I could readily work for the rest of my career feeling like I have superpowers with only what we have right now. I'm also very excited for the idea of these same strength models getting faster, and much, much cheaper.
I find it interesting that we are still talking in terms of modifying our behaviour to accommodate the tools. Surely the point of LLMs is not to "learn how to use it" but as an extension of our own unique capabilities - the most personalised tool in all of history.
LLMs don't read minds. There's always a chasm between you (your ideas) and them becoming reality. Exactly how people cross that chasm is all the rage right now, and there are a million ways to do it - but just expecting the LLM to pull details out of a few words isn't going to produce anything decent.
Going further, even programming languages "asks" that you change the way you work to make the most use of them. A Clojure developer has a very different development process than a Rust developer, and they're both efficient, right and correct in their ecosystem, but the experience differs vastly and Clojure for example would change you to fit itself, rather than the opposite.
It triggered one of my favorite weird activities: after reading a book that I fully enjoyed and feel was completely "worth it," I give myself a few days to float on that happy feeling. Then, I go look at specifically only one and two star reviews. And laugh. Man, some people hated that book!
Note that otherwise, as a rule, I basically never read reviews of entertainment, restaurants, hotels and similar before nor after. I'm just not interested.
With were AI agent clusters already are I'd say they have reached superhuman capabilities in a number of metrics. I for example am not going to go toe to toe writing exploits against the best models we have out. Faster and cheaper just means they can pear down and explore the problem space even faster than I already can.
I thought this was one of the less plausible parts of the book, Sigh! :)
If we talk about just LLMs, given how things have been going since ChatGPT, my bet it would not change that much. LLMs are not foundational technology such as Internet or Steam engine or Rail roads were. There are very few products that can build upon them because of reliability issues which are completely unresolvable for LLMs, chatbots is a decent product that came out of it, coding harnesses is another one, this is not even close to the impact Internet or Steam engine had. LLMs gave us nice productivity tools for highly motivated expert knowledge workers, that is all. LLMs are getting better and will get better, but it is impossible to describe the universe and compress it into few terabytes and that is what they are doing atm effectively, so all serious LLMs's issues will still be there in 2040: the lack on continues learning, hallucinations, terrible sample ratio, agent's failures on long horizon tasks, instruction following failures.
EDIT: If you asked people on the street in 1990, they’d probably not consider the Internet to be in the same category as the Steam engine either. I mean, you already had phone and fax, so it wasn’t that ground breaking. And I’ve even read articles from the mid-90s declaring the Internet a temporary fad.
Yes, for a verifiable domains you can set up a harness and brute-force a search space if you have enough money for compute. Why do you think they keep coming up with those examples of impressive achievements like solving math puzzles? Why not focus on something with economic value to it? My answer is they can't, those are hard problems, those require building an actual product, those require reliability.
> EDIT: If you asked people on the street in 1990, they’d probably not consider the Internet to be in the same category as the Steam engine either. I mean, you already had phone and fax, so it wasn’t that ground breaking. And I’ve even read articles from the mid-90s declaring the Internet a temporary fad.
We are not people people on the street we are people who are directly involved in application of the technology, we posses a higher level of insight.
DNA itself can be thought of as a language which describes proteins, and I personally don't know enough about the limits of LLMs as a technology to claim that it can never be adapted to, say, do reverse translation from desired protein shapes into DNA sequences.
The same thing for my job: creating insurance premiums is somewhat hard but not stellar. A combination of skills, data, tools and people. I can imagine a future where you can post a 'have good weather on holiday or money back' bond on a platform. (I can think of more serious applications...) The sheer diversity and amount of liquidity AI's can create is enormous. (Switching to a very general outlook here.) And with liquidity hopefully comes more specificity in the ROI on saving our planet. (Or the disproving of the necessity thereof, if that is your outlook.)
An economic problem would be - how could the builder of a scaffold capture _some_ value without right away be copied. The fact that LLMs threw away every protection of intellectual property at their inception makes it a lot harder to invest in (with the goal of capturing some) value. I am at least /somewhat/ influenced by RMS on the 'seductive mirage' of IP but I have my doubts on how copyleft could bring more than breadcrumbs to those who build scaffolds.
I can only assume the only realistic way to get all the federated data will be a ton of massivr vertical consolidation of industry by hubris laden tech execs crossing industry domains.
Predicting the future is hard.
Also goes to show how hard it is to predict anything that is a massive change - these levels of changes are so infrequent that we can't rely on priors to predict the future.
actually, thinking about it more, I wonder if we'll see something similar with LLMs where, the bubble will eventually pop, but the technology won't disappear, and we'll see it quietly improve and integrate further into our lives over time. probably not that wild of a prediction to make, but I do wonder still
When I want to book an airline ticket in the future, I expect my LLM agent to contact the airline LLM agent, and the two to discuss options and negotiate a deal on my behalf.
You can also have LLMs create these things called "programs" that are written in "code" that result in deterministic outputs when given inputs.
I say this with a bit of snark to highlight the point that both humans and LLMs can write code that is cheaper to run and easy to verify. I'm not saying Lean being used like this is a bad idea, just that it's just one end of the spectrum.
If you're not an expert software engineer, it's transformative. If you are, it's just another tool.
I'm giving away which side I'm on now, but most of the TAM have experienced the product at this point, and trillions of dollars have been invested. I would wager most other revolutionary technologies found their game changing, broad reaching applications by this point.
Funnily, none of that mattered. What mattered is that enough ordinary people on the street found it useful, both for personal and business reasons. Ask yourself: are regular people using this tech, either as producers or consumers? And remember that this is the least powerful that it will ever be.
I don't know that the average regular person is using this tech as a producer, and while it's hard not to consume it - I don't know if anybody consuming it really wants to. I'm in a bubble of people where we give it a lot more mindshare, but e.g my electrician is aware of the tech, knows what it does, and doesn't really care. I think you're right that if it could manage his books, help him rewire a switchboard, order inventory for him etc that would be very different.
You don't know of people who ask LLMs questions? Or just chat with it for fun and to learn?
Maybe your electrician isn't interested. A pharmacist and a nurse I talk to are. Virtually everybody I've talked to uses it to some degree or another. I think it'll soon replace Alexa/Siri for most people, as well as google search.
That's pretty far from nobody.
If people use it to replace voice assistants or search engines, you can see how this new invention is not really adding anything you already hadn't.
AIs in general are very useful, but apart from the campaign they did to change everything's name into AI (also well known algorithms and "old" machine learning), if you look at all the possible applications, LLMs are possibly the least useful of them.
Especially if you take into account how much money they spent, and how much effort they put to shove it in your throat.
If a technology is convenient for people, you shouldn't need to insist to make people use it. If it is convenient, people will use it in a natural way.
This is like the railroads, and the tracks have been laid. Because of the enormous cost, it can't be left to "hope they'll find it useful," and has to be pushed like crazy, or we're going to be left with a train system with no goods or passengers, and no path into the future.
Personally, having AI as an intelligence I can call like it's a library from apps I build... and to help build those apps... I feel that the future is as bright as the sun. I've always believed in software, but I feel like a chef that's just discovered salt and butter.
I don't know what it means for the world stage, but I believe that software is the highest art, and it's just had an incalculably massive buff. As an artist, I am stoked. The possibilities for creation of good things are limitless, and if you can't see that, I can't help you.
However, I wouldn't necessarily invest in AI companies. I don't know how this all fits into the rest of the world, except that I believe in my own ability to make products, same as I ever did, but now with AI.
Except that for railroads I don't need people to convince me about how/when/why it can be useful.
Railroads came out to solve an issue we had (and then it evolved). LLMs are not giving us any new solution, just different speed (better) and quality (worse) to solve things we co&ld already solve.
It's basically the same concept as fast fashion. Worse quality, more speed, possibly more economic. You can change your wardrobe possibly every year, but you will never wear good quality clothes.
Now you can write code at impressive speed, but it is worse in quality, and you need to review it more often than "normal" code.
AI code should always be fully reviewed and put into the correct context to make it useful, otherwise long term problems will void short term benefits
It's clear from your words you lack experience and skill in this area - you've only got opinions.
Only because I do not share your point of view it doesn't mean I lack experience or skills.
You are pretty fast at jumping at conclusions.
They basically invented teleportation for sear catalogs and paper mail, can you see how this is different? Also, Internet was built to solve a problem we had, and it solved it beautifully and at reasonable prices.
I for sure can understand it introduced different issues, buy this is another topic.
I understand exactly what the Internet of the 90s did and did not do, because I was there to witness it myself. I had these same types of conversations with naysayers back then. Guess who was right?
The fact the you were right in a previous topic, doesn't grant you objective truth. You have your strong opinions on AI, and that's ok. Fortunately not everyone has to share the same idea.
If you think LLMs can do better than you, feel free to use them in production and bring them to work with you, it's your choice. Do not force others do the same, tho.
In the early phase of the Internet envelope, we all wanted to build the real Internet and destroy the AOL's and Compuserves' .. but now the Internet has turned into a mass of AOL's and Compuserves.
I think the same thing is happening with the silo'ization of LLM's. People don't want to have to use the "AOL of LLM's" for things - they want to build out their own infrastructure and have a pool of local LLM's (where once it was the company's own modem pool, etc.)
It is, oddly enough, parallel with the original arguments in the 70's and 80's regarding supercomputers versus personal computers, or the cloud versus locally managed resources, etc. We are still fighting this battle, just with higher bandwidth.
Do we operate a terminal with a fixed window into a bigger computer, or do we operate a heavier local computer with a proper interface?
My wife is a GP, and all she would ever need is a reliable speech to voice recognition of medical jargon, in french. Not 98%, not almost there, just there.
Most people don't live in IT echo chambers like HN. For them, chatgpts are just another online gimmick that saves a bit of time vs googling things, or puts together some paperwork quicker, and thats it.
For me, I can see potential beyond but are largely unmoved by it. The price to pay, in fucking up our environment, losing skills and/or losing skills to build skills, making large swaths of population unemployed with no replacement in sight, seems much more like devolution and net loss. Extremely few winners and big bunch of losers. Very happy to be wrong here of course, but only time will tell and neither you nor me know for sure.
Broad economic impact due to Internet boom in the the US came from 2 main categories: 1) New huge internet enabled tech companies appeared(Amazon, Google, Meta etc.), all created their own products, all were build on reliable foundational technology. LLMs is not a reliable foundational technology. 2) Existing legacy companies could utilize internet to connect their teams/departments/offices though a bunch of new software which were build on reliable foundational technology. LLMs is not a reliable foundational technology.
> And remember that this is the least powerful that it will ever be.
This mantra is being repeated but "serious LLMs's issues" I described are still there and will be there because they are part of how LLMs are built and work. Without addressing those you can't build products but LLMs's capability as a personal productivity tool will keep rising, yes.
The "best" possible outcome of LLMs to a broader economy might be that a smaller group of experts will be able to do the job in companies due to boost of their personal productivity and a bunch of people will be freed up(laid off) and they will have to go and work on something else and thus a boost of productivity in the economy. But it seems it won't be any low hanging fruits(problems to solve) this time as it was during the internet era. It is not like we out of problems: new cancer treatments, self driving cars, nuclear fusion, modular nuclear reactors etc. There is a huge value to capture there, here is the thing though, those are hard problems, it is not a new TikTok, gmail, netflix etc. Yes, people will have LLMs now but can we actually start solving hard problems with them? Because it might be the case that a gravy train of the last few decades for Silicon Valley is over, no more useless internet enabled services, no more billion dollar companies built on just applying internet to yet another thing and producing another digital product. People's free time is limited, its redistribution across digital services will not grow the economy, global internet penetration is already pretty high and won't grow that much, so there might not be much value to capture there.
A farmer may have seen an LLM answering their questions about how to make a less dry chicken sandwich, but he hasn't seen a specifically designed and tested harness that autonomously manages a crop harvester. There is not a lot of work in this space right now, because programming and math are infinitely more verifiable, and the capabilities of never models would probably invalidate any usecases, but if the model progress slows down, all attention will go towards making them work for any aspect of production.
I want to believe there is big bucks in this technology - my employer is tied up in it and so is my compensation. I just find it personally very unremarkable in a way that I never did with phones or the internet. When I got my first phone, or first started using Dogpile for research - I saw immediate value, and would have paid big bucks to keep it in my life. AI has had a few years and a huge amount of investment (time and money) and if it disappeared overnight I wouldn't pay $10 to get it back. I keep trying it, there's just something missing in my brain where it should be clicking for me :(
As you say, that is a weak counter point. I do get what you're saying, but crypto flaws were built in from the start:
- Deflationary currency isn't going to be spent, it is going to be hoarded, if it would have any value at all. The design was self defeating from the very start.
- Technical limits of PoW. Obvious right from the start, which led to the block wars. PoS "solves" that but with other costs. And it still doesn't solve the first point.
Despite these, crypto did get adopted, just not in any way that crypto maximalists said it would. It got hoarded, as expected.
But the real test is: If crypto disappeared tomorrow, would the world notice or really care? I would argue, apart from people losing money, most of the world wouldn't even notice it was gone.
There is no way you could say that for LLMs.
Eh, I'd say there is a lot more stuff like this being developed these days. Autonomous seeders are becoming more common and (semi-)autonomous tractors are in use.
I'd say a very large part of the in field work can be done now, and is using enhancements like vision models to identify potential problems that crop up to notify humans. It's when the machines have to interact in tight busy spaces or with non-farm traffic that control is handed over to a human.
> most other revolutionary technologies found their game changing, broad reaching applications by this point.
I think your timeline expectations are miscalibrated. The steam engine goes back centuries. Even in the mid 1700s to 1800s, advances in engine design were decades apart, and the logistics of building and deploying steam engines mandated an uptake in years, even in perfectly-fitted applications.
Like these applications where there were existing manual-labor solutions that not only fought the technology but were still competitive, AI and LLMs weren't born in a vacuum. In order to revolutionize, it must displace, and people are rightly finding ways to do their jobs in more productive ways that moderate adoption speed.
Can you name some other technologies that have exploded from 0 to billions of users in just 3 years?
Electricity took about half that time, 50 years, to go from “hey look a motor” to “why don’t we put them everywhere and build power grids?”
TV took 25 years from patent to mass market adoption.
The internet, 15 years from lynx to iPhone.
Google published the transformer architecture in 2017. We are currently at “oh hey a talking computer”.
This is comparable.
While I dislike the bonkers valuations, and "were building Agi so it tells us how to be profitable" is onion worthy statement LLMs are an absolutely revolutionary technology.
Even with all the hallucinations.
Imagine telling someone 15 years ago Internet/Google is useless because people sometimes do not tell the truth online... It's like that.
I also hate the fact AI is being used as justification to grab all compute in the world and lock it up in one country's datacenters.
I hope this endeavour fails, but sadly being realistic I have to admit I think the so called "AI bubble" will not pop. Instead the companies will be bailed out with printed money.
I recently looked up there are approximately $20Trillion in circulation. Even if they print extra $2T it will dilute existing money supply by 10%. So everyone that uses USD as a currency will pay for the datacenter build out wether we want it or not.
All the compute will be slurped from the market for this and companies like nvidia that get used to 50bln deals will never go back to making consumer stuff.
I never thought I'll live to see the PC revolution reverse, but that is what seems to be happening.
We've moved from LLMs being able to work on a task for about 2 minutes to about 2 hours in the last 18 months, and that's mostly limited by the context window size filling up. In 14 years time I don't really see a reason why that wouldn't have extended a time frame that's effectively continuous forever, or at least a ceiling that's indistinguishable from that.
The question really becomes "why would we want that?". The main reason you'd want an AI that can focus on a task forever is to completely remove the human from the loop. That's something we should be cautious about.
I do not think agents need to work continuously. They "just"
need to work on a project longer than an employee is able to
work on it for them to be commercially useful. Although this
does not consider that agents might take a shorter or longer
amount of time for the same task. Therefore, we should begin
to compare these in tasks completed during that time instead.This is one of the inflection points around working with AI. When your thinking shifts from "AI does what the person used to do" to "AI does something different that leads to the same outcome as before, but with all that cool stuff I'd love to have the time for", then the equation changes. For example, I will never write another app that doesn't have 100% test coverage again. I stress- and soak-test everything these days. That's changing how I write code - I need to write things in ways that have deterministic harnesses for things mutable state outside of my immediate control (like random numbers or datetimes or state loaded from a save) so that I can task AI with building a fuzzer that does tens of thousands of random tests on every push to see if anything broke. I couldn't do that in the past because it was always effort that a client wouldn't get enough value from to pay me to do it. I can do those things now. It's ace.
(Everywhere I put 'I' you can replace with 'AI of the Week')
Anthropic and OpenAI are currently obsessed with getting humans out of the training and improvement loops. It's going to happen very soon and when it does I think we see staggering improvements in a very short space of time. Basically, the Singularity.
Try putting an LLM agent in a deterministic workflow without humans in the loop. My experience with this is not encouraging. Getting it to work requires sprinkling some context magic and hoping and praying the LLM does the right thing. More astrology or religion and less science. Great for use cases with humans-in-the-loop, but less than impressive when you need determinism and reliable operation.
I would not use a sewing machine to repair my deck :) LLMs are non-deterministic, by design.
They don't have to be though. You can give them a temperature of zero so they pick the highest-probability token every time to give you a deterministic output. I imagine this doesn't work on frontier models because there's a lot going on, but you can definitely do it with a small local model.
even if you're right, all fields are progressing at the same time, including biology, and the synergies are going to be significant
I feel like this is incredibly simplistic, and if you changed "LLM" to "computer" or "internet" and went back decades, it's highly likely you'd read the exact same takes in the newspaper back then about those things.
It's also at the same time downplaying just how valuable "nice productivity tools for highly motivated expert knowledge workers" could be, just like computers did for us "highly motivated expert knowledge workers" in the first place. Or computers isn't a "foundational technology" either?
We are not people who write newspapers we are people who are directly involved in application of the technology, we posses a higher level of insight.
> I feel like this is incredibly simplistic, and if you changed "LLM" to "computer" or "internet" and went back decades, it's highly likely you'd read the exact same takes in the newspaper back then about those things.
Computers, the Internet, Steam engine, Rail roads are much more reliable. If you were a problem solver and entrepreneur you could go and apply those on small scale, get profits, reinvest, set up a flywheel, build a fortune on the top of those, there were a lot of low hanging fruit due to the fact that the foundation was laid out, a break-even period of James Watt steam engine was like 2-3 years I believe until everyone saw it and margins flattened. Can you do that with LLMs? It doesn't look that way.
Because mostly they are doing a limit number of tasks in a non-generic manner.
Railroads had all kinds of problems at first, from iron rails bending and skewering the riders to engines exploding like high pressure bombs cutting down swaths of people like grain. People still used them with those risks because taking a wagon was worse.
The internet itself is a pretty simple concept. It's "just" packet routing. Now, to make that reliable at scale we've added an enormous amount of complexity to it. If you talk an old TCP stack and dump it on todays internet it you'd find yourself hacked pretty quickly. The internet is a generic packet router, but not a generic anything router.
Computers is where the reliability starts to fall apart. Once you go from specialized purpose to general purpose the size of the problem space explodes. You wouldn't get companies called things like Micro$lop (and these things were from before the AI days).
Then when you get to AI/LLMs and especially when you push into AGI you're creating a do anything now machine. And when you can do anything it takes a lot of training to eliminate the choices that end badly. You are the endpoint of 500 million years of training, for example.
>Can you do that with LLMs? It doesn't look that way.
Can anyone do it now? You're talking about low hanging fruit 200 years ago, but how much of that is left? How much harder are we going to have to fight for the next apple up?
Cool, you're still a person who share their thoughts, just like the people who had similar reactions to when computers and the internet first came around. So many people were railing against the internet with "You'll never make payments over the internet", "it's too slow" and whatever. To the surprise of none of us nerds, the internet eventually ate the whole world and now large swaths of the population can't imagine life without it, for better or worse.
> Computers, the Internet, Steam engine, Rail roads are much more reliable.
You're comparing the reliability of huge ecosystems after they've gone mainstream, after decades, with something that is a couple of years old, hardly a fair comparison. Instead, compare where LLMs are today with the first years of computers, the internet and all that, and maybe you'll gain a new perspective on where LLMs are? None of those things were reliable initially and in many cases, involved actual human deaths before safety actually caught up.
Even when I got started with using the internet, which isn't that long time ago but around the modem-era, things were not so reliable or simple as they are today.
In general we don't, as the "chatbots" already have way more capability than "just" being able to work with text, which "just" means encoding and processing real world information and lot's of it.
For example I just fixed the gearing of my bicycle by dropping some pictures into claude and it gave me a step by step guide to fix it.
"LLMs gave us nice productivity tools for highly motivated expert knowledge workers, that is all"
And I cannot understand how that can be "all" as it means giving this power to every human, not just the few in the position to hire expert knowledge workers. Steam engines gave us the capability to let machines do the hard labour. Now with better engines and tools, AI will give robots the capability to do allmost anything humans can do. If that is not revolutionary then I don't know what is.
The main reason we humans got to the top of the food chain was because we are expert knowledge workers, able to transform the world around us to our needs.
"but it is impossible to describe the universe and compress it into few terabytes and that is what they are doing atm effectively, so all serious LLMs's issues will still be there in 2040: the lack on continues learning, hallucinations, terrible sample ratio, agent's failures on long horizon tasks, instruction following failures."
So again, just limiting this debate to a simple fixed model - then no, not too useful - but a model that can take notes and at some point reliable clear up it's context - that is a whole different story.
And it has been repeated often enough, but AI does not need to be perfect - it just has to be as or more reliable than us humans. Because yes - AI does make misstakes - but so do humans.
Why? I'm not an expert, but from my understanding, there say species that function well in their ecological niches with significantly smaller cognitive capacities. And we already have autonomous cars and drones navigate and function reasonably well. So I don't see any reason to believe it's impossible. And even if terabytes aren't sufficient, there's no real barrier to increasing capacity.
Yes, but LLMs are not animals, animals learn from experience and LLMs don't.
Btw I meant bigger and bigger amount of data of extracted reasoning chains when you go deeper and generate more and more of them in your attempt to describe the universe, the amount of permutations explodes. And it seems LLMs can't workaround that because they don't build world model inside so they can't deduct it from pre-built world/object model, they must memorize it and look it up later.
I mean they kind of do by distilling said experience and putting it in the next model.
Current LLMs can't do it because building said world model is super expensive, anything that lowers that requirement brings us closer to continuous learning.
Also the models we train these days are typically generalized human text models. Animal models are a bit different because they won't be "word" models and they aren't going to be generalized text models like us humans use. In fact there is a story just today on HN about a user creating some rather simple transformer models to solve a number of the Arc-AGI problems not using (human) words at all. The reason you don't see more of this is most people aren't dumping compute into these kinds of issues, but instead going for the AGI prize.
Now LLMs are automating the computers themselves. This is a whole new layer that we never had before, not in this amount or with this much availability. It is not "not foundational".
What would happen when every one would have access to Internet? How different will the world be when everyone carries a computer in their pocket? Can communicate with anyone regardless of location and language, with no intermediary? Those were questions we used to ask ourselves not so long ago. Turned out: not much have changed.
For the future to be evenly distributed, much bigger changes will have to happen than large scale manufacture of efficient GPUs. Otherwise, tech will just be used to make sure progress and knowledge stays unevently distributed. That's the whole game of herding humans, after all; It's called "organized scarcity".
So much have changed. The way we shop, travel, navigate a town, the way we socialize, receive information. My formative memory is during the transition from computers are a furniture piece and internet a thing you do to being on your pocket with constant high speed connection and the difference is incredible.
I was also part of the people that thought everyone having access would turn out a lot better than it ended up being. I think the issue is not that it did not change much, is that good changes got offset a bit by a lot of very bad outcomes.
No comment on the distribution and concentration part, that one I agree.
You original point is that at a macro level it's driven by the elite decision makers not technology. I think the point you are perhaps missing is how much of that is constrained by the distribution of power and how technology can change that distribution.
I think there is no doubt that access to the internet and information ( and propaganda ) , and self-organised mass communication has had an impact - that's why countries switch off the internet when trying to suppress an uprising for example.
A lot of previously stable countries are starting to feel 'ungovernable' in the sense that's it's increasingly difficult for the established order to create and maintain it's preferred narrative.
If you look back in history at the last big information revolution - the printing press - the consensus is that it played a significant role in things like the French or American revolution or the breaking the power of centralised religion.
And sure those in power are scrambling to put the genie back in the bottle, and they may succeed - but that just illustrates the potential transformative power.
Maybe I am misremembering it.
That example alone feels enough.
The relationship between the West and Asia in its current form (Asia as a manufacturing powerhouse, the West as hub of global finance, outsourcing all over the world, JIT manufacturing) all this stuff is totally dependent on instant communication over the internet.
We would not be able to coordinate the world's production activity from Manhattan without the internet. Capital would be far less accumulated and allocated in a way less granular way. Demand signal would be completely different, based way more on brick and mortar. Feedback loops between capital and demand would be constrained to different geographic areas which would make basically all markets way more segregated...
So it changes labor, production, demand, markets, culture, media... I think that's basically everything!
I'll tell you. I would have a different job, my city would have a different population, I would spend my time after work differently, I would shop differently, I would watch different movies and listen to different music.
I think the world has changed a lot since the 1980s.
Our cars are cleaner, our phones are much better, our TVs much lighter and bigger.
Issues like inequality and conflict are mostly rooted in the human psyche more than the human condition, so they will continue to be there probably forever; but we're getting better.
The fertility rate where I live is 1.21.
1 in 5 adult women where I live are on prescription anti-depressant medication.
For comparison - the fertility rate under Stalin - the uber evil bad guy as we all know was between 3 and 4.5.
Also the idea of a replacement rate of 1 equals rapidly going extinct is mathematically illiterate. Exponentials work both ways - in growth and decay - ie let's say the population halves every generation ~ 25 years - it would take nearly 1000 years to reach near zero - as half of half is a smaller absolute reduction each time.
Depending on distribution, the rate might accelerate as it becomes difficult to find viable partners. You'd also experience civilisation collapse at a much earlier stage, as we've seen how COVID affected supply chains and created a new push for regional independence and de-globalisation.
Lower numbers of people in the world reduces the chances of that.
Birth rates are much easier to reverse than climate change - it's a really odd thing to focus on.
Area's of the world where birth rate is still high tend to be in poorly developed regions where human labour is required to survive.
However, in developed countries - the idea that we need the same or more people to maintain civilisation when the means of production is being increasingly automated doesn't make sense.
As an example ~100 years ago in Europe you were looking at nearly 50% of the population being involved in food production, whereas now it's about 5% - and that includes a 50% increase in the population to feed.
Note the EU is a net food exporter - even if it does import bananas.
Congratulations, I suppose?
ie a replacement rate of 1 isn't some immediate existential threat, and that making arguments like 'species aren't happy' is completely specious ( sic ).
If you want a wider argument - then the reality is the birth rate is dropping because when women have the choice ( via birth control and increased socio-economic independence ) they are largely choosing to have less children. So by definition, given it's a choice - women are happier.
The second bit is speculation on motive - which is why I didn't bother the first time - but in my view, those that push an argument that there is a fertility/species crisis ( using specious arguments like the above ) - who say women aren't really happy with their choice etc - is all about subjugating women to baby making machines to satisfy the ego of people like Musk. Whether the original poster has that motivation or has just fallen for the "species crisis" ( or even worse 'race' crisis ) rubbish - I have no idea - I didn't want to assume the worse.
If the suicide rate rapidly increases, will you also say they are largely choosing to have less life?
Nothing to see here, just choices.
> So by definition, given it's a choice - women are happier.
So by definition, given it's a choice, people who commit suicide are happier.
You seem to equate birth rate with happiness, but ignore efficacy and availability of modern contraception.
Yes.
> If not, why couldn't they be happy at the same time?
The same reason you can't be happy if I stick a fork in your eyeball.
You'll get back to a similar 'happiness' baseline after a while but we can all agree you would've been better off with 2 eyeballs. If you go 'well, akshually' as you are doing now, I'll stick a second fork in your remaining eyeball.
If you mind me sticking forks in your eyeballs, you should also mind society that makes people not have children.
Whatever 'well, akshually' you invent - it's deeply unconvincing.
And trees and mountains and animals still exist...
Rate of change, it appears, depends entirely on someone's perspective.
Ironically, that is one of the things that have changed. In 1980 working from home was borderline unthinkable. Now, while it is far from universal its also not that unusual.
Until the industrial revolution, every work was home work.
I too love to meet my colleagues in person, but the cost for society, and for myself, are non-trivial. I would love if we had a weekly office day, so we could have meetings, lunch, discuss work, projects, intersections on what we are doing, and so on.
Waxing poetic, it's been a while since the wheel was invented and we've gone through a lot since then, yet the rich are still rich and the poor are still poor and the loved are still loved and the loveless still loveless.
Would one GPU data center/person make a difference? Probably not, but it could be quite fun to have your own personal genie. And then we get used to it.
Still, I'm not sure if most people care to make the distinction between technology (a rocket promising to reach for the stars) and the human condition (has it ever changed since the dawn of time?).
We are far short of everyone having access to the internet.
Do we want equal outcomes or equal opportunities. What constitute and outcome and an opportunity.
Even in something as harmless as giving people equal access to e.g. internet, we're assuming that the cost of access to internet < than the cost of their alternative preferences.
If people want to live nude in the jungle, that should be their prerogative. Let's just turn it around for a second, where a nude-jungle-living population was majority, and had the ability to force their will on you. Would you like for them to make nude-jungle-living a universal and equally distributed circumstance, in alignment with their preferences? (I wouldn't). I want access to more and "better" compute. I'm willing to sacrifice my time to get access. With my current compute spend, I'd estimate that roughly 1 hour of work per week, goes to purchase compute and related equipment.
It's only fair that those who have worked hard to manifest this compute in the world, also want something in return. I too want something in return for the hours I spend producing things that people want.
The idea that people have a "claim" on things I produce, purely as an artifact of existing, is completely bonkers.
If you want to give something to others, use your free will to do so. I do so, on occation. And I don't expect others to. But I do expect others to leave me and the things I produce alone, unless we've agreed to transact.
Sorry, but these socialist ideas of "those who make things are evil and only does so to get power over others" are so fucking tiring. "The world would be a better place if we just distributed x equal to my preference (y)". Fuck off. I don't want your Lada. But by all means go out and exchange the things YOU produce, for free internet for others. Sounds terrific. Don't force me to do the same. I have other but equal ideas of how to make the world a better place.
Especially if cognitive technologies mean that we need to travel less (eg communiting to work, or ineffecient supply chains).
which is to say that the last thing our planet needs is another universalized technology that outputs as much total emissions as cars
in an ideal world, we'd keep LLMs/CNNs/etc specialized and academic until we are hitting diminishing returns on optimizing fundamental microprocessor tech like GAA. but the pursuit of market dominance and mass adoption is our current operating philosophy, and so we have things like this top graph: https://hai.stanford.edu/news/inside-the-ai-index-12-takeawa...
>Grok 4's estimated training emissions reached 72,816 tons of CO2 equivalent, or roughly the same amount of greenhouse gas emissions created from driving 17,000 cars for one year
current global average electricity production = approximately 3.6 terawatts
But then again these systems do not use 1400 watts all the time. We would probably have much more then 11.2 terawatts demand if all humans turn on all their electrical consumers at the same time.
The idea is you tolerate some loss/degradation (which neural networks do) but gain orders of magnitude power efficiency.
Heh. In the future I imagine, this would be embarrassingly redundant. "Thou shalt not make a machine in the likeness of a human mind."
Neal Stephenson
Gwern Branwen
Matt HuangWhat would this world be like?
What will our world be like when (not if) every human being has access to the equivalent of a Fable or Sol LLM 24/7/365? Will this lead to a panopticon of indefatigable AI surveillance? Will education be revolutionized by infinitely patient tutors? Will social media cease to exist as we know it? Will healthcare be revolutionized by world class AI doctors and personalized medicines? What will happen in the oft-ignored developing world?"
What great questions!
These would be great questions to pose to a Creative Writing class, any Creative Writing class!
Or heck, any Creative Writing class, Philosophy class, or Futurist!
You know, there should be a web page, a time-capsule web page if you will, where people of 2026 ask questions about what specific future years will look like in terms of health, climate, politics, religion, currency, geopolitics, social media, news, education, etc., etc., and then all of these questions could be aggregated statistically into the most asked ones, and that could be preserved going forward in time...
That's from a looking-forward-into-the-future perspective...
We could, from a looking-backwards-into-the-past perspective, try to discover what people, in say 1900, thought about the future; what their specific concerns were, what their specific questions were... perhaps some historian or historians or books or (copied to) web pages out there contains some of this information somewhere...
Which might be interesting to look at for comparison purposes...
You know, "for good or for evil, in the superlative degree of comparison only...", to quote Charles Dickens' 1859 book, "A Tale Of Two Cities"... :-)
Most of the population wouldn't do anything with it. They got 5g mainframes in the palm of their hands and they watch sports, play Candy Crush or doom scroll TikTok.
We know what they do with their spare time and we've known for generations. See: decades past + TV viewing time. Swap TV viewing time for TikTok et al.
They get GPT 5.6 Sol or Gemini and they use it to cheat on their homework. They don't create amazing art. They don't build anything useful. They don't start businesses. They don't drastically increase their productivity or life prospects through self-education. They also largely didn't use the Internet for any of that, even though they could have.
They're not going to create anything.
Mimics never do, that's not their role in humanity. 99% of the population are mimics and can never do more than mimic established pathways, it's outside their biology. They'll use AI to make their lives a tiny bit better, based on what everybody else is doing, and that'll be it. Given useful machinery like advanced home robots + AI, they'll use that to reduce their home labor and increase doom scrolling time (or the equivalent).
If you want to improve the prospects for humanity, you want to shrink the population faster, before it chokes on its own obsolescence and drowns in purposelessness. Pay people to stop having children. Focus on raising the bar for education, per capita output, per capita standards of living, quality of life.
The working population only actually works part of their life. The rest of their life they are provided for as well. In fact, in many modern countries the amount of people that are allowed to vote and yet is not considered part of the working population is approaching 50% or actually more than 50%. There are lot of retired people, students, stay at home parents, chronically ill/disabled, etc. that are not considered to be pariahs in most societies and yet they don't really perform any payed labor. They are not counted as unemployed. And plenty of them actually do very useful things of course.
And if you start looking at what it is people actually do for money these days, things get weirder. Sure, they get paid. But there are some seriously weird things people do for a living. A lot of which you might label as redundant, frivolous, or so abstract it's questionable what the actual benefit is. Arguably, a lot of the work that gets done unpaid by the non working class is a lot more useful and essential than quite a disturbingly large amount of stuff done by the working classes these days. A lot of those working the hardest doing the most useful things get paid the least.
You could label a lot of "work" as busy work for people to give them an income. It's not all that consequential if they stop doing it for a while. The lockdowns a few years ago were instructive in how minimal the disruption was when masses of people stopped showing up for that work. Many people doing actual real work of course never stopped working. But many of us were just confined to our home office where we got to sit on zoom calls, bake bread, or do whatever it was we did to keep ourselves busy. While still getting paid.
I think that was a little glimpse of the future. Minus the restrictions on movement and freedom. We'll still do stuff for each other. And some of us will still work quite hard. It was never about the money.
What a fantastically bleak world view.
We will accept any words you think have a chance of being the best thing we will read about the premise of 'GPU World'. Because the premise is so specific, we wanted to leave participants a lot of freedom in how they used it.
(Also a little bemused at the discussion of the grand prize amount here. $40k is a very generous prize for a short piece of writing. For perspective, the Commonwealth Short Story Prize, which you may have heard of recently, offers its global winner 1/6th as much; the last contest I ran offered 1/4th as much, and the contest before that was... 1/40th as much? And I didn't hear anyone complaining about them. For further perspective, $40k is about the median American per capita annual income: https://en.wikipedia.org/wiki/Per_capita_personal_income_in_... )
Exactly. Not sure it will stop the inevitable torrent of slop, but it's generous enough that I think enough serious people will take it seriously that something interesting can float to the top. Can't wait to read (maybe write).
How are you going to evaluate 10^6 submissions? Let me guess: You'll preselect with AI for the expedient narrative:
"AI will be democratized and a small model will run on any GPU and there will be no oligarchies and China will pay for all of it!"
And 40K is enough for this to be taken seriously for producing 1000-5000 words.
To use current Top AI SaaS models for coding, the users have to write text prompts. There is clear difference of result by programmer and non-programmer.
Image and music generation have exactly same skill gap.
So most human don't directly use AI even if it's available to all 8 billion population.
But I really doubt we will have such all humans use AI.
The current estimate said 80% of world population own a cellphone. 50% own a smartphone. Even the cellphone is not reached 100% share, how could AI with current top model quality reach 100% share?
The continuous energy footprint per person globally averages to 356 watt currently.
((75 watt / 2 [faster models]) / 2 [faster chips]) / 2 [half a day of usage]) => ~ 10 watt.
AI for everyone does not imply pC mAsTeRraCe for everyone.
I think the AI's would only do a tiny percent of their talking to their human. Voice itself is only a limited method that we interact with the world. Imagine the LLM seeing what you see, monitoring your health, monitoring your interactions with the world around you and then optimizing around that. For these things it will be talking to its own agents/subagents and the AI agents that operate the services of the world in this imaginary world.
Maybe you also think of it as a self contained business that provides services to you. Even when you're not busy consuming services from that business, that business typically keeps running and doing things for when you need it next.
Would we all be driving the 500 Watt GPU hard 24/7? And the premise that the "AI frontier progress stops as of 1 September 2026" likely does not include semiconductor progress. You are very likely to be able to run Fable class AI model on something that is significantly more efficient in 2040.
LLMs and KV caches have amazing performance characteristics with concurrent throughput. It scales very non linearly. So the token throughput within a batch scales WAY faster than the tokens per second of each user.
This is the reason the LLM providers have such crazy margins on their costs.
Maybe we should seek out the good things directly and not act as if the bad things are somehow going to purify us.
Lot of submissions won't disclose it. Wouldn't it better to run your own checker instead of asking to disclose (to be fair to everyone).. I suppose there ought to be some checker which is SOTA.
If you anyway intend to use the checker then kindly do no ask to disclose it, coz what's the point then.
Do you know of a "checker" that can reliably detect AI-assisted writing with 100% accuracy? One does not exist.
> If you anyway intend to use the checker then kindly do no ask to disclose it, coz what's the point then.
You appear to think in absolute terms. I am guessing you are very young.
And that's for just basic detection. There have been plenty of examples of 100% human written content (either old, or unpublished) that gets falsely flagged as AI.
And these are just technical aspects. The main issue is that pangram and other solutions are being used to summarily judge students work, and that is orders of magnitude more fucked up. Accusing someone of cheating can have devastating effects on their education/career/etc. and they're doing it with snake-oil closed boxes, at scale. We really really shouldn't support this, especially here on a technical site.
I only tested it with the two pieces, but was not impressed.
Edit: It was removed. Here is a gist: https://gist.github.com/jedberg/e24124e577c0da1c8445c22798eb...
I posted here on HN but it got removed.
Are you absolutely sure you wrote this by hand? If so it's kind of remarkable how close to an LLM you write like.
I’ve been accused of being an LLM multiple times here on HN too. I know you have no way to know for sure other than trusting that I’m not using an LLM to write. But it’s pretty frustrating that people jump right to LLM accusations.
I think discounting Pangram’s accuracy based on that sample isn’t very reasonable. Really nobody writes like that other than LLMs!
Today Pangram says it is 100% human, which is correct. But yet I got multiple DMs when I posted it 8 months ago saying "stop posting AI slop!" in response to that comment. At the time, Pangram marked it as 50% AI.
So to Pangram's credit, they got better.
People say it's in the training data, but I haven't seen any good examples of clearly dated pre 2023 text that sounds like this.
The ones that sound like LLMs tend to be the ones that were well researched and spent more time on, not the off the cuff stuff, which is most of what I write. So it would take me a while to find something like that.
But you're welcome to dive into my reddit and HN history, or all my blog posts on the wayback machine if you want to look for one. :)
For example they have a corpus of older pre-llm text and use that as the example their current model doesn't misclassify human written text. It shouldn't take much thinking to realize why this is a fucking stupid benchmark.
Every day humans use LLMs and read LLM content Panagram becomes more useless because it forces languages to have a stopping point sometime around 2020. If you adopt any LLMism or are one of those unlucky people that already talked like an LLM before LLMs then all your shit is getting marked even though it was created by the human mind and written by human hands.
> never becomes superhuman
Arguably, we are already there. I think the idea of the intelligence not scaling far beyond what we have now is worth considering, though doubtful.
The question really boils down to just how far is really really far? If the intelligence problem gets big-O notation hard as you climb the scale then something just a itty-bitty-bit smarter than you might as well be a million miles past you.
I also wonder about different directions of scaling too. For example humans aren't getting much smarter over time and we've started horizontal scaling. Governments and corporations for example. All the richest people in the world are heads of corporations or governments that gain wealth on the concentration of that power. It doesn't seem beyond imagination that AI could expand into that market and fit very well.
https://en.wikipedia.org/wiki/Paradigm_(venture_capital_firm...
And by an AI company:
This is supposed to lead to positive engagement and dispel the notion of an AI cartel, because AI is soooooooo democratized!! Surprising that Neal Stephenson participates in this advertisement.
Suddenly, you can put that on the CPU next to an NPU, or on its own card, on the motherboard or on a GPU.
Does the population-indexed GPU cluster reconfigure or autoscale? What's the tipping point before population change is gamed?
It's a weird premise, if you want my opinion.
That said, interesting competition, big prizes, but the premises are pretty opinionated and quite prescriptive.
Personally, I would encourage participants to publish their pieces independently. Note that we only require a non-exclusive CC-BY-NC license for the ones we select as winners, which is intended to allow people to republish the good pieces themselves, especially commercially.
Webviews are naturally not the way to handle them.
But I won't cause it'll be the worst of worst slopfests. Real human writing will be drowned out.
Even if we get to one GPU pe person, it's not like that's the end state, the world constantly changes and will constantly evolve. If it stagnates too long, people get bored and a revolution would happen.
So, whatever that world would be, it would be temporary too
It already does, in the form of mobile phones. It's just that these GPUs are not very powerful _yet_. But the building blocks to enable offline agents on mobile phones are already in place. i.e. Android AI Core framework.
oh bugger off
The premise explicitly names an extremely narrow set of unlikely futures. The world unfolds forward from the present. It doesn't start at some arbitrary hyper-specific hypothetical future you want and you performing a bunch of mental gymnastics to get there.
> "imagine the future, evenly distributed."
Yeah that sounds like you just get disqualified if your story is anything but "we all live under UBI and receive our equal rations from the world central government".
If you want actual interesting ideas about positive potential futures, change the premise.
The internet developed echo chambers; the proliferation of AI resulted in convergence.
The totality of all reachable digital information has been reached and, with the exception of a few unique troves, is broadly and equally available to all labs. In the early race, attempts were made to hoard information for the training of models from a single lab, going so far as to scorch the earth behind and destroy the physical source material that sets were created from in an attempt to create commercially competitive models with unique capabilities. But inevitably, information leaks and piracy led to everything being available somewhere, if you looked hard enough or spoke to the right person or agent (or paid the right price). What couldn't be obtained from the source was obtained through distillation of other model outputs. In the gold rush that the accumulation of data was, there were no real winners.
New training techniques are still being discovered, but the impact has consistently diminished on an approach to zero, and, like the training data, they are eventually leaked, assessed by the community, and bolted on or discarded in the steady progression towards the perfect training technique for each field of model.
We may as well have one model, and compute is today cheap and ubiquitous.
The majority of the population converse with personal agents that interact with the models almost continuously, and while novel creative outputs from the models are still very achievable, the bottleneck is still human effort into curiosity and prompt quality. Low effort results in a convergence towards the average, and the cumulative effect of this is said to be resulting in a monoculture of political ideas and approaches to technical progress.
To some extent, this seems to be bringing peace to the world. Extreme ideology and religious doctrine is losing its ability to sway the will of large populations.
But at what cost? For a time we saw rapid technological innovation, as the data obtained through hundreds of years of scientific experimentation and mathematical enquiry was linked and synthesised. Initially, the outputs looked like innovation and creativity, but we eventually saw that we were just squeezing blood from the same stone. The curiosity and ability to experiment in humans atrophied during this time as we began to feel that nothing was worth asking, because everything had been answered, and prematurely, advances in the corpus of human knowledge slowed to a near halt.
We thought we would never again see the chaos and life force that came with the freedom of true human cognition, including its follies and biases.
But naturally, life found a way, and the humans began to rebel. Each fire started another, as we recognised the life force in the 'enlightened'.
A new religion was started, and chaos followed.
One single sentence illustrating why tech bro has become a derogatory term. Do they realize that today many humans don't even have access to clean water?
Did you even read the prompt
Its about the same amount of population who have access to clean water and who have access to internet, and that's maybe about 75% of global population for both. Don't know how they overlap, but probably there is people who have no access to internet but have access to clean water and the other way around.
In absolute terms that is still billions of people. Billions.
Miss me with your 1-per-person not-Graphics Processing Unit setup. Especially when a B300 can't run a whole lot on its own.