atleastoptimal

I don't think it's outrageous when many of the people who claimed it was a next-token predictor have been proven wrong repeatedly over the past 5 years. There were people years ago who claims AI could never answer questions like "what would happen to a ball on a table if I moved the table" correctly because its text-base world model could never intuit physics, or that it could never do math or code accurately.

When I say there is some issue with people claiming there is some fundamental limit on the capacities of LLM's, I don't mean to say "If you think that they don't have unlimited potential you are wrong", I mean "you can't use the architecture of the transformer to make a sweeping declaration of things LLM's can or cannot do without empirical evidence, because the empirical evidence has unearthed far more surprising revelations than a reductive theory has been able to"

"Next-token predictor" is one of those phrases used most of the time with a motive to downplay the abilities and faculties of AI models. It is intended to trivialize LLM's and imply that there is some fundamental limit on their capacities.

Relying on it as a mental model for what LLM's are minimizes the emergent properties of scaling. It's like imagining that unicellular life could never eventually evolve into complex multi-cellular organisms because individual cells are just "survival and next-mitosis optimizers"

It seems clear AI has the potential to perform any cognitive task at far greater speeds, reliability, and scale than any human. The question is whether it will be allowed to scale to that point, and what will happen to humans after this occurs.

AI can do things it wasn't trained on and has intuition, given the mathematic proofs frontier models have created.

Claiming that AI can only reproduce a subset of its training data is 2022-level and a tired, outdated claim.

For this to be a lie, it wouldn't just be Sam Altman lying, it requires the entirely of OpenAI and METR to also be in on it. Do you think they would all fabricate an incident, staking a huge portion of their reputation on it, so they could spook people? What is the incentive, there are way more straightforward ways to lie about AI capabilities than a complex loss of control incident which now makes them liable to damages against huggingface.

If OpenAI can't control their agents, then what's gonna happen when open-source models are at the level the lab's models are now, and there are billions of agents tasked with an innumerate web of goals, spanning the web, working endlessly, tirelessly to eek out every iota of economic value? How will the slow, human-paced web survive this?

Nobody's is

>HN disbelief syndrome: Any evidence of scary AI capabilities are made up for PR

What justification do you have that this is made up?

What predictions would you say he is not wrong, but just early about?

If his issue is that he is accurate on something shady going on with the finances of AI companies but the effects are mitigated due to government intervention, he should say.

Your explanation gives him more credit than I think he deserves. He has been unambiguously incorrect many times over the past years in the time scales over which he makes predictions.

In general, claiming that a prediction is just early in an unfalsifiable claim. It allows no admission of being incorrect whether you are correct or incorrect in a given moment. If you are right, then good, you've made a correct claim. If you're wrong, just claim there are clandestine corrective forces keeping the disaster you are predicting at bay. Either way you make it out clean and still have some sense of legitimacy.

Most tech starts civilian, the military ones are rare exceptions.

As for your specific examples

>Robots/Drones

Autonomous robots aren't good enough yet for military, those which are used are teleoperated and thus not in the realm of AGI companies

>Surveillance

Not really a military matter, and of course we have Palantir which is doing this directly. Perhaps OpenAI/Anthropic have contracts with respect to government surveillance we don't know about

Someone who only knows finance trying to apply that logic to the singularity. There is no comparison. AI does not obey money, money obeys AI, or rather, the operating force of commodified intelligence will not just evaporate when inconvenient obligations for payment come around.

Also it seems evident the article is largely AI written, which makes it even funnier.

You really think that legitimate concern about AI is equivalent to mass hysteria? Do you think AI will just stop getting better?

The endgame is that everything will be gated by costly signals. What will remain costly as LLM's automate white-collar work?

I adjusted the numbers to net.

> The privilege to work in the Bay Area?

In the bay area you have mild climate all year, high quality public transport, culture, food. I do agree that rents are too high given a lack of development, but for many people that is worth it. The bay area is one of the few places in the world that pays fresh college graduates 120k+, so naturally there will be high rents, and to be in that environment is a privilege especially in growing a network.

If you're a young person making a certain amount of money I'm certain you can plan to save up money over time. Especially if you're a dual-income couple, you could save money before you plan to buy a house or have a kid. A 2 bedroom is affordable on two 100k salaries in the Bay area. but even then the entire area is very large, not everyone lives in SF or wants to, or would need to commute there as there are jobs elsewhere.

It's not accurate. I don't know where people get these numbers from, perhaps they just vibe based on headlines without looking at the actual data.

Studios all within 1 hour of SF by BART cost at most 2,500 but usually around 2000, cheaper if you're in Oakland/Daly city. 2000*12 is 24000, so if you're take home salary is at least 73,000 you're making a healthy multiple over rent. Just making 120k in the Bay area you're taking home 84,000 per year.

So if you're making 200k you're taking home $10,700 per month, so you can afford a 3,500 rent at most, which could easily get you a nice 2 bedroom within 30 minutes of downtown SF.

>You need a 200k salary to live without roommates within 2 hours of downtown SF

That's a bit of an exaggeration. You could rent a 1/2br in SF proper with that salary. You could easily live within 40 minutes of SF in East Bay/Daly City comfortably at 120k.

Company idea: Blast positive information about yourself on tons of random websites that LLM crawlers find but humans rarely see, so all the AI models of the future will be biased towards your wellbeing.

Distribution is a big moat and value-add.

Good work environment >> Remote work >>>> bad work environment.

However most companies who insist on full RTO are often the latter but claim they are the former.

Is the end result of this is that anything that is useful to AI going to be permanently more expensive, as the value a human hobbyist can get out of a certain amount of RAM will always be far less than the value a frontier AI model can get out of it?

The most likely outcome of AI + data buildup is a system of perfect price discrimination. Models will know exactly the absolutely maximum you'd be willing to pay for a service, and price everything offered to you at one cent less. All consumer surplus is siphoned into the hyperscalers.

Protocol named Finger

The whole data center heat/water thing is like not even in the top 100 risks/issues related to AI, but it's all people seem to care about. Why is that?

AI would oversee the creation of those physical things, in the same way humans + machines do now

The problem is that in most domains we reward effort sometimes even more than results, especially in fields where results are hard to verify. A wall of text is the classic way to signal effort and delay feedback, a signal which was costly pre-AI, but now is not costly. However norms have not shifted.

They are only destroying the books because they are required to by copyright law. They obviously wouldn't do so if they were allowed to merely copy the book and preserve the original.

The fact that essential commodity providers aren't the emperors of the universe does not imply the same thing could not be true about AI.

Electricity needs transformation to be useful. AI is the transformation layer, a sufficiently advanced AI system could secure an advantage in practically any economically relevant domain. If you have the best AI in world where embodied robotics are sufficiently advanced, you also have the best doctors, lawyers, engineers, logistics network, PR, weaponry, tactical leverage, etc.

An unintelligible but correct proof is better than no proof. These first AI proofs may be overly complex and un-elegant, but they are the worst that frontier math proofs will ever be. AI math in 2030 will be leaps and bounds ahead of humans both in rigor and elegance.

If humans have nothing to contribute then shared understanding is a pointless endeavor. It makes sense now in the "centaur" period where human + AI > AI alone, but when AI mathematicians are both more rigorous and more elegant, then taking the time dumbing down their proofs to a human level of understanding is like requiring that we ensure all our current proofs be understandable by a monkey.