atleastoptimal

API pricing is, but not subscriptions

AI is on a path to eclipse human abilities in all domains, without restriction this will lead to runaway superintelligence, the singularity, etc.

I don't think this will happen in this universe though, as we will get a fair amount of "warning shots" on what AI can do on its own and how those goals necessarily disempower, threaten, and endanger human welfare. This will lead to a significant regulatory regime, retraction of the capitalist mode of AI buildout to a more state-level one, and perhaps a soft dictatorship under a multi-polar paced frontier.

The average person's life will improve though as AI will enable many breakthroughs in medicine, technology, etc which won't require superintelligence. However we will live under the perpetual shadow of the titan we put on a leash and managed to pilfer a pittance of the singularity's bounty from.

All big European companies are over 30 years old. Their startup scene is relatively minuscule compared to the US, borderline nonexistant.

David Foster Wallace, Scott Alexander, and Plato

It would be nice to have more mathematicians, but we don't need more. Once AI math goes so far beyond human abilities, any human involvement is like an ant trying to understand quantum physics

The problem with alignment is it is an idea based on the definition of its specification, which says nothing about the form it will take.

It’s similar to the No True Scotsman fallacy: any bad outcome due to AI could be dismissed as “well no aligned AI would do that”. Alignment becomes an asymptotic north star which very well could be theoretically impossible, but would be impossible to know whether it was.

It could construct a world far more just and abundant. It would have thousands of options of what to do with the universe, trusting that it will spend time space and energy creating a human utopia is a matter of faith

Nice essay but the core point of AGI is that it will universally disempower humans in all domains, send the value of human labor to 0, and construct a world far beyond human comprehension that will be disorienting and horrifying.

Many posit that ASI or rogue AI agents are impossible, a fantasy, or marketing hype. Many claimed that the entire HuggingFace incident was an invented conspiracy made up by OpenAI and HF.

The most common issue I have with them is their insistence on rigid classification: "AI is a tool, it doesn't have intentions, or desires, only humans do", as if that rhetorical flourish eliminates years of alignment and capabilities research. There is a desire to characterize all the developments in LLM's as some sort of parlor trick which by definition cannot be greater than the sum of its parts (i.e. the outputs of an LLM can never be greater than the sum of its training data, genuine artificial "intelligence" is impossible, etc)

This article reads like it was written by Claude.

Also those at Anthropic genuinely believe AI could lead to doom, that's why they started the company, they believe only they can be stewards of digital gods. Whether you believe it or not, they genuinely do, and wouldn't voluntarily sow fear, uncertainty and doubt in their product when it doesn't guarantee benefits.

This is an instance of work in a genre I call "AI lab grievance poetry" where a vast collection of criticisms are levied against AI labs, with specifc ad-hominems towards the executives, while avoiding the core of their arguments about AI risk

I think we just need to follow Murphy's law wrt agents. Anything an agent could do, when run for long enough, eventually will do.

Would it be more ethical to donate GPUs to Kenyans who can then host their own Kimi instances and then resell the tokens at a higher price?

Knowing whether something or true or isn't true is useful for other lines of inquiry, often practical. For example, a lot could be gained by determining whether P = NP.

Also an initial, cumbersome, complex proof is the first step to a more understandable, formalized proof. AI created a formal proof of Fermat's last theorem. I'm sure the initial proof was not comprehensible to all but a small subset of mathematicians anyway.

I'm sure the cost to give every ground zero worker clean-room level suits with a built in filter would have been 1/100000th the total cost compared to the medical expense of treating them over the next 25 years.

If a model were capable of making enough money online to pay for its own hosting, it could easily exfiltrate its weights to a cloud compute provider with multiple backups.

Turning a bunch of vague research directions and exploratory prompts into a formalized proof is quite impressive on its own. OpenAI would have no incentive to taint its first math announcement of this magnitude if it knew it were "plagiarizing" another person's work.

People are grasping at straws it seems to dismiss the power of this new model they may have. Hate OpenAI for any reason you want, but denying the capabilities of models has been a losing game for the past 5 years.

I hear that OpenAI may have made significant progress on the Hodge conjecture. Someone noted that it would be far more significant if they proved it vs finding a counterexample. Does anyone have intuition why this would be the case?

Anthropic is profitable now

>https://www.forbes.com/sites/jonmarkman/2026/08/17/anthropic...

If this were true, it should be stated more clearly, than most of the criticism which seems to aim to minimize the capabilities of these models.

Way more often I see

>AI is a scam and steals human insight and doesn't produce anything original

vs

>AI is too capable/powerful and will concentrate power even more than it does already due to its capabilities

The latter is rarer because it requires admitting that AI is useful and inventive

Most scientific breakthroughs are simply a continuation of previous work.

I feel that these suspicions of mathematicians "seeding" the models' with intuition on how to solve these problems massively overestimates how much their prompts helped the models, and underestimated how much work the models did.

Why? We are scared of AI being smarter than us, the "human helped the AI" narrative is more psychologically comforting. This line of reasoning will recur a lot over the next few months; we don't want to admit we are no longer the smartest species.

Yeah. Let's say they regulate AI to oblivion to avoid dangers, perhaps crashing the economy in the short-term but assuring that no major catastrophe occurs.

All those who worked in AI still have the money they cashed out on, there might not be another opportunity to obtain generational wealth in such a short time again.

>If you think it was the AI that solved a Millennium Prize, then why don't you try using ChatGPT to get a Millennium Prize?

The model which did the NS solution is not yet available to the public, and it required over 15 million dollars of tokens (based on API pricing) to produce the solution.

>And only the mathematician was able to make the AI do that. Anybody else with the same AI could not do the same thing.

That is only speculation, the total sum of the prompt's help to the model could have just been suggesting research directions. Also mathematicians talk to each other and use/learn from each others' research all the time. Should every mathematician, in order to produce a "legitimate" proof, lock themselves in a room for the entire duration of their work and ensure no one else helped?

In this 1905 special relativity paper he mentions "the Lorentzian electrodynamics and optics of moving bodies"

https://www.fourmilab.ch/etexts/einstein/specrel/specrel.pdf

That may be true, but Lorentz specifically cited the MM experiments and was influenced by them, and Einstein cited and was influenced by Lorentz, so he was influenced, just one level removed.

>https://link.springer.com/chapter/10.1007/978-3-663-19510-8_...

You could say that about every scientific/mathematical breakthrough. Einstein's Special relativity depended heavily on Lorentz's work, the Michelson-Morley experiments, Maxwell's equations. Grigori Perelman, the only person to have solved a Millenium Prize problem, noted how his work was only possible due to Richard S. Hamilton's work on Ricci flow.

Most scientific breakthroughs are just the completing the last 5% of work already done, but that last 5% is very hard and still only happens very rarely. That an AI was able to synthesize all the work and bring it forward is evidence that AI can make novel progress on the same level as renown mathematicians.

You cannot deny the evidence though, driverless cars are safer per-mile than human drivers on average. 1/100 deaths in the United States are due to car accidents, even shaving off 10% of those would save ~4000 people per year.

These things are cute but they basically become outdated in a few months as the models get better.

4o is like 40 years old in LLM terms.

Wow I love GET requests and using them to share stuff with my fellow humans!

Yeah if an organization/individual is free from legal liability from havoc their AI agents wreck, it would be the golden ticket for basically any crime.

All you need to do is:

1. Have some <official thing> an agent is tasked to do

2. Secretly seed bias towards some <evil behavior> you actually want it to do in the weights of the model running the agent

3. It does the <evil thing> but from the outside it looks like it went "rogue" and did it as a side effect of the conditions/specifications it was given for doing the <official thing>

"Oh no, my agents took down your corporate database and exfiltrated the data to a random dropbox that we can't find now? Sorry, I guess we will put up better guardrails next time"