Comment by jaykru | Hacker News Reader

Comment by jaykru | Hacker News Reader

Parent Full thread

I wrote something [0] that might answer a bit a few weeks ago. Today's slopdrop certainly is challenging my stubbornness, but everything I've seen so far indicates that these (hugely impressive, world historic) capabilities won't extend past verifiable domains. Math yields especially impressive results because it so broad and deep that essentially no person can know of all of its parts; pretraining and deep search capacity is a huge advantage. Gowers has recently written about these capabilities and gestured [1] toward some human capabilities lacking from the current frontier models, though he isn't convinced they won't develop soon. If you assume we don't get a total mathematical superintelligence (which to me seems already sort of AGI-complete) and only amplify the capabilities we have today, it's not obvious to me that we get takeoff from recursive self-improvement, unless you happen to believe that 1) we can clearly specify what AGI or ASI is 2) all of the requisite ideas are out there and need only be combined and/or optimized.

[0] https://dank.systems/posts/2026-09-15-ai-bear.html

[1] https://gowers.wordpress.com/2026/08/12/what-sort-of-maths-a...

> everything I've seen so far indicates that these (hugely impressive, world historic) capabilities won't extend past verifiable domains

But they’re already extending into politics, military, journalism, art, and many other fields that aren’t verifiable in any meaningful sense of the word.

The magnitude of improvement in unverifiable domains is small, mostly down to models doing more careful research before answering and hallucinating less. They are more thoughtful, but I expect you'd have to drop 2 major versions of Opus before you'd start to see most people really clearly be able to differentiate them.

> The magnitude of improvement in unverifiable domains is small,

What makes you say that? What is an example of a domain where the improvement is small?

I can't think of any at all. Compare something as unverifiable as "Make good music". Models now are many times better than 3 years ago.

How are the models making politics better? I don't count AI attack ads as an improvement.

Is this a serious question?

Improvement in this context means "better quality results".

You can use better quality models to do worse things with.

I'm not making any claim about second order effects like that.

Yeah it's a serious question. What are the better quality results in politics from AI?

My argument is that if you were to compare "analyze XYZ geopolitical situation" or "explain the ramifications of XYZ law" from Opus 3.5, 4.5 and 5.5, the difference would be marginal, at least for 4.5 - 5.5. Almost all the crazy capabilities newer models have is from RLVR variants, whereas capabilities driven by RLHF are inching along.

This doesn't seem true.

I'm not aware of any benchmarks that measure the first "analyze XYZ geopolitical situation" but legal reasoning is very closely related to "explain the ramifications of XYZ law".

Legal Bench[1] measures legal reasoning. It's close to saturated (ie, there isn't a lot of room for improvement) but Fable scores 88% vs eg Opus 4.7 at 85%.

There probably isn't a lot of room for improvement on something like this - there is enough disagreement in legal reasoning to mean 100% is going to be impossible.

[1] https://www.vals.ai/benchmarks/legal_bench

That benchmark agrees with my point, GPT-4o scored an 80% on it. Even if you could construct a much harder legal bench that could resolve smarter models, the bottleneck is getting the absolute top legal experts in the world to give feedback, so progress would be slow compared to stuff like math and coding where you can programmatically generate scenarios and validate/score performance.

These models are spikey as hell, they can gain incredible capabilities but that doesn't make them godlike minds, it's more like a scaled up version of rain man.

Agree about the spikeiness, not sure about the legal example.

I think there is a roof on the level of reasoning needed for legal reasoning, and I think we are pretty close to it. Legal arguments just aren't that complex in terms of logic chains.

I'd bet the cyclometric complexity of any given court case is lower than a dense piece of code for example.

> "analyze XYZ geopolitical situation"

Just realized this is closely related to geo-political forecasting, where AI is now beating the best humans

https://www.economist.com/science-and-technology/2026/09/16/...

https://archive.is/IVreS

> we can clearly specify what AGI or ASI is

We'll have plenty of time for this, while living off UBI.

>> slopdrop

Really? Do better.

this is the term of art in the mathematics community. considering that the vast majority of the results don't come with a typechecking lean formalization, i don't think it's off base at all either.

Reply on news.ycombinator.com