Natural intelligences do this too
It's a relevant comment in this instance because we're discussing concepts you need to be both trained and practiced in to reason about, and that our discipline has traditionally been blind to. Plenty of people working with LLM context issues who've never been exposed to the idea of 'subtext' or could tell you why it would matter to their direction of effort.
how much of the training data had thinking traces that dont make sense to people as being actually a description of why the output should be that way?
A naked "natural intelligences do this too" might be a bit too short to be useful. But if we can add when/where, cite papers, or show ways in which the parallel operates, then it might be useful.
Compare, eg, talking about a robot arm, and someone goes "a natural arm does this too". You can tell about the fact that it has the same degrees of freedom in the same places, or how this pertains to inverse kinematics, or etc...
Same way here, "this happens to be how natural intelligences seem to solve this too! According to Foo, Bar, Baz et al (2026) the gadget is always twiddled beforehand in macaque apes. " or "Same for natural intelligence: I've noticed I use the same general algorithm myself. I've always considered this the correct way to do translation between languages". --
A more concrete example of a useful answer here.
Natural intelligence does this too! When given the question "explain your reasoning" humans are indeed quite prone to post-hoc confabulation. [1]
[1] https://home.csulb.edu/~cwallis/382/readings/482/nisbett%20s... "Telling More Than We Can Know: Verbal Reports on Mental Processes" (this citation is quite old and may have been superseded, mostly just to illustrate how the rule might work)
Project 2025, Maralago accords.
So far the only major policy item the Trump admin seems to have not intended was the Iran War. Israel killing the intended replacement, Iran leveraging the straight of Hormuz, and dropping three Tomahawks on an elementary school really botched that one.
In other words, a lot of this depends on what scale/scope is being inspected. On the high level, the world is chaos rather than a meticulous and inscrutable plan of the Illuinati Shadow Cabal. On the low level, people do things for reasons, even if they're dumb ones.
With respect to the "current political climate", I'd like to suggest that a lot of dumb or seemingly "against their own interests" stuff is due to people prioritizing costly in-group loyalty signals. Their interest in staying good with the tribe is just higher than their interest against a dumb national policy.
It's just peter theil's texting groups.
The ability to conspiracy both willing and unwilling is such a low threshold now, it's virtually indistinguishable.
You watch one billionaire do something and you're like, I'm a billionaire, I should do that too.
The fact that there's so few billionaires, the probability that they conspire together both direct and indirect approaches 1.
The inverse of course is rediciously hard to conceive: the working class bands together to get something like universal healthcare.