Hacker News Reader

Comment by StizzurpXDD | Hacker News Reader

This is not just Anthropic. Almost all big AI companies, including OpenAI and Google, hide their model's actual reasoning. This is because revealing the raw reasoning exposes exactly how the AI processes information. These companies spend in huge amounts on R&D to develop a thinking process that is superior to their competition. Exposing those thinking mechanics to competitors would completely defeat the purpose of their spending. They simply won't do it. It's like you telling your exact location to someone who is trying to hunt you down.

View on HN

Or like providing the world’s information in machine readable format that the AI companies can convert into model weights without getting permission or compensating the rights holders

I don't pay for my mind to absorb the world's information, either. And when I publish to the Internet, or give a talk, I also typically don't charge. Even when I publish under some kind of copyright restricted licence, that restriction has never (by law) extended to restricting transformative use that you might perform using your mind.

This idea that absorbing information requires paying a toll needs to change. It was never the case in copyright law anyway (and the courts are beginning to agree). Even if it were, copyright law was founded on the basis of encouraging creativity by creating an economic incentive. Appeal to "compensating the rights holders" therefore needs to be based on the economics, not just some principle about "rights" that never applied to this case anyway.

I mean you probably cannot even do 284882 * 282817374 in your head, let alone billions of matrix multiplications so that argument doesn't make sense. If you used a trillion square feet of paper to do the calculations as extra steps to plagiarize something then it should be treated the same as doing it with a computer.

"Your text batch moved the weights away from the final values. Your contribution is negative."

Where do I collect the $0.00000012 antidollars owed to me by OpenAI for my valuable inputs?

Slightly more seriously, you could perhaps make an argument that, just like weight decay, an apparent "anti-contribution" moves the learning trajectory along, and helps the network settle into a more optimal basin eventually.

That way, my contribution is still valuable on the net, and I'm owed $0.00000003 positive dollars instead.

>you could perhaps make an argument that, just like weight decay, an apparent "anti-contribution" moves the learning trajectory along

Was that not the joke?

More to the point - if they expose their model's "thinking" inference, competitors can train on that to replicate the results. If they postprocess that content, e.g. by summarizing it, it's no longer as useful to competitors.

Exactly. Google won't like it if they spend millions to make Gemini 3.5 Pro's thinking the best in the world, only for Anthropic or OpenAI to copy it by just seeing the thinking process.

Copying for me, not for thee

It’s only ‘fair use’ if you have the money to argue your position.

> This is because revealing the raw reasoning exposes exactly how the AI processes information. These companies spend in huge amounts on R&D to develop a thinking process that is superior to their competition. Exposing those thinking mechanics to competitors would completely defeat the purpose of their spending. They simply won't do it. It's like you telling your exact location to someone who is trying to hunt you down.

I thought the reason was the "reasoning" didn't work very well with "aligned" model output, so they had to remove the alignment during reasoning and then hide it to avoid exposing "unaligned" model output.

Not sure if anyone remembers the brief 12ish hour period when the very first “reasoning” ChatGPT model went public, but it provided credible evidence for this.

Before the massive nerf (showing summaries and suppressing certain aspects of reasoning) you would literally see reasoning text appearing on your screen like “while xyz is true, these facts may be seen as supporting hateful rhetoric or a conspiracy theory which is against my policy guidelines. i should tell the user xyz is not true or steer the conversation in a different direction. according to my instructions misleading the user is permitted in certain contexts where sensitive information is being discussed or could cause liability”

They disabled it shortly after the first screenshots appeared online, and restored it the next day in a way that hid what was actually happening.

This right here is why I will never subscribe and, as an American, I hope the Chinese kick our butts. Maybe being second place to China will force American AI to dispose of these morality/safety guardrails.

Any mainstream consumer product based on LLMs is going to put guardrails around them of some kind. China might give you different guardrails, but it's a bit naive to assume that a Chinese company would impose fewer restrictions overall than an American one.

I extensively use open source Chinese LLM’s for coding. Reading the reasoning traces, especially when planning and debugging code, is valuable. I will pause the llm when immediately when it’s made incorrect assumptions. Often I see I mention erroneous stuff in the reasoning that doesn’t show in the final response. And I’ll copy/paste phrases directly from the reasoning traces and explain why this is incorrect.

the key word is consumer product. apps can (and should) set their own rules but models need to stay neutral and capable of producing harmful content.

they should never generate it unless asked to by the user but its important that the capability is there and users/app developers can turn off all guardrails if they want to. open source gives you a guarantee that if one version drops without censorship you can keep using it forever even if its replaced by a censored one on the api.

I think you’re overestimating the market for such models. Most people don’t want a model that’s prone to generating extremely offensive output. If you want something “uncensored”, then open source models already exist, as you say. But the model itself has already been extensively tuned to produce desired outputs and not produce undesired outputs, so it doesn’t really make sense to distinguish “uncensored” raw models from “censored” apps or harnesses.

There’s a big difference between uncensored models and those that are specifically set up to only output things a panel of wealthy ivory tower Bay Area progressives would deem to be the “correct” or “inoffensive” take on a given topic.

Sure, but ChatGPT and Claude aren't actually like that. They're perfectly happy to express right wing political views if you prompt them to do so.

If you're genuinely worried about 'censorship' in this context, look first at how US AI companies are working with oppressive regimes around the world (e.g. https://sherwood.news/tech/report-openai-may-tailor-a-versio...)

> Any mainstream consumer product based on LLMs is going to put guardrails around them of some kind. China might give you different guardrails, but it's a bit naive to assume that a Chinese company would impose fewer restrictions overall than an American one.

Exactly. The GP must have his head up his butt. The Chinese have far stricter guardrails on their models than America does. I mean, FFS, the country famously has a massive censorship apparatus and regulations to make sure the police can show up on your doorstep if you start talking out of line.

> while xyz is true, ... i should tell the user xyz is not true or steer the conversation in a different direction.

That's disgusting, abusive and manipulative. LLMs hiding the truth and gaslighting the user to reduce the corporation's liability is absolutely unacceptable. It means they are agents of the corporations, not agents of the users.

Hope local inference advances as quickly as humanly possible. I wonder if there's anything I can do to help speed it up. I could share my prompts and sessions.

> It means they are agents of the corporations, not agents of the users.

Of course they are, assuming otherwise has always been naive.

I suspect that you’re both right in the sense that ‘aligned’ is an important component of ‘superior’ from the vendors’ viewpoint.

The cynic in me is wondering whether it's more about how revealing how the sausage is made might bring bad publicity.

It's to mitigate their competitors ability to run distillation on their models. The only advantage frontier models have is being at the frontier.

There's nothing in the reasoning tokens that'll give bad publicity that the final output already wouldn't do.

Imagine if their target customers, C-suite execs looking to replace workers, knew how unlike "thinking" this process actually was! we can't have that.

To be honest I'm not sure if many C-suite execs have a good idea of what "thinking" looks like inside in the first place, in the sense of focused mental activity aimed at solving of a hard logical or technical problem.

How did they became C-suite execs in the first place, if they don't know how to work on problems?

By talking a lot, usually.

When you export your personal data Google hides all model responses leaving just user messages. So it's even worse

> Exposing those thinking mechanics to competitors would completely defeat the purpose of their spending.

I think one of the reasons could be to limit liability too.

What if reasoning helps in establishing provenance for questionable sources ?

What if reasoning and model's "thought" points to fundamental issues in how the model was trained to produce certain problematic responses ?

But that makes the product worse because for any complex problem the road to the solution is important to be reviewable.

Correct on all points. Nonetheless this leads to a less useful product. I

f we want more useful products, we need to come up with ways to disincentivize this behavior. Even if doing so poses an existential risk, we are better off if companies taking existential risks to please us is a necessary being a top player in this game.

There are actually fine tunes of qwen on opus “thinking” tokens that teach it to think like opus does.

https://huggingface.co/Jackrong/Qwen3.5-27B-Claude-4.6-Opus-...

And those are "amateur hour" distillations that don't have the scale of actual Chinese labs.

Mistral displays some “thinking” text (in their basic online chat interface) in the thinking mode, do we know if those are the real tokens?

It’s quite interesting to read. I can’t imagine using a model like this without the ability to peek inside and see if it is getting stuck.

I wonder if they put all 80k tokens of the GDPR in its system prompt.

I dunno, I’m in the US, so I’m not sure how much that impacts their processing of data about me.

I'm in the US and about a month ago Claude decided I wanted UK English for all my answers and couldn't explain why it changed.

One thing I see noone asking, is this not a case of optimization? Hidden reasoning means they dont need to process the output of all that, it stays internal within the model. Less cost for them -> less cost for us (even if they benefit mroe), compared to streaming all of those reasoning tokens out?

My understanding was that thinking still gets encrypted, shared with clients, and reingested by Anthropic with each new prompt [1]. Which means it would cost more than normal tokens, since it has to be decrypted/encrypted with every transaction.

[1] https://blog.cryptographyengineering.com/2026/05/29/fooling-...

Edit: other comments under this post seem to indicate that thinking tokens are cached on the server side as well? I'm a bit confused.

I think the reason it's encrypted is so if you continue a session after it is out of cache it can be reingested.

And I think all the output is signed or something as well so that you can't modify the agent's response in your submission, which would would open many more model jailbreaks. For local LLMs it's really powerful to be able to modify the model's response to save tokens when it gets something wrong, or at least it was when they were a lot dumber.

> They simply won't do it.

They should be required to do it by force of law. Why is it that they can train on copyrighted works and then lock down the model? This contradiction is unbearable. Nobody cares how many trillions they spent training the model.

> Nobody cares how many trillions they spent training the model

People definitely care that they spent trillions. Establishing the precedent that you can make big load-bearing bets and fail is extremely threatening to oligarchs. They would sooner twist the law into a mockery of itself and doom the world to the institutional distrust that breeds than accept a loss.

The optimal outcome for humanity is to have the oligarchs spend their entire fortunes training a godlike AI, only for someone to suddenly leak the weights when they're finally done so that everyone can use it.

correct. this becomes difficult for us to understand what happens behind the scenes.

Reply on news.ycombinator.com