jsrozner

If two consumers purchase the same item at the same time, it should be the same price. That's what we're debating. If an airline raises prices closer to departure, then that's not a violation of privacy, nor discriminatory in the sense we're discussing here.

I do, however, object to firms aggregating any individual's purchases across different interactions. That data should belong solely to the individual and it should be illegal to retain enough information to aggregate interactions across interactions with the same customer.

If you somehow arrange for a redistributive effect. But in practice, the firms are likely to charge each consumer the maximum that consumer can afford. These firms are not engaging in some philanthropic process here.

If you want redistribution, implement a wealth tax.

It can never be better for consumers. The only way a business would adopt this practice is if it leads to greater revenues. On average, that necessarily means worse prices for the average consumer.

> But if you tell them they can't do this profiling, then they'll presumably resort to some mix of (1) no discounts with slightly lower overall pricing or (2) some loss leaders but otherwise regular pricing.

That's not true. Competition with other market participants should in theory (assuming competition) be sufficient. The firms are in general already charging the profit-maximizing price. They could, however, increase profits with more information.

But there is the additional problem that they will also monetize this information by selling it into advertising markets. The whole point of this business model is to capture previously uncaptured value that would otherwise have been shared in the commons.

The best solution is a constitutional amendment that actually enshrines a right to privacy. Among other things, the retention, aggregation, correlation of any personal data should be illegal (including for commercial purposes). (Storage on behalf of users in encrypted form could be made OK. Could also be refined to support retention of data of the medical, legal, etc kind with the attendant non-admissibility protections.)

This would fix this issue, it would destroy the surveillance models of Google/Facebook, and it would fix the Flock issue, etc. It would also fix the Roe v Wade issue: women would be able to get abortions in the first couple months of pregnancy without the possibility of harassment, since law enforcement would have no capacity to detect pregnancy until then.

Also, tech won't save us had a podcast on the dynamic pricing topic: https://podcasts.apple.com/us/podcast/how-data-is-changing-a...

god, fk this

The people producing and consuming the slop don't realize that it is slop.

I, uhm, think this idea would apply in both directions.

It turns out, oddly enough, that it's possible for it to be both. AI can simultaneously be used to destroy the commons with slop and also make contributions to new math (though isn't the jury still out on whether part of the idea was stolen from human mathematicians?)

It is hilarious that this got downvoted, even when it's entirely factually accurate. I encourage the downvoters to respond to the content.

Shame is the best driver of social change (see, e.g., Rob Reich's The Common Good).

Now we just need the same shame for broader surveillance capitalism (Google, Facebook, AI companies, etc).

A bigger problem is that it will be impossible to know when you are seeing an ad: political groups (or the government, perhaps) will partner with openAI to subtly express different values.

It's still an advertisement, and the underlying marketplace is similar (pay for access to change behavior).

The best uses of AI will be surveillance, propaganda, cyberterrorism, and automated military tech.

See, e.g., https://www.nytimes.com/2026/09/18/technology/iran-china-aut...

This is basically what Google did when they pioneered the model of surveillance capitalism (see, e.g., The Age of Surveillance Capitalism).

Google simply provided an index on top of an existing library. Of course, a librarian has no value if he has no books to index over! But it's also worth noting that the Google "librarian" also leveraged the existing "social" structure of the internet: their core contribution (page rank) was a clever, efficient mechanism to extract the latent value in the pre-existing link structure of the internet. This structure (much like the pages themselves) had been curated by actual humans. Undoubtedly page rank was clever, but it was worthless without the existing websites (books) and the existing indexing information (the pre-existing, crowdsourced librarian work). Nonetheless, they successfully monetized it.

AI companies are even worse in the sense that initially Google was still sending traffic to the original webpages. (Until they didn't - https://www.eater.com/2017/9/12/16294380/yelp-google-scrapin...). So yes, the AI companies have even more thoroughly stolen the collective work of humanity than Google did.

Because it's another surveillance adtech company from SillyCon Valley. Duh.

I keep meaning to set this up but haven't. What's the simplest best setup for my own DNS blocker?

People think they're interacting with an "intelligence," when actually they're just getting a maximally optimized Weizenbaum feed. We're living through the sloppification of the human mind.

See e.g., https://www.science.org/content/article/ai-chatbots-are-beco...

Cars are decidedly less dangerous than tanks, which are less dangerous than nuclear weapons. I am certain that giving a nuclear weapon to every person in the world would not go well.

Democracy includes a broader array of governmental organization than just pure direct democracy. If you do believe that individuals should have some ability dictate the terms of their own social organization, then you believe in some amount of democratic principles.

Economic power eventually manifests in the political realm. The wealthy effectively get more votes, which means that society moves away from being democratic. Thus substantial wealth inequality is incompatible with democracy in the long run. We have been witnessing that corruption for a while now.

No system of governance can deal with immense concentration of power. The US Constitution was about separation of powers. Democracy is about (in theory at least) giving each person a meaningful say in their own governance, which in turn implies not allowing any single person to become too powerful.

Political leaders become a problem when they amass too much power. Corporations become a problem when they amass too much power. It doesn't matter what Sam and Dario's purported values are. They aspire to power and absolutely power always corrupts absolutely.

Technologies which are infinitely powerful or whose power grows too quickly outrun any reasonable attempt at regulation. If you imagine that tomorrow everyone were given a tank, we might think, "alright, everyone has a tank so it's not too bad." But humans are squishy, and our houses are (relatively) squishy compared to tanks. Substantial collateral damage would result from everyone having a tank, and it seems likely that substantial collateral damage will result from everyone having a cyberterrorism-capable slop machine.

Seems to lead us down the slippery slope of requiring an Apple device, or a Google device (e.g., https://cybernews.com/privacy/google-qr-code-recaptcha-requi...), or the device of some other entity (that may be mostly non-aligned with democratic values) in order to participate in society.

The unfortunate result of AI slop is reduced trust, which in turn is responded to with surveillance, which ultimately leads to the loss of liberty. Is it possible to do these sorts of verifications in an open way? I kinda doubt it, since someone has to control the hardware manufacturing process.

There's a solution: personal liability for the executives and managers at the company, and for the investors.

For example, every person who has ever worked for IDScan at any level of management should have all lifetime compensation clawed back and then pay a further 2x of that in fines. All VCs in the company should face personal liability up to 10% of their net worth. (Fines should be based on net worth; see e.g., https://www.nytimes.com/2018/03/15/opinion/flat-fines-wealth...)

> There's no particular reason to expect any particular formal system to be complete, sans some demonstration that is.

Well, Godel's First Incompleteness Thm says that any consistent, effectively axiomatized theory that is strong enough to represent basic arithmetic must be incomplete. So there cannot exist a demonstration of completeness for any such system. ZFC is such a system/theory.

(And yes, of course some formal theories are complete and can be demonstrated to be complete, like Presburger arithmetic).

> Gödelian incompleteness is the specific kind established by Gödel's proof, where theory T can't prove Con(T) without being inconsistent....

That's the Second Incompleteness Thm.

But CH is an example of a (an important; or believed to be important) mathematical statement that cannot be proven / refuted from within ZFC. So it is an example of a statement to which the First Incompleteness Thm applies.

You're right that Incompletness #1 does not prove CH is undecidable; (assuming ZFC is consistent) it proves that undecidable sentences must exist. CH is one of those sentences.

>...is a similar phenomenon as that the group axioms neither prove nor disprove commutativity

I haven't done a course in abstract algebra (though I have studied Incompleteness), but the brief research I just did suggests that there is a meaningful difference in ZFC and the ordinary group axioms. The latter are not sufficiently axiomatized to represent arithmetic, so Godel's thms don't apply. ZFC is sufficiently axiomatized to represent arithmetic (and much more).

The ordinary group axioms are so weak that they can describe many different structures; they also cannot represent arithmetic. So the ordinary group axioms are incomplete, but they are not Godel Incomplete.

One funny thing is that in order to tune the models to make what they're doing explainable to humans, you need to have humans involved in the RL pipeline to indicate which explanations are good.

You can understand this as learning a mapping between the model's internal "world" (i.e., 'meaning,' which is hopefully coherent and consistent -- but definitely not always! see, e.g., https://arxiv.org/html/2505.11581v1) and language (i.e. 'form') that reflects that world.

For this to work, you need both coherent / consistent internal model worlds, and also good mappings onto human language. Supervision by mathematicians has provided the signal for both internal coherence (though this can also come from interacting with a proof oracle) and for good explanations. If models exceed human capacities, you could imagine that aligning their explanations potentially becomes harder (though not necessarily). Also, humans naturally have to do the same thing: as researchers we must find analogies to make our work legible to collaborators or laypeople. Often in doing this, we further clarify our own understanding!

More deeply I think the "end of the world" vibe arises not only from the practical need to have models that explain, but also Litt's (and many other fields' researchers) grappling with being relegating to not mattering.

Unlike Europe, in the US, most PhD applicants do not have a masters. 10 years ago, most had not even conducted substantial senior projects (e.g a semester or year). Though increasingly senior projects / undergraduate "theses" have become more common.

That's right. You could list all those propositions and search for proofs of them. In fact this is very similar to Hilbert's very program to which Godel's First Incompleteness Thm was a response (https://en.wikipedia.org/wiki/Hilbert%27s_program).

Godel showed that there are true statements that cannot be proven, and also that among the unprovable statements from within the system is the consistency of the system itself.

As I understand, mathematicians are still trying to figure out how much this matters. One of the best examples of its mattering is may be the Continuum Hypothesis: CH is consistent with ZFC, and ~CH (not CH) is also consistent with ZFC. In other words, you have enough flexibility in constructing your ZFC world such that in some ZFC-consistent worlds CH is true, and in others CH is false.

Litt's statement is not wrong; it's just that what he wrote sounds so much like Hilbert's program, that I'm surprised we didn't get some even minor comment on what kinds of truths we could reach if we embarked on such an effort.

"A computer or monkey could easily start at the axioms of ZFC and iteratively apply deduction rules....simply conjecture all mathematical propositions in alphabetical order...The prospect of automating mathematics by enumerating all conjectures, and all proofs of ZFC, is probably not so disturbing to you."

I thought we were going to get at least some brief comment on Godel here?

Why is this being published as a blog post and not as a peer-reviewed submission? If it's going to be a blog post, why isn't there a corresponding scientific version for me to look at?

Someone else already found it. I don't understand why the link isn't in the blog post. https://arxiv.org/abs/2606.11045

Use of claude for writing it should be disclosed.

For every benefit that sillycon valley has produced in the recent past, there have been many more harms. I am confident that this will be no different. Of course, benefits and harms depend on one's vantage point.

In the short term, I think AI may do something like this, but in the longer term, I fear that its capacity for propaganda and surveillance will produce something even worse than social media.

The posted paper does use supervision to find alignments. I need to do a little more math to figure out how well the supervision critique applies, but I think it does. And although the current paper mentions DAS, it fails to discuss recent critiques of supervised abstraction methods (the various citations, including my own, that I provided).

A big problem with some of these supervised* interpretability approaches is that they can find spurious structure. (There are lots of ways to make the model do what you want; which is roughly what Hewitt and Liang 2019 showed). This paper draws a contrast to a previous method, DAS (distributed alignment search) on page 20. These and related methods rest on theories of causal abstraction, which are great in theory, but harder in practice. DAS, for example, has faced numerous recent criticisms (Makelov 2024, Meloux 2025, Sutter 2025, Grant 2026, Kumon 2026). My favorite is the quite approachable Meloux et al.; Sutter 2025 is also really good, but relies on a sort of real number argument that allows a lossless encoding of every input.

My forthcoming paper at EMNLP offers an alternative that instead grounds the notion of representation in a very simple notion of the effect it has on model learning/behavior when you adversarially perturb it. For example, if I tell a model that in the context "I saw a duck quacking" it should replace 'duck' with 'glam', how much does it desire to replace 'duck' with 'glam' in "I need to duck out of the meeting" vs. "At the park a duck protected her ducklings." This method turns out to work quite well, and as we use only a single example, avoids the need for supervision.

The linked paper argues that their method, DISCOVER, is not supervised in the same way as DAS, since it does not directly optimize for causal effect. I have only skimmed this, but I am not so sure it might not suffer from a similar issue. They're still supervising to align representations with their underlying hypothesis, even if they don't directly supervise for causal outcomes.

Refs

- Hewitt and Liang 2019. Designing and interpreting probes with control tasks

- Kumon and Yanaka, 2026. Fine-grained analysis of shared syntactic mechanisms

- Meloux et al., 2025. Everything everywhere all at once

- Rozner and Shain 2026. Perturbation: A simple and efficient adversarial tracer for representation learning in LMs. https://arxiv.org/abs/2603.23821

- Sutter et al. 2025. The nonlinear representation dilemma