Is there still a market for physical media? I assume they're transitioning to download-only because physical media are expensive to produce and most customers prefer downloads anyway.
You're confusing the weights of a model and internal chain-of-thought with the output of the model. Yes, we don't know a lot about how the internal mechanisms work. But with the correct prompt, agents will produce a worklog that documents exactly what solutions were tried and how the result was obtained.
Obviously, I'm not talking about the verbatim quotes I provided. There has to be some level of evidence that proves intent to commit a crime and the second quote is meant to represent that whole class of statements, but it depends on context. Any given quote won't constitute evidence in every case, but it will in the cases where it proves intent beyond a reasonable doubt.
I'm not sure why you think my argument holds no water when there are clear legal precedents that speech is not protected in some cases where there is "imminent lawless action".
I agree up to a certain point, but there has to be some legal boundary between freedom of thought/speech and literally planning a crime. I'm protected under the First Amendment to say "someday I'll rob a bank" but not necessarily "I'll rob this bank on Friday and here's how I plan to do it".
Right now, I would be willing to pay ~$20 extra per month for strong privacy, assuming the same capabilities as Opus 5.5. I don't see local models making sense economically any time soon unless you value privacy at $1000's per month or are fine with much lower performance on hard tasks.
You're completely right. I've internalized this on the technical side, but I still find it puzzling politically. Surveillance has been a salient issue in U.S. politics for almost 100 years now, since the invention of the telephone. I would like to think that most people are at least vaguely aware of Watergate, the Patriot Act, and Snowden. Then again, I should probably stop assuming that people know basic history, given the current state of education.
>I guess we probably want our tech to work exactly like this.
Do we, though? What if someone started an AI company that uses end-to-end encryption to make it impossible for anyone but you to access your data? Personally, I would switch to it in a heartbeat assuming it's competitive with the other products. I don't think it's the tech companies' job to surveil the population and prevent crimes. That said, I'm not necessarily against Anthropic or other companies reporting suspicious activity if their existing systems are detecting it. I'm just not sure we want every product to be forced into that data model.
Your follow-up about purely private tech seems to contradict your first statement. We can either have privacy or surveillance, not both.
I'm still puzzled that people's default assumption isn't that somebody is reading all of their internet communications. Ever since the Snowden revelations of 2013 I've more or less assumed that everything I type into a computer is stored in a government database somewhere. Not that I actually believe it is 100% of the time; there's a spectrum of trust, so I'm more confident that local apps on my Linux desktop are secure, somewhat confident in the end-to-end encryption of certain apps on my iPhone, but all bets are off for non-E2E encrypted data going across the internet.
I'm not sure what you mean by "correlation" in this context. The things I mentioned are strong evidence of causation. CO2 in the atmosphere causes warming by well-understood physical proceses, and human activities account for almost all of the warming above pre-industrial levels. In fact, the net amount of warming undercounts how much we have caused because we have also released aerosols into the atmosphere which cause cooling.
As I said, have a conversation with a top AI model and try to convince it that there is no evidence of causation between CO2 and climate change. AI is better than me or any other human at pushing back and presenting the right arguments and evidence for a specific line of questioning.
You can always have a conversation with AI about it. Relevant topics: physics of the enhanced greenhouse effect, IPCC methodology on attributing global warming to human activities, transient climate response to cumulative CO2 emissions.
I agree with your first point but not the analogy. Climate change is causally related to carbon emissions; if we had 100% carbon-free energy, there would be no concern with waste beyond inefficiency. Data centers can increase electricity/water prices, but it's not a linear relationship like climate change. Prices can stay constant or even go down if we increase supply by building more infrastructure, or decrease demand by limiting how much non-residential buildings can pull from the network. But yes, the conversation about utilities is ultimately more of a proxy for political frustration. If data centers became 10x more efficient, the same people would find something new to complain about because they're fundamentally against the technology itself.
I'm not against local governments setting zoning laws and regulations if they're concerned about the pace of growth. Personally, I'm not concerned because I think we can and should invest in infrastructure to keep up with demand. This will benefit all industries, not just AI. My point is that I suspect (I'll admit that I haven't done any research on this) that people wouldn't be against facilities with the exact same footprint if they didn't carry the label "data center" and this is primarily driven by a backlash against AI rather than the resource use itself. In that sense, they're trying to rebrand a political/culture war issue as a zoning or utility issue.
>Get out of here. Either you have a simple mind or you are commenting in bad faith.
I find it more and more common that people who disagree online respond by attacking the commenter personally instead of addressing their arguments. Saying "it's bigger" isn't enough; you need to explain why a building full of computers is fine if it's 100 acres but not if it's 1000 acres. You also need to explain why we shouldn't oppose comparable scale manufacturing, research, or warehousing facilities.
Cameras are different than data centers. Trying to connect the two is a rhetorical technique, not an argument.
The concern around data centers looks a lot like a moral panic. They've been around for decades and nobody cared before the backlash against AI. They're like any other industrial building, subject to the same resource constraints and zoning laws. I suspect if the owners of that land built a steel mill, a semiconductor fab, or anything other than a data center with similar power and water requirements, nobody would care.
Obviously the scale of retrieval from memory is different, I'm not disputing that. The empirical question I was referring to is what percent of its training set the model is capable of reproducing. What looks like "large swaths" of text to us could be less than one in a million for all we know.
I'm skeptical that there is a single spectrum like you're describing. It's not well defined. Say a human and an LLM prove a new theorem independently (without external help, i.e. from their own neural weights and reasoning). How do we measure how much each of them copied from previous work, as opposed to having learned from or been influenced by it?
Humans are also capable of memorizing long texts and sequences of numbers. It's more a question of scale than a fundamental difference. The point is that LLMs are like human brains in the sense that they're not databases that store text, even though they can and do memorize things.
Just because a model can reproduce parts of its training set doesn't mean that's what it's doing when it solves a programming problem. It can also reason about the problem, draw on its knowledge of algorithms and data structures, write tests targeting APIs it's never seen before, generate synthetic data and run experiments, etc. etc. Also, the claim that it can reproduce large swaths of its training data verbatim is an empirical one. I would be surprised if it could even reproduce 0.1% of the books it's ingested, for example.
Saying that AI can't do anything but copy or steal from humans seems to be a rhetorical technique used by people who are still unaware or in denial about the capabilities of the agentic systems released in the past few months. They can now one-shot theorems and programming problems in a few minutes that would be difficult and time-intensive for even the 99.9th percentile human expert.
I don't agree that AI tools will have the capability to escape any sandbox. It's a question of convenience and engineering effort. For example, you could design systems that run under a formally verified hypervisor in a physically secure airgapped network. Companies aren't doing that because there aren't currently any incentives to do it.
It didn't copy any source code from any external projects. I had it write a stratified sampling renderer for ground truth, then had it implement feature by feature by matching the pixels. Unless you mean it "copied" it in the sense of third-party code being part of the training data. I don't think that definition of "copy" makes any sense given how these models represent embeddings. It would also imply that humans are "copying" the things they've learned from.
I've been a vibe-coding skeptic for years, but because of the math breakthroughs of the past few weeks I decided to experiment with the latest models on some test projects. They're a lot more capable than I thought they would be. I agree that it's easy to create an unrecoverable mess, especially when you're one-shotting a lot of features without detailed instructions. But I find that as long as I'm strict about the API boundaries and force the agent to work in small chunks, it's pretty effective. As one example, I got it to write an SVG renderer in a few hours (not the whole spec, but most of the path features and text rendering), which would have taken me at least a week just for the coding part, plus extra time to learn the algorithms.
I provided an argument: mathematics is a discipline where we invent axioms and try to find interesting consequences of those axioms. By its nature, there are an infinite number of possible problems. Even if there is a finite number of problems interesting to humans (which could itself be questioned), we don't have any reason to think that we are anywhere close to running out of them. Saying there is "no evidence" for my view is a non sequitur; the burden of proof would be to establish that there's a finite number of interesting problems and that we've, say, exhausted 80% of them.
I don't think I'm the one adding noise here. You've done nothing but put words in my mouth, attack me personally, and use argument from authority instead of responding to the substance of the debate.
I dunno, it's a weird take on a website dedicated to casual discussion of technical subjects to say that I shouldn't chime in because I'm not an expert. I stated an opinion, and I don't see anywhere that I presented myself as some sort of authority. Also, it's beside the point, but for what it's worth I have taken upper level undergraduate courses in pure mathematics (real analysis and two semesters of combinatorics), so I have at least some idea of what research-level mathematics looks like.
Because that's an appeal to authority, which is a fallacy. Things aren't true or false because one expert says they are, they're true or false on the basis of arguments and evidence.
There's no lack of humility here. I'm stating an opinion and giving my reasons for why I think it's true, while acknowledging my limited perspective. If I heard a convincing argument, or saw data that 90% of research mathematicians think we're at risk of running out of problems, then I would update my beliefs accordingly.
I'm not a mathematician either, but I find it hard to believe we could run out of problems in pure mathematics. Pure math is like a game where if you get bored you can invent new rules. Maybe eventually all fields related to applied problems will be settled, and the value of pure mathematics will be relegated to the status of other games like chess and go. But that's so far away from current reality that I can't imagine it happening in the next century at least.
Frontier labs aren't under any obligation to release every model they develop. Either way, I don't see much evidence that they're withholding them in perpetuity. Open models aren't far behind so there's no incentive for that.
Difficult math problems are useful benchmarks and milestones. It's well worth throwing money at solving a problem if it drives competition and improves models significantly. The benefits become available to everyone, including the thousands of professional mathematicians who can use them to become more productive.
This. I've recently (re-)discovered the importance of using writing to force myself to confront my assumptions and gaps in knowledge. It's easy to fool yourself into thinking that you understand something if you haven't made it concrete and explicit by exercising your own brain. Arguably, this extends beyond writing in the conventional sense to many cognitive domains: mathematics, programming, physics, engineering, etc. AI should help the process of thinking, not replace it.
George Orwell's essays. I love his simple, matter-of-fact prose style and I'm trying to absorb as much of it as possible to improve my own writing. Many of them are available online if you don't want a full book. "Politics and the English Language" is a must-read.
I just asked ChatGPT to multiply two 4-digit numbers, and two 7-digit numbers without external help. It got both right. I'm sure it wouldn't have a 100% success rate, but saying it can't do arithmetic is just false.
I wouldn't say that information is an upper bound on knowledge because we don't measure knowledge in bits. The number of possible sequences of N bits is 2^N and knowledge involves selecting the sequences that are useful in some way. I don't know how to quantify it, but in principle it could be much larger than N.
I don't think I agree with your characterization of the second definition. Time scales matter. It's not much use to be able to solve human-scale problems if it takes millennia. And it only takes months to train an LLM to the level that it can solve cutting-edge math problems.