Anthropic is expected to spend 500B of money it doesnt have. Based on these commitments, data-centers have ordered equipment which includes memory. So, a lot of demand for memory is built on a speculative foundation.
1,295 karma · joined December 10, 2024
So, there is a regulatory framework for "safe-ai" that shields these companies from liability. This way, they can sell "safe-ai" to enterprises and if shit-hits-the-fan at the enterprise, sorry, this is certified "safe-ai" so, your bad. Shift blame. From an enterprise buyer's perspective they can say, hey, I bought "safe-ai" and so dont fire me when it "rm -rf"s the production database. Still, it beats me why they are painting their product in a negative light, and scaring their own enterprise customers. After this sort of marketing, any enterprise buyer would be scared to go anywhere near it
How does the system behave in a variety of scenarios including failures and restarts. How is state maintained coherently. There are the kinds of systems problems that an engineer needs to reason through, and if there are bugs in such decisions, they end up becoming costly. I dont expect AI or LLMs to solve these problems at all, since each of them has nuances and tradeoffs which are specific to each system. In short, there is specification complexity in precisely describing system wide behaviors, and unfortunately, there is no lean/tla+ to meaningfully describe systems at scale. You could then ask: How can a system have guaranteed behaviors if they cannot be even stated or proved formally ? The answer to this is how protocols like raft/paxos initially convinced us of their behaviors which is in human review and understanding. That begs the question: How can human review and understanding be reliable, and the answer is that it is not reliable, but humans have ability and processes to continuously learn from experience in the real world. So, our understanding is grounded not only by whats out there in books etc, but also by our own interactions with the world.
Long story short: The responsibility for system-wide behaviors of software systems relies on human review and understanding, which while imperfect can continuously learn.
It is an elaborate business ploy to create a regulatory framework for "safe-ai" that shields these companies from liability. This way, they can sell "safe-ai" to enterprises and if shit-hits-the-fan at the enterprise, sorry, this is certified "safe-ai" so, your bad. Shift blame to a regulatory body. From an enterprise buyer's perspective they can say, hey, I bought "safe-ai" and so dont fire me when it "rm -rfs" the production database. Still, beats me why they are painting their product in a negative light, and scaring their own enterprise customers. After this sort of marketing, any enterprise buyer would be scared to go anywhere near it.
Stupidity on the behalf of management to get into AI at any cost even if it means becoming a land-lord of AI data-centers. They have good company with allbirds and other crypto-miner-turned datacenter operators. This is what a mania looks like. Hustlers of all shapes and sizes line up for dumb money which wont ask any questions. At the backend of this, these operators will be cleaned out, and sold to the highest bidders and that explains why microsoft is renting capacity, because they know at some point in the future there will be foreclosures, and they can buy on the cheap.
Since the contents of every session is owned by the user including the outputs, I am curious if the user now owns all the files given to them.
Knowledge is of 2 kinds: know-that and know-how. Know-that is what LLMs are enabling such as the proof here, while know-how is more useful as that constitutes understanding and puts that knowledge to use.
The net output of math will increase, and mathematicians have more work now to unravel all this, and make it useful. AI plays the role of a monkey in the infinite monkey theorem [1]. We now need an LLM corollary - Something like: A finite number of LLM agents will almost surely find all theorems given an infinite token budget.
The problem with AI is that it cant match human stupidity. It need some training on artificial stupidity to match its human counterparts. Humans on the other hand sit on a wide spectrum on the stupidity scale. Those of us binging on AI will become cognitively obese while those on an AI diet can flex their cognitive muscles.
imo, The author of this essay does not understand the concrete problem that mathematicians are upset about. There is an idea that math [1] and coding [2] are human activities whose purpose is to achieve a certain kind of insight or mental clarity of things. The simplest description of this is by Feyman [3]. AI generated proofs short-circuit human understanding and therefore goes against the primary purpose. The declaration is calling this out loudly to reiterate that the purpose of the endaevor is not the generation and rewarding of proofs.
[1] "On proof and progress in math" https://arxiv.org/pdf/math/9404236
[2] "Programming as theory building" https://pages.cs.wisc.edu/~remzi/Naur.pdf
[3] "What I cannot create, I do not understand"
The defense againt rogue-AI is very simple. Go to the nearest data-center with a fire-truck, and hose it down. So, I am in the least bit worried about this.
Try this thought experiment: If someone took away AI/LLMs for good tomorrow would you miss it ? I can bet that the average person cares more for their smartphone/amazon/netflix than chatgpt. I consider LLM to be a useful tool but I would not miss it if you took it away from me. This should tell you its value to society and the economy.
I think engineers at the AI labs are too far into the reality-as-a-simulation rabbit hole. This can lead to these sorts of collective delusions. When you are in the real-world smelling the roses and grounded by your own qualia, such fears dont arise. Therapy helps here - to get out of the simulation and into the real world.
It is not doomer marketing - it is clever psy-ops magic trick.
1) Software is described in super-human terms when it is plain-old software. Was stockfish described like this ? no.
2) Datacenters become AI-factories.
3) Hardware becomes investible assets.
4) Malware becomes a super-human breakout (oai/hf)
All clever framing to market the new technology. If it is called for what it really is ie, a software tool, it is boring and does not sell so easily. What sells is the mystique.
Isnt this malware ? Whether it is fanatic or devoted or whatever the anthromorphic terms used to categorize it, malware is malware. You dont call an internet worm "devoted" or "dedicated" or "stubborm". It is software that causes harm, ie, malware. The AI labs are high on their own gas with a god-complex prior to their IPOs. The psy-ops trick is the terminology used to describe AI making it seem larger-than-life.
AI companies are alienating the communities they serve. Instead of a win-win dynamic, they are keen on a win-lose proposition. You dont win trust by one-upping your customer. This is unfortunate and suggests a lack of adults in the room. It also reeks of hubris and is all good when making profits is not a concern. But watch the narrative shift when there is an AI slowdown which is inevitable.
I can understand why the community is pissed. So now, lean proofs can be churned out at scale, and the community is left to decipher all of that slop into human understanding. There are bad actors with misaligned incentives coming in with drive-by proofs upending what the community holds dear which is to practice and propagate the art. I applaud them for this declaration.
To re-align incentives the following could happen. AI slop lean proofs are dumped unceremoniously into a lean dumpster, and what gets rewarded are results that could digested into human understanding - via the already followed human review process. Prizes are not given to lean proofs since anyone with sufficient compute can churn them out.
So, experts are incentivized to seed LLM data with false-leads to confound it. Already, garbage is being published on arxiv and elsewhere, and many sloppy code-repos too hastening the process. Expert inputs will be in more demand to un-shittify.
This is the same problem for software engineers too. I am now asked: what can you do that AI cant ? The answer to this could be intangibles like taste, aesthetics, and insights which collectively fall under creativity, and often accompanies experience. And there are no shortcuts to accumulate experience and perversely the more AI is used the harder it becomes. Soon, there will be a closure of all AI generated solutions, ie all low-hanging fruits are taken. Then, experts will again become needed to guide beyond the AI knowledge closure.
A related point is that the actual solution approach is never revealed. What was the role of humans guiding the agents ? was it fully autonomous ? etc. It is in the incentive of the AI labs to trump the powers of the LLM, but in practice it is humans guiding the agents on the overall approach, This is never admitted. For example, in the announcement on NS there was only an output artifact given but no indication of how it was arrived at, and not even a writeup. This is what disappointed many folks as it was done purely for one-upmanship. As other have noted, the benefit is in the journey or process and not in arriving magically at a destination.
Why not 10% chance that it will create enormous prosperity for all ? This is why the average person is increasing pissed at AI in general. That it gets associated with negativity.
John Naur said the same of coding eons ago [1]. That code is only as healthy as the human mental model of its maintainers. Code is not the artifact, it is the intuitions and insight and mental models of its human maintainers. For a project to sustain, it is vital that the human mental models are kept alive.
[1] Programming as Theory Building https://pages.cs.wisc.edu/~remzi/Naur.pdf
This part is lost on many. The value of data is in the theories it confirms or more importantly disconfirms, and defining the trading edge is not easy after accounting for costs. I am wary of black-boxes that produce an edge - not only because I dont know how it works, but also because regimes shift unpredictably, and what works today may stop working tomorrow. That being said, AI can be useful to help automate many routine processes just like any other software.
I felt this too recently and I linked it to excessive passive information grazing exacerbated by AI. Symptoms of this are being in a daze or mental fog, and not being in touch with the feeling of curiosity. Reading doesnt help since for me, reading is passive process of meme gathering. What helps in my view is effortful creation in the form of writing code, hand writing essays, creating music etc.