Should probably have a (2018) or (2024) (latest revision) on the title, especially given the current buzz surrounding AI and security/existential threats.
3,614 karma · joined March 3, 2015
They need AI to replace the expensive not cost-effective humans.
No one can actually tell you what ASI is or entails because it isn't a legitimate, operational concept. It is a fairy tale.
We can't even define "alignment". As people have finally started pointing out, humanity has never had a collective agreement on what values it should uphold or what ultimate goods are. Your alignment is not my alignment.
AI is not even autonomous. Every system we have today has to be initiated by a human actor. "AI" wouldn't create catastrophic bio weapons, it would help humans create them. The humans are the source of the intent.
Everyone has just completely given in to empty language and marketing nonsense. Honestly it seems like been the people at the labs are drinking their own kool aid and are themselves deeply confused about what they are even building at this point. It is a stateless statistics function running on a bunch of data centers. We aren't even close to an embodied, conscious synthetic being. It doesn't even have state, which is like prerequisite number one, nor is it plastic.
But even assuming that's the case, the natural thing to do in that scenario is to not contribute to the development of the doomsday device. But anthropic's approach seems to be to help build the supposed doomsday device faster?
The only generous interpretation is that Anthropic thinks that only they, the people at Anthropic have the morals/intelligence/whatever needed to build the doomsday device ethically, which is basically the perspective of an egoist despot and which should terrify you.
Clearly they aren't that scared.
If I hold a gun to your head and tell you to fork over your wallet, you will do it, because you sense imminent threat to life. Yet somehow we're to believe these people have the equivalent of a technological gun to their heads but are choosing to keep their wallets?
We've slid so backward, so quickly.
Consider the massive power and wealth differentials these companies stand to gain if they "win".
When the other participants have given up all reason and reasonable bounds on power, you'd do well to abandon any pretense of generous interpretation. Now is the time for critical analysis, not faith in the inherent "goodness" of someone whose entire job is presently to make as much money as possible by disrupting literally the entire white collar industry and who stands to gain much more than you or I do, and let's not forget, this outstanding success would never have happened without our work.
Keep in mind this is not a person who thought "wow, this is an existential threat, I'd better not contribute to building this". If anything Dario's past alarm ringing only proves that he's not really doing this in good faith. Otherwise he would have stopped a long time ago. The most generous interpretation of his behavior is that he literally thinks only anthropic is responsible and smart enough to build and control this stuff which...yeah, that should be pretty telling (esp. when you consider their recent security mishaps).
They've managed to dupe the dull eyed masses into thinking these products have some kind of agency of their own and can thus bypass the responsibility that should be falling on them to control their software. Amid all the marketing fluff and hype people seem to forget easily that ultimately these things are stateless functions running in a data center. We ought to be demanding these companies take culpability for their actions. Of course, the current political environment doesn't help.
The propositions: (1) humanity has always dealt with crises and (2) absurd wealth and inequity gives those crises particular shape, makes new crises acute, and creates crises unheard of and (3) absurd wealth creates small, uneven amounts of prosperity; can all be true.
This is why I think the formalists studying complexity back in the 50s had the right approach. You can only give "simple" precise meaning within some kind of formal system with a shared set of initial axioms or assumptions. From that point you can define it quantitatively over some set of objects (relations, programs).
Funnily enough, this approach also touches on Hickey's etymological derivation. The root words also fundamentally have to do with the quantity of relationships.
I think a lot of people believe that, as long as they consume "high quality" or more intellectual content, the negative effects of constant consumption are mitigated, but I think that's false. It's less about the content and more about the structure and process. If you never give your brain moments to stop basic processing of wildly different inputs and to just critically examine them instead, you will suffer brain rot, even if the content you consume is "educational".
I think because a program is textual, people incorrectly draw analogical conclusions for programs based on their experience with written works, in which the medium of expression and the artwork coincide. However, this is not the case for programs because many equivalent programs can produce the same end result (runtime user experience) and we do not typically incorporate the program itself in our judgement of the artistic merits of the runtime experience. In writing the form and content are inextricably linked to a much stronger extent.
Just as they produce writing that isn't structured well for humans, they do the same with code. Sure, if you're willing for that code to be write-only and for agents to be the sole interface through which you interact with it, that works fine, but it is not better than what a professional, skilled human could produce. It's often full of pointless crap and design decisions that over index heavily on local structure that make very little sense when global system structure is considered.
> A compiler for the interface your agent already has: the shell. MCP makes an agent carry every tool's schema on every turn; declick compiles an API, an MCP server, or a database once into named verbs the model loads one at a time, and every verb returns one envelope with five exit codes. A team pushes a compiled adapter to a shared folder or git checkout once and every other machine pulls it. Ten engines, zero runtime dependencies, Node 24.
What the hell would it even actually mean to have a "compiler for the shell"? How do you "compile" a database into a "named verb"? What would this mean? These notions don't meaningfully cohere together. Why exactly is an "envelope with five exit codes"? Are we getting all of the exit codes at once? That wouldn't make any sense. LLM writing is full of this kind of arbitrary collision of concepts that sound like they plausibly go together thanks to frequent co-occurrence but that have no coherent logical meaning when you apply even a modicum of critical scrutiny. This is why I find it astounding that anyone believes these things are "intelligent" and not still just fundamentally a matrix machine completing words.
LLM generated text always lacks these elements. It's always some bland, shapeless sequence of words pulled from the ether. I'm sure with time and effort you could combat this and give LLM text more of a human feeling by giving the LLM enough context about your communication, but that would take work, which in many cases would defeat the purpose of using the LLM instead of doing the writing yourself. And if you care that much about your communication with others (and you should!) you'd probably just find using LLMs frustrating to begin with.
Orwell is a master precisely because he understood language deeply enough to be able to produce a monstrosity on both the level of content (undue abstraction) and structure (winding, breathless syntax).
No it isn't. It trades concrete imagery for empty abstractions which could encompass just about anything and are basically meaningless.
If you think this is good writing, you need a serious detox from corporate America.
What these comments all miss is that ensuring your 13 million lines actually encode what you intend them to encode is still a major problem and yes, extremely difficult when you have that many lines to pore over. But, if you're using LLMs to vibe code millions of lines of "proof" you've already stopped caring about that and presumably given your critical reasoning and concern over to pure faith in machine gods anyway.
I'm not sure why everyone on hacker news so often falls for the "my friends are jumping off the bridge, so I can too" fallacy. Not to mention, there are clear differences between the processes under consideration, which might bother you, even if the outcome is the same. No, I don't care that publishers destroy books because usually they only do that when the book has proven to fail to sell (not the case here, clearly, since the books were sold!) and out of necessity because of resource constraints and not because they are using them to build a proprietary alternative product they plan on charging me for. I don't understand why people are engaging is such shoddy, shallow thinking here.
I think the difference in our viewpoints ultimately lies in our faith in the companies involved. I have little to no faith that the digital scans of these texts will make it out to the public based on historical behavior of companies and the current legal complications around copyright. As such, I find it grossly unfair for several companies, which have significantly more economic and logistical resources than interested individual buyers, to systematically purchase rare works and thereby prevent individuals from accessing those works, possibly in perpetuity. The point is that even if they were to eventually release these, they are effectively, as of now, as of my lifetime, no longer available. Again, we have little information on the specifics, but it's the principle of the matter, the gross asymmetry here, that bothers me. It's all worsened by the fact that they are doing this so that they can use a commodity which you could have paid for once to train a new utility, tokens, that they want to charge you for on a metered basis.
If I understand you, you seem to be more confident that they might eventually release the works, or you are less bothered by the power asymmetries here. I actually do hope you turn out to be correct, and they find a way to release the material, or at least tell us what they bought. However, I doubt this will happen.
I hope you have a good day too! Thanks for sharing your perspective on the matter.
You realize "market value" isn't the only kind of value too, right? Even books that are completely obsolete can have inherent historical value insofar as they enrich our understanding of the past and how human beings used to live. Contemporary market value isn't everything. There are plenty of works of scholarly interest that wouldn't command nearly requisite value to a NYT bestseller when you adjust for scale. That doesn't mean these books aren't worth keeping or aren't intellectually important.
Also, these books presumably even have some perceived value, otherwise these companies wouldn't think they were relevant for training, and wouldn't be spending money to purchase and scan them in the first place. The whole premise is that these books are valuable enough such that training on them will make the models valuable and make paying customers line up for tokens. This is an inherent imputation of value to these works on the part of these companies. You can't say, simultaneously "these books are worthless" yet somehow, at the same time, "they are worth something for producing artificial intelligence". Your assumptions don't make any sense.
> They're books that are no longer in print. Doesn't mean there aren't a lot of copies around.
Sure, but there also could be limited copies around. We don't know. We won't know unless these companies are more transparent about exactly what they are buying and scanning. It's that simple. If it's foolish to assume there might be limited copies, it's equally foolish to assume they have plentiful copies. We simply don't know.
> Nobody has an obligation to spend money to preserve every extant copy of every book forever.
I don't think anyone is claiming that. People are bothered because they (in my mind, rightly) recognize that old books are a historical record of human accomplishment and have some amount of cultural value. Letting a handful of companies further plunder the collective output of humanity and now, potentially keep it locked on their servers in perpetuity is something that should make you upset if you have any inkling of interest in the history of humanity or human accomplishment and if you have any sliver of curiosity about these things. If all you care about is the present and getting better LLMs, sure, I guess I understand why you don't care, but then I also think you are very unwise and myopic.
> Millions of books are destroyed everyday.
So what? Millions of crimes are committed every day, that doesn't mean we shouldn't be bothered by the crimes we hear about. Billions of pollutants are spewed into the environment every day. That doesn't mean we shouldn't try to combat pollution. That's a pathetic attempt at justification. Also, there are clear distinctions between processes like libraries removing works they can no longer store and companies scooping up books for training and personal gain.
No. It definitely isn't. There are plenty of US companies, one of them large and recognizable, that actually make it their mission to give you access to information in some fashion.
We have different definitions of "bad thing". In my opinion, unilaterally potentially erasing human history is already bad. Is it the worst thing someone could do? No, probably not, but that doesn't mean I shouldn't be bothered by it. I'm not sure what kind of argument you're really trying to make here.
I don't find LLMs that valuable, a few companies being the only providers of "frontier" models bothers me, and their possibly destroying works of historical interest also bothers me. Until they come out and find a way to make their scans available to the public, I'm not going to be happy about this. In my opinion, you shouldn't be happy about it either. These companies have everything to gain by making it harder for you to access knowledge (so that you are forced to pay them continuously for their "knowledge" in the form of information sparse LLM tokens). They have almost nothing to gain by enhancing your access to knowledge.
I don't think LLMs are going to change this. They won't change what companies and individuals value. Yes, they can enable those interested in performance to write even more performant software, more easily, but they won't convince the people who aren't prioritizing performance in the first place to prioritize performance. This ship sailed a long time ago.
Performance was prioritized when it was necessary due to resource limitations. As soon as those lifted, performance was deprioritized in favor of other properties. I doubt the balance will really be restored until limitations re-emerge or consumers put pressure on companies around performance. Unfortunately, the past few years have shown that the latter is unlikely to work.
Ultimately the most affordable worker of the future is going to be the person who knows just enough to prompt an agent to first build a feature and second fix it "fast enough" when it ultimately breaks.
Companies might have a couple of people who actually know how to design systems and think deeply on hand for overall architecture, load balancing design, and serious problems but this will be a small cadre of employees. In some sense, I suspect, in relation to degrees, computer science will eventually look like the field of curation: an extremely small pool of outstanding positions looking to be filled by an orders of magnitude greater number of degree holders.
Unless the bubble does eventually burst, in which case people will hopefully return to their senses.
One could obviously reproduce films too.
Ultimately, you don't know how big N is, and that's what matters. Part of the problem is there is no transparency around this. The population of all books != the population of a specific book or set of books. Assuming N is large when you have basically zero information on specifics is foolish. These books might be just as rare as a film for which only one source exists you don't know.
I don't know why you're so quick to assume this will all just "work out" such that the scans are ultimately accessible. Arguably the most likely scenarios are either Anthropic survives and holds them away in perpetuity or Anthropic fails and they are sold off to the highest bidder who does the same.