> You’ll be able to read it for free in any dentist’s waiting room...
Methinks that this says quite a lot about your socioeconomic situation. I've not seen The Atlantic in a dentist's waiting room.
4,331 karma · joined July 31, 2010
Methinks that this says quite a lot about your socioeconomic situation. I've not seen The Atlantic in a dentist's waiting room.
Ten bucks is pretty close to "I don't even need to think about it" money. Ninety is -for most folks- nowhere near that.
On the one hand, yes, they're companies like any other.
On the other hand, I can count on one hand the number of companies that have publicly declared «We're working on WMDs [0], we don't think we are capable of working on them safely, and we refuse to stop working on them. However, if we get special legal and regulatory treatment we'll be quite happy to put in the stop work order.».
So, yeah, there are some special things about the major LLM manufacturers and none of them are good.
[0] Anything that has a 10% chance of suddenly destroying all of humanity is a WMD.
I guess you didn't bother to watch the video that is TFA. Greg K-H mentions that if you're going to use LLMs to do bug-finding, you should use open-weights models that run locally and "harnesses" built by members of the community You really should watch the video to hear his reasons for why.
There's nothing wrong with LLM-the-technology. There's everything wrong with the major LLM manufacturers.
That sounds like a great story.
> Do not discount the many benefits, which might be difficult to quantify!
Yep! Anyone who thinks that a public transit company should make a monetary profit from rider fares is completely wrong. Public transit and public roads exist to facilitate happy lives as well as economic activity. If those things aren't included in your "are we making a profit?" analysis, then your analysis is just fucking wrong.
Every now and again, folks talk about making San Francisco's city transit system (MUNI) free of charge. The most frequent argument against that that I've heard is "but then the homeless will use the vehicles as mobile homeless shelters!"... which, like, there are ways to fix that, if you think it's a thing that needs fixing. As far as I know, MUNI has always required subsidies to operate, which -IMO- is how public transit should be run.
It's just that Nvidia doesn't care much about Linux, and -as always- Nvidia ignores what everyone else is doing and does their own thing. Sometimes doing their own thing works out really well in the short run, but -long term- they always fall behind.
No, you do.
1) Refuse to spread the "Well, okay, the tools aren't good now, but imagine how good they could be in the future!" meme. And -because these tools are SAAS and the resources allocated to them can be adjusted at any time without warning- evaluate the tools soberly and dispassionately three to six months after release, and ignore 100% of what the manufacturer claims the tools do.
2) Remind people that the major LLM manufacturers have pretty much never not lied about the capabilities of the tools that they produce and sell. Remind folks that the rational thing to do is to ignore the claims of both the manufacturers and boosters and -given that the tools are deliberately designed to aggressively flatter [0] the operator- to evaluate one's interactions with these tools with a huge serving of skepticism.
I get that you feel like there's nothing you can do to change things. But -as I've repeatedly said- you can stop carrying water for these companies by repeating their propaganda.
Assuming that the regulatory-capture and retroactive-immunity [1] gambit the major LLM manufacturers are currently attempting fails, when the fucknormously huge bill for the tools and the fact that so many GPUs are sitting in storage become public knowledge, things will sort themselves out pretty quickly. I'm fairly certain that continued progression of the -IDK- ten or twenty+ lawsuits against the major LLM manufacturers for the crimes they've committed over the years will help push that process along.
[0] ...I think kids these days call this "glazing"...
[1] ...don't believe that Congress would retroactively make obviously illegal conduct legal thereby mooting all in-progress lawsuits seeking justice for that obviously illegal conduct? Go read up on the FISA Amendments Act of 2008.
I run games both new and old and one of my favorite guilty pleasures is looking at the Steam forum for a four, eight, or ten year old game that I've started playing because it has recently become popular again and reading the complaints from Windows users about how a driver update screwed up the game... whether because of glitchy or incorrect graphics or unavoidable crashes. Meanwhile, I'm cruising along on Proton with zero issues. :smug-face:
[0] ...that small handful includes those that go out of their way to be incompatible with Proton...
Unless that was your money being invested, and it was a substantial fraction of the total pool of money being invested, was there ever a time when "normal people" had a real say in where the money was being invested?
AFAIK, the only thing "normal people" can do is vote with their "feet" and pick a different prepackaged investment product, different investment company, or take their money and do the investment themselves.
Honestly, this comment of yours seems a non-sequitur and doesn't really address anything I said... you don't have the power to bend investment firms to your whim, but that doesn't mean that you need to -knowingly or not- carry water for the major LLM manufactures by perpetuating the "But think of how great the tools will be in the future!" meme. It has been years now, and everyone who has been paying attention can say with confidence that the LLM-based tools of the future are never that great... they're often not useless, but they're not worth the billions of dollars that have been and continue to be poured into their manufacture.
Related to your "We little people don't have any power anymore!" commentary, I note that TFA mentions that the kernel community has found that these LLM-based bug-finding tools have a false positive rate of ~50%. TFA goes on to mention that Coverity spent a huge number of years trying so hard to get people to buy its automated scanning software that had only a 20% false positive rate, and could not get enough people to buy the software.
Coverity went under because everyone hated how stupid and annoying the tooling was... at a 20% false positive rate. Once the hype machine starts slowing down, no one working at the coal face is going to buy a tool with a 50% false-positive rate. I personally very strongly believe that even if the tools had a 10% false positive rate, no one would pay the actual price that OpenAI and/or Anthropic would have to charge to recoup the research and manufacturing costs of a cutting-edge LLM-based bug-finding tool.
* «We've observed that these things have a 50% false positive rate. Coverity tried so hard but couldn't get people to buy their software, and it had a 20% false positive rate. No one is going to buy something with a 50% false positive rate.»
* «If you're going to use these tools, run them locally. The open-weights models that you can run locally are quite good enough. Anything you upload to the SAAS ones will be shared with other people... we've seen so many examples of it happening.»
In regards to the first point, I think he's failing to consider the fact that you can get most upper management to buy anything by providing them enough food, drugs, sex, and/or fear... but -otherwise-, yeah.
Do you mention this to disagree with the claim that the LLMs have been built to please their operator? If you do, I see no conflict between the claim that an LLM has been designed to please its operator and the claim that operators of LLMs tend to be absolute dogshit at considering statements from humans that conflict with claims made by that operator's LLM.
To rephrase my previous paragraph: It seems likely to me that an operator that has their ego repeatedly stroked by the output of the LLM they're using [0] will react very defensively when a human tells them things that disagree with the output of that LLM. "How dare you disagree with this thing that seems very human to me and consistently tells me things that I like!? Don't you understand how much I trust it because of how pleased it has made me?", yanno?
[0] ...thus, being very pleased by said output...
[0] ...which is apparently somehow involved in LLM manufacturing...
Much like the mitigation of Morris or Slammer. Self-replication is -in fact- an essential part of what makes a program a worm, the first of which was built and released in the early 1970s.
I...
Look. Mythos was hyped up as the absolute best bug hunting tool ever made... no software was safe from its awesome bug-finding and exploit-writing capabilities. So strong was it that access _had_ to be limited to a select few pre-vetted entities, lest these awesome capabilities fall into the hands of Evildoers(!!!). Mythos' claimed capabilities were absolutely an important part of the "The LLM-based tools we're building are so dangerous that we must have new laws made to regulate us, or else all of humanity is likely to die!" story that the major LLM manufacturers have been building for a while and are telling now.
Now? Not even six months after release? "Well, yeah, okay, it's actually not that great. But imagine how great the next one could be!"... which is the story I've been hearing roughly every six months for what feels like five years now.
As an aside: I often wish we lived in a world where it was illegal for companies to use hype or any other types of emotional manipulation when advertising (or otherwise speaking in an official capacity) about tools that are to be used in a professional setting. Is it anything other than a bare statement of verifiable facts? Big fines, and repeat offenders get jail time. I know it's never going to happen, but it sure would be nice.
Then why did they remove the option that has them "work[ing] at taking your fare" once per month and leave in the one that has them doing that four times per month?
Hell, one can pay cash on the bus itself for a ticket to ride the bus. If cash-handling were the motivation, that'd be the first to go... the author of TFA mentions "First's weekly bus pass" and that they're in Scotland. This leads me to a company called "FirstGroup", which operates "First Bus" and "First Train". First Bus has this to say [0][1] about acquiring tickets on a bus:
With a wide range available, including weekly and monthly tickets, there’s no need to have the right change as you can make your purchase in advance. ... Contactless payments, Apple pay, Google pay and cash are accepted on our buses.
[0] <https://www.firstbus.co.uk/buy-tickets/buy-ticket>
[1] You might note that First Bus says it operates in England and Ireland but makes no mention of Scotland. IDK why that is, but this Scottish tourism website [2] links straight to First Bus's website to answer the question "Where can I find accessible bus and coach travel in Scotland?", so I believe the omission of a mention of Scotland to be an oversight.
[2] <https://www.visitscotland.com/travel-planning/bus-coach>
Really? From TFA:
When comparing the prices of First’s weekly bus pass in my area, which is available to purchase on the bus, and its monthly pass, which is exclusive to its app...
This may be true, but State and Federal law prevail. In California, this [0] looks to be the relevant regulation:
1748.1. (a) No retailer in any sales, service, or lease transaction with a consumer may impose a surcharge on a cardholder who elects to use a credit card in lieu of payment by cash, check, or similar means. A retailer may, however, offer discounts for the purpose of inducing payment by cash, check, or other means not involving the use of a credit card, provided that the discount is offered to all prospective buyers.
[0] <https://law.justia.com/codes/california/code-civ/division-3/...> The official place to find this would probably be [1] , but that's super down for me at the moment, so I link to something unofficial.
[1] <https://leginfo.legislature.ca.gov/faces/codes_displaySectio...>
Based on some of my personal experience and on informal interviews with folks I've known, the floor for that "some level" is so low that I'm not sure why you're bringing it up.
> UBI removes that.
To twist this a bit, one could say similar things about the move from community aid to aid you're legally entitled to receive -whether that's because you paid for it, or because it's welfare-. Making everyone dependent on community aid means that receiving aid is contingent on pleasing those who control access to that aid as well as those who directly provide that aid. Society might be -superficially- more polite if those who are consistently rude or angry, or were frequently too distressed or in pain to summon the energy to be polite were refused aid, but -IMO- we'd be way worse off in that society than one in which everyone is entitled to aid, regardless of their attitude. [0]
[0] Note carefully that I'm following your lead and specifically talking about attitude and sociability. "Gotchas" such as "Well, what if someone kills every aid giver who comes to give them aid. What then?" are out of scope... but the answer to that one is "Well, you imprison that someone for life and ensure that they can't harm any aid giver that comes to give them aid while they're in prison.".
And because this mechanism would enable guardians to set up reasonably-effective automated censorship for the vulnerable humans in their care without requiring that they provide any information about those humans to any third parties, neither big "social media" companies nor the politicians into whose ears they whisper want anything to do with it.
We won't be able to keep up with the pace of changes Google makes to the "living standard". That was one of the intended side effects of making web browser behavior into a "living standard".
When your work process depends on SAAS you depend on the vendor of that SAAS.
"Everyone" tells me that "noone" wants to use LLM-based code generators that you can run locally because they're so much worse than the newest ones provided by the major LLM manufacturers. If the gossip is true, then it seems like -for the foreseeable future- you're absolutely dependent on one or both of those two SAAS vendors.
I wonder if -should we ever be so lucky as to get a complete account- we'll learn that something vaguely similar happened with the OpenAI/Anthropic situation... not that one is financially dependent on the other, but that the fact that there are two companies doing the same thing instead of one gives the appearance of competition, which tends to ward off antitrust regulators.
Spamhaus is nearly thirty years old and the notion of electronic distribution of IP and domain "reputation" lists is at least that old.
I'll bet my hat that the Internet "advertising" [0] industry has been calculating and determining the reputation of individual households (if not individual users) for at least a decade.
[0] The scare quotes are because its primary purpose these days is for dragnet private-sector surveillance.
You and I couldn't disagree more.
The major LLM manufacturers are begging for new laws and regulations so that they get a huge hand in writing them. Regulatory capture is absolutely their goal. Given that they claim to believe that they're working on WMDs [0] that they cannot adequately control, they'd just stop work if safety was their goal. Their collective cries for regulation demonstrate that they'll happily coordinate with each other if they think the issue is important enough to do so. I guess "preventing the extinction of the human race by way of weapons we built and let slip from our hands" isn't sufficiently important.
[0] See the second paragraph and associated footnote here for a justification for the use of this term: <https://news.ycombinator.com/item?id=49839682>
In the "management" vs "line worker" split, they absolutely are. You appear to think that line workers cannot be highly skilled, which is absolutely not true.
Anyway, I'm super done here. Hopefully one day you'll learn to judge a company based on its actions [0], rather than what it claims about itself or how you feel about those of its employees that you've met.
[0] ...especially when considered in light of standard practice for companies in similar industries...
I'd be quite impressed if you could correctly hand-decode a five-meg JPEG in less than a workday. You do get that I'm not talking about having an understanding of the file format, but actually being able to convert the encoded data into human-readable [0] output?
> Your post is conflating lossy discarding of information with extracting abstract concepts from information and encoding them into weights. This is why those weights do absolutely nothing until you run a prompt through them.
I can play that game too. The JPEG process extracts perceptual shorthand from information and encodes that into "quantized coefficients". These "quantized coefficients" do absolutely nothing until you run them through a "reconstituter".
Most things sounds quite high tech when burdened with new jargon. It's something you inevitably learn if you work at a Big Software Company for long enough.
> Those [incorrect claims and assertions] you [receive] are not hallucinations, they are the [LLM] just filling in for missing details.
FTFY
> ...take a lossless image like a BMP and zoom in, you'll see those blocks again!
I take it you've never seen a highly-compressed JPEG?
> ... if you think lossy compression is sufficient to avoid the "verbatim" requirement of copyright claims...
Quite the opposite. It's why I even bother bringing up the fact that LLMs are the output of lossy data compression programs.
> I'm not sure what those links are in relation to?
Go back and re-read the paragraph that referred to them, and then consider it how it and the posts relate to the quote that sits right before it. Someone who suggests that they can quickly and accurately hand-decode a non-toy JPEG file definitely has the capacity to read and understand ten-ish Mastodon posts.
Here's a hint to prime your intuition pump: The linked posts are about plagiarism generated by LLM-based tools.
[0] ...in the case of picture data, "convert into human-readable output" means "turn the data back into a picture"...
In all my years, I've never known anyone who serves up breakfast OJ in four-ounce glasses. Twelve to sixteen is normal. I mean, shit, bro... you've seen a red Solo cup? The standard size for those is eighteen ounces.
> ...I assume at the time the notion of doubles, shots... [was] alien to you...
gestures back at this thread fork [0] you've probably not seen
I stand by my claim that diluted drinks like Screwdrivers are what you drink if you want to get drunk on the cheap [1] and straight liquor is what you drink if you want to get fucked up. Your continued reference to careful measurement does make sense if the only drinking glasses you've ever had around you were four ounces and smaller.
[0] <https://news.ycombinator.com/item?id=49890725>
[1] ...and if you're too strapped for cash to buy stuff like OJ, you cut it with tap water...
The major LLM manufacturers removing the safeties from their next-gen computer-attacking software, testing it with instructions to attack computers, and performing that test on an Internet-connected network is -at best- willful negligence.
The major LLM manufacturers getting together and declaring that they're working on WMDs [0], declaring that they are so scared that are incapable of safely working on said WMDs, and begging Congress to write new regulations so that they -somehow- become capable of safe work again looks quite a lot like anticonsumer collusion. It looks even worse when you consider that instead of begging for someone else to make them stop, they could all have chosen to stop... because -like- not only are they the major LLM manufacturers, they've locked in nearly all of the compute needed to work on this stuff. [1]
And I'll just include by reference all the dirt that the ongoing NYT case is digging up, and then gesture at the fact that software that happily executes attacker-controlled code cannot be made safe.
Nvidia is a shitbag of a company, but they provide hardware and hype. They're not the ones attacking other people's computers, claiming to be extremely serious about safety while releasing software that ignores the last fifty-ish years of computer security lessons, or -if the claims of the major LLM manufacturers are to be believed- threatening the entire human race with annihilation unless they get new regulations created just for them.
[0] ...it's fair to call something a 10% chance of killing all humanity a WMD...
[1] "What about China?", you might retort. What about China? The major LLM manufacturers go on and on about how the only reason China has made any notable progress in the field is because China is "distilling" the models they've trained. They think so little of China that they want to exclude it from the conversation about LLM manufacturer regulation. Seems like if you bring OpenAI's and Anthropic's models offline, China goes absolutely nowhere, no? Where is China they gonna get all the compute needed to build new cutting-edge models? It's pretty much all locked up in the US!