Anyway, you might have more luck just writing to it in your native language. It’ll be equally crummy, but maybe you’ll find it easier to decode.
Claude is very much the “stupid person’s idea of an intelligent person”[0] which, I suspect, is why it is so popular.
It certainly explains why half the internet is huge chunks of Claude-authored gibberish copied and pasted and published. If people didn’t think it sounded clever they wouldn’t put their name behind its ramblings - but very few of them seem to realise that a lot of people see straight through the bullshit and know instantly that they didn’t write it themselves.
But equally, a lot of people can’t tell, and read whatever it is and think “that person must be clever!” So you have people incapable of coherently expressing thoughts who are using Claude to write on their behalf, with the result that the people they want to think of them as clever think less of them and the people who can’t distinguish clever from AI slop think they are clever.
And the people who can’t tell don’t care, and the people copying and pasting Claude slop seemingly don’t care either.
And then I remember that more than half of the US populations reads at Grade 6 or lower[1], and nearly 1 in 5 people in England is functionally illiterate[2], and I simultaneously despair of - and am thankful for - the bubble of literacy I inhabit.
[0] https://quoteinvestigator.com/2018/01/05/clever/ [1] https://www.thenationalliteracyinstitute.com/2024-2025-liter... [2] https://literacytrust.org.uk/parents-and-families/adult-lite...
it makes me think about how people engage with movies and television - as passive, plot-and-character driven consumption (eg I hope Walter White survives) with no critical analysis of how and why the writers added ABC thematic element (eg Walter White as a motif of a toxically masculine narcissist with specialized knowledge as a larger critique how mass media tends to valorize their male leads in the same vein as many other prestige shows at the time like Mad Men), and the larger, downstream sociocultural impact that piece of media has on how people see the world (eg people who now have the Heisenberg tattoo, unironically)
there's been some musings on why this the case like Hofstadter's Anti-Intellectualism in American Life - the valorization of obedience and trust in hierarchy and the state are net wins if you're an institution that seeks to increase it's power, whether religious or governmental. I was talking about this with a few friends the other day and it's a dismal future reality where not only did we make anti-intellectualism normalized and politically legitimate in the USA (eg Fox News, clickbait articles, and all the other forms of yellow journalism that have emerged), we now have tools by which individuals can even further remove themselves from having to critically engage with thoughts, feelings. I heard a story about how someone scanned a group activity at a baby shower into ChatGPT and had it answer for them instead of, well, socially interacting with the other guests and forming a memory of the moment with their friends
the counterargument to that might be that Claude/ChatGPT/etc have more epistemic rigor than your average American (sure) but the sycophancy of modern day LLMs is an actual danger that enables more harm than good. it does seem as if Claude is the only one interested in guarding against some small amount of it (though to the detriment of people just trying to get work done. as an aside, I get the feeling Mythos was intended to be the bespoke enterprise solution without the guardrails but the Anthropic marketing department or some power-hungry department lead made it about how dangerous/effective it was from a security perspective which threw a wrench in things). but then I think about people like my parents asking ChatGPT which specific house to buy in their retirement only to later find out the house was sold weeks ago, or just in bad condition, or in a neighborhood where the housing value has already reached equilibrium, it makes me think about how it's not enough and the future is bleak
I'll also say that I think Claude sounds the way that it does because it, like many other LLMs, are RLHF trained largely by lowly paid gig-workers, many of them ESL speakers. if their trainers were, for example, dedicated and highly trained academics, scientists, and other researchers, you'd likely see a lot more concise and more importantly skeptical reasoning and responses. but that won't happen in our current reality of capitalist-driven development so we get encoded solutions like MoE that still largely depend on the messy, imprecise RLHF training at baseline
in the right hands, I do think AI is a wonderful tool. one of the first things I did with it was to create a research skill that reviews white papers from the lens of someone who knows how to read/interpret research methodology, is aware of things like p-hacking, and deterministically assigns weight according to the hierarchy of evidence. even still, I'll still read the studies because there's so often nuance that's missed if the sub-agent read only a search snippet but that takes effort, time, and the practiced knowledge of critical analysis to even want to do it
(So here’s a big wall of text of my own!)
However, a lot of what is written here makes sense.
And particularly “if your comprehension level stops [here] you get 'big words in complex sentence structure sounds smart and right so it is smart and right' even if the reasoning and process is poor”
This is exactly the problem.
And another point you make:
> but the sycophancy of modern day LLMs is an actual danger that enables more harm than good
I don’t think it is necessarily the sycophancy that is the biggest problem (though that is definitely a problem) but rather the combination of authoritative sounding text plus “complete answers” which sound wholly believable but are deeply flawed unless you have domain expertise.
I moderate a forum that deals with people who face a relatively common but somewhat complex (and nuanced) set of legal problems.
The purpose of the forum is peer support, shared experience (“lived experience”) and community.
It’s not legal advice, though moderators will sometimes step in to highlight relevant legal resources (e.g. case law/precedent or primary legislation/instruments).
Prior to AI infecting the forum someone would post their problem, people would respond with their often incomplete or poorly communicated thoughts, the OP would ask more questions - or argue - and a dialogue would occur. That created a community and people would post updates and ask more questions and find common shared experience. Many of them became correspondents with each other and some became actual friends.
In the past 12-18 months the discourse has changed from “here is my personal experience and here is what I did” to “here’s a bunch of stuff an AI says and I’m pretending it is me giving advice”.
Almost without exception the person who has started the thread will react positively to the AI generated content, even when it is egregiously incorrect - but won’t ask questions.
More problematically, these AI posters will often argue specific incontestable points of law “because I asked ChatGPT/Grok/Claude and it says this” and ChatGPT clearly cannot be wrong. And the border of precedence seems to be ChatGPT, Grok and then Claude some way behind.
I’m slowly seeing a pushback from people as “normies” begin to spot AI. But it’s ruined a community because the advice sounds so authoritative and complete that people won’t argue or ask questions.
As a result we have banned AI generated posts and remove repeat infringers.
That’s significantly reduced the volume of posting (below what it was pre-AI) but has significantly increased the value the members are getting.
it's the old tortoise vs hare parable, I think. go fast, make a bunch of mistakes, get too arrogant, and you lose out. your forum might be slightly lower engagement now while people are caught up in the latest fad but your rules are proactive for a future where average people hopefully realize that you can't trust an LLM that has zero context, no real harness and determinstic tests to speak of, and a propensity towards probabilistic rabbit holes that result in hallucinations. at least that's the kind of space I'd look for now and largely why I've given up on a lot of other forums
Which is basically weather the AI storm and come out the other side with something that is essentially purely human.
And then we might - where appropriate - use AI to help surface or explain relevant external content. “Idiots guides” but human reviewed.
I can’t help feeling like this is the last gasp of the old internet. Those tiny corners of expertise can so easily be eliminated through a few months of “AI! SHINY!” and there’s no coming back. I’ve seen a couple of other communities decimated by AI. The participants start posting AI slop and then remarkably quickly everyone else just stops commenting. It’s awful.
If you don't know better, you don't know better to question what the AI says.
I've seen this in the work environment with a coworker who insisted that I implement my side of the control system using the control law ChatGPT recommended instead of building off the empirically tuned control law. I eventually sectioned off a part of the codebase for him to work on independently.
Needless to say he didn't get a whole lot farther.
Later characterization of the entire system end-to-end showed the existing system was already close to the theoretical limits and ChatGPT's tearup would have bought us precisely nothing except for more work to tune the new control loop.
And I see this in everything that requires expertise. You need to know enough to know when it's bullshitting you, and it's hard to be enough of an expert in everything to tell when it's bullshitting you for something you aren't enough of an expert in.
it makes me wonder if the solution that businesses/users need to implement is just the same solution to everything since the beginning of time ie standardization. skills/harnesses/agents.md/etc maintained by codeowning teams that must be invoked for AI-assisted code changes on ABC part of the codebase, these existing as replacement for the bevy of other documentation required for the days of hand-written code. a company-wide orchestration skill knows how to search and pull down the relevant .mds, cleans it as cruft at the end of a session, every merge with a short changelog saved to a corpus somewhere with a TOC + appendix that an LLM can navigate to and read for context, major changes in the logic documented in the working skill doc, all of it generally automated but requiring HITL vetting
this wouldn't fully solve the problem of subject matter expertise but it seems like it would remove a lot of the friction for new employees and other teams with dependencies on your work or with whom you have dependencies
That's not the only smart-person way to read that show. And even if a character has flaws, or even if it's an outright villain, people can still like the character. If I tattoo Scar on me from the Lion King, does it mean I didn't understand that he's not a positive character? I can still think he's cool. I'm sure people also put Darth Vader tattoos on them. Also you're using phrases of political ideology that one doesn't have to subscribe to in order to enjoy the series.
everyone's free to enjoy media however they want, with whatever level of interpretation they like. I provided the BB references as short examples, they aren't meant to represent the definitive diagetic experience of the show. if you have a different view, great. if it was triggering for you to hear 'toxically masculine', also fine but... might be something worth self-examination on given that it's very much also a clinical and academic term [0] as much as it is one steeped in the artificially manufactured culture wars by people who don't want to change their anti-social behaviors
I would also say that understanding Scar as a villain is the sixth grade reading level understanding of the character. and you're free to stay at that level of understanding. someone who wants to engage more critically might map the character to Claudius, comparing and contrasting how they're characterized given the context of the audience for Disney and Shakespeare, and appreciate the character that way, as a standardized trope utilized throughout all other forms of media. they may even get a tattoo of Scar, symbolizing their dive into the analysis
my point is not that people should or shouldn't engage critically. it's that this practice of critical engagement, of being skeptical and analytical provides you with the skills to not be a total sucker who falls for the latest manufactured fad that someone with a strong theoretical understanding of semiotics and social capital created (ie most modern marketers). the pertinent example being how people engage with AI - seeing it either as a specialized tool with a set of flaws that need to be accounted for and checked against or as just some kind of authoritative voice because it sounds smart and so-called smart people like Elon are terrified of it and AGI. which, again, if you prefer the latter engagement all power to you but the chances of you taking some really bad advice forward is not negligible
[0] https://www.wi.edu/news-Shifting-the-Conversation-From-Toxic...
It's like a jumbled up string. You pull on the two ends and it turns out to be just a loop, it resolves to a big null.
I'd personally trust a rigorous, well-evinced, and methodical dissection of a topic by someone with completely average intelligence over a lazy broad generalization by someone who scored high on a paper test that people (wrongly) assume maps onto the ability to assess and analyze the world [0]. which isn't to say there isn't any clinical significance from something like the Stanford-Binet, it's just that there's a wide gulf between the lay person's understanding of 'high IQ' and clinical applications [1]. and, accordingly, I am someone who did score high enough on a psychologist administered IQ test when I was but a wee lad that universities were sending my family letters requesting that I be a part of their high-IQ child studies but you're convinced I'm middle of the curve so how valid is it, even?
the thing I value from academia is predominantly peer review and constant critique by people whose entire job is to perform epistemically rigorous analysis. this is something that's very uncommon when it comes to culture warriors on social media. and you must be very far removed from any kind of academia if you think they all universally agree - put a post-structuralist in a room with someone who still favors deconstructionism and you won't hear the end of it
funnily enough, clinical psychology is the research I linked to about toxic masculinity that you broadly generalized as worthless ivory tower ideology. so it wouldn't make sense then to value something like 'smarts' or 'IQ' since those, too, are concepts that emerged from the very same ivory towers, in those exact departments rife with ideological bias
[0] https://som.yale.edu/news/2009/11/why-high-iq-doesnt-mean-yo...
[1] https://opened.cuny.edu/courseware/lesson/48/student/?sectio...
Yes, I threw in a smiley just to be a dick. You even quote LeGuin, thank you for being you.
Very confused about this: how would you write the character differently?
The whole point of leads, whether male or female, is to be larger than life. All their characteristics, including growth or change, are exaggerations of reality.
This specific character is supposed to be more villainous than reality, and if you take that away then you're simply going to have a show no one will watch.
No really, that's not particularly accurate, they use so much gig work because no-one else wants to work for them not because they would be unwilling to pay a little extra, or only want the absolute cheapest labor they can get on the planet.
They want senior white collar professionals and scientists and researchers especially since these companies already on some level believe their models are as good as any senior employee in any field (it's probably the models generating text saying that, but that's besides the point). But who's going to work on contract for a company that wants to automate them out of a job? Realistically no-one unless they get some shares in the thing that will destroy their future earnings potential and ability to control their own destiny if it works out.
But they can find enough educated white collar professionals on unemployment or in unstable academic employment that will take an extra job on even if it's only 50 $/h or 70 $/h and compromise on any solitary they might have but the work output you get from that is only going to be as good as what you ask for, if they had better respect for the professions they want to automate, it would be better.
Like is that an acceptable wage in the US for difficult skilled work, not particularly but it's not rock bottom exactly, and it's not bad for other English speaking countries, working conditions and stated mission are more of an issue than being cheap.
Training pipeline on a modern LLM is also going to be quite indirect during the long tail of post training, and heavy on automated RL, the human feedback might end up getting used in the form of automated grading guidelines like what you did for research, with the same issues as that, compounded by the input being LLM generated and models being biased towards model output by default. It's more of a feedback on the loop rather than in the loop.
definitely a utopian vision that is not likely to happen in our current reality but I like to imagine better worlds that are possible. as LeGuin once said, "We live in capitalism, its power seems inescapable — but then, so did the divine right of kings. Any human power can be resisted and changed by human beings."
You could make it a state capitalist society with theoretical public ownership and this disrespect towards people won't go away. Like LeGuin - a famous non-anarchist - pointed out in the dispossessed simply liberating the social relations and saying you have no rulers is not enough to build a society without unjustified power, and what power even is or isn't justified will rarely be an easy matter.
I don't know you can build this technology otherwise that is without coercion, with consent, the people that want it have convinced themselves it's too important to try to justify to anyone else why they need it, I think you could eventually do it. But for us at the very start of the development of what became this systems we went into it with a handful of admittedly brilliant people so convinced they have a right to reshape the world they didn't care if anyone else agreed to this, they would have been bad anarresti. If you wanted to build it in a fair way it might be another generation or more before the project would be complete, you would have to first convince people this is something that should be built in the first place, not just that you can build it in a safe way.
this is essentially the PRC in the 21st century post-Deng* - there's probably a cultural difference at play here too given how embedded the CCP is within academic and business institutions, how kinship tends to be extended on a filial level leading to larger networks. mobilizing a large group of subject-matter experts for post-training annotation (eg - https://ojs.aaai.org/index.php/AAAI/article/view/29907) seems to be fairly simple as an ask but, like you said, it doesn't matter ultimately with their largest firms like DeepSeek utilizing automated RL to skip that whole step and this type of training isn't the norm
I sometimes wonder if the problem with the PRC's sinking back to exploitative labor standards would happen in a vacuum. if you weren't surrounded by adversarial nation states with leaders looking to squeeze every advantage, would you, yourself, resort to the race-to-the-bottom of profit sharing?
a much better SF visionary than I would be able to write a story about a very slow growing AI project* trained for specialized use, and how this was the norm and everyone was happy because it was happening at a reasonable pace relative to hardware capabilities (presumably also reduced to avoid all of the slave labor inside of the rare earth trade). I think it's the capitalism side of things that says 'we should be everything at the fastest speed for everyone' that we get things like Claude and ChatGPT which necessitate outsized, disproportionate resources that doesn't allow hardware efficiencies to catch up and mitigate the worst of the externalized costs
*realistically it's maybe more accurate to say post-Jiang given the extent of opening up Chinese labor markets but Deng's trajectory steered the way
*Ted Chiang's Lifecycle of Software Objects does this a bit but it's more of a social commentary on the capitalist abandonment of the functional for the shiny than it is a sharp political critique of the pace of modernization and its effects on people and the environment
Now politicians also know something about their supporters so they will adapt their statements to what they think they can get away with it. But, I wonder if this leads to a two-party-system where one party attracts stupid followers and another attracts the smarter ones?
In terms of AI, we might see LLMs specialized to attract more stupid audience and others meant to attract those who appreciate correctness and facts.
Yes, we truly live in a “post truth” era.
> But, I wonder if this leads to a two-party-system where one party attracts stupid followers and another attracts the smarter ones?
I think all political parties aim at the lowest common denominator.
In case of Hungary, in the past 20 years, they moved contradictions inside an article, to inside a paragraph to inside a single word. Trump is on the single word/expression level since 2016 for sure: “alternative facts”.
You wouldn’t happen to have gone to Westlake HS? lol i may have been that person… Life has since slapped me in the face a few times.
This is potentially expensive advice (at least for many mainstream options). Where an English word like "literature" is one token, a couple of Chinese characters that spell a word can be 4 tokens. You'll pay more for input/output and get less of a context window (per word) too.