saaaaaam

Lawyer here.[0] Definitely possible.[1]

[0] not a lawyer [1] what does “nationalise” even mean?

I’m inherently skeptical of big walls of text like this these days.

(So here’s a big wall of text of my own!)

However, a lot of what is written here makes sense.

And particularly “if your comprehension level stops [here] you get 'big words in complex sentence structure sounds smart and right so it is smart and right' even if the reasoning and process is poor”

This is exactly the problem.

And another point you make:

> but the sycophancy of modern day LLMs is an actual danger that enables more harm than good

I don’t think it is necessarily the sycophancy that is the biggest problem (though that is definitely a problem) but rather the combination of authoritative sounding text plus “complete answers” which sound wholly believable but are deeply flawed unless you have domain expertise.

I moderate a forum that deals with people who face a relatively common but somewhat complex (and nuanced) set of legal problems.

The purpose of the forum is peer support, shared experience (“lived experience”) and community.

It’s not legal advice, though moderators will sometimes step in to highlight relevant legal resources (e.g. case law/precedent or primary legislation/instruments).

Prior to AI infecting the forum someone would post their problem, people would respond with their often incomplete or poorly communicated thoughts, the OP would ask more questions - or argue - and a dialogue would occur. That created a community and people would post updates and ask more questions and find common shared experience. Many of them became correspondents with each other and some became actual friends.

In the past 12-18 months the discourse has changed from “here is my personal experience and here is what I did” to “here’s a bunch of stuff an AI says and I’m pretending it is me giving advice”.

Almost without exception the person who has started the thread will react positively to the AI generated content, even when it is egregiously incorrect - but won’t ask questions.

More problematically, these AI posters will often argue specific incontestable points of law “because I asked ChatGPT/Grok/Claude and it says this” and ChatGPT clearly cannot be wrong. And the border of precedence seems to be ChatGPT, Grok and then Claude some way behind.

I’m slowly seeing a pushback from people as “normies” begin to spot AI. But it’s ruined a community because the advice sounds so authoritative and complete that people won’t argue or ask questions.

As a result we have banned AI generated posts and remove repeat infringers.

That’s significantly reduced the volume of posting (below what it was pre-AI) but has significantly increased the value the members are getting.

Gird your loins for a cease and desist from Sirius XM for trademark infringement.

The satellite radio operator Sirius XM owns the internet radio on demand operator Pandora which owns the Rdio trademark, having bought Rdio the defunct streaming platform.

I suspect they would argue that something that’s operating in the radio space is a direct infringement.

So you might want to change the name.

>it feels like reading an impression of a literature book by a high school English class’s most overconfident student who’s only ever read LinkedIn-speak.

Claude is very much the “stupid person’s idea of an intelligent person”[0] which, I suspect, is why it is so popular.

It certainly explains why half the internet is huge chunks of Claude-authored gibberish copied and pasted and published. If people didn’t think it sounded clever they wouldn’t put their name behind its ramblings - but very few of them seem to realise that a lot of people see straight through the bullshit and know instantly that they didn’t write it themselves.

But equally, a lot of people can’t tell, and read whatever it is and think “that person must be clever!” So you have people incapable of coherently expressing thoughts who are using Claude to write on their behalf, with the result that the people they want to think of them as clever think less of them and the people who can’t distinguish clever from AI slop think they are clever.

And the people who can’t tell don’t care, and the people copying and pasting Claude slop seemingly don’t care either.

And then I remember that more than half of the US populations reads at Grade 6 or lower[1], and nearly 1 in 5 people in England is functionally illiterate[2], and I simultaneously despair of - and am thankful for - the bubble of literacy I inhabit.

[0] https://quoteinvestigator.com/2018/01/05/clever/ [1] https://www.thenationalliteracyinstitute.com/2024-2025-liter... [2] https://literacytrust.org.uk/parents-and-families/adult-lite...

> I usually think of it in terms of having a "good" or "bad" session. In a bad session, there is a harmful bias that you can only get rid of through a new session. For example, if you exposed too much context about, say, a variable that features prominently in a doc. The entire session will be anchoring on the importance of that variable. Or if you introduced the notion of CC having to ask for permission for stuff you will have a hard time getting it to "think on its feet" or propose an effective solution (you have made CC so insecure that it now relies on you even for little things that wouldn't normally require your input). In some cases (let's say you have important context in that session) you can overcome this by upping the reasoning level or switching to Fable, but usually a new session is the way to go.

I find this very interesting, particularly your points about "made CC so insecure". I know that we have a tendency to anthropomorphise around these tools, but I have definitely noticed instances where Claude becomes quite hysterical about things - and if you look in the thinking output, it's often after I've pushed back on something, or told it it is going in the wrong direction. It spends a lot of time in agonised second-guessing of itself, going round in circles, before outputting a cringeing hand-wringing response. It's very strange.

Good tip on upping the reasoning level - I've not tried this. I have tried switching to Fable though, which does help. But it obviously very hungry, particularly in longer chats because it presumably needs to remind itself of everything that has occurred so far in the chat.

The point you make about tools that pretend to give Claude "a brain" or "remember" things is also interesting - I find the "memory" feature in Claude so destructive to good outputs that when I'm using the chat interface I am very strict about using Projects, and usually turn off the project memory, or make efforts to manage the project memory and review and delete things that are skewing the outputs.

Oh, that's interesting - because that's absolutely what's happening in my experience.

If I look at the thinking (which seems to have become unavailable in Opus 5 a lot of the time, but was present - and often useful - in 4.8/4.6) you're right - it's having the discussion with itself, and seems unable to distinguish that discussion from discussions with me. BUT it also seems to be related to the length of the chat - this seems far more likely to happen in a longer chat.

I don't understand why they have removed visibility into thinking - I found it very useful, not only for spotting things like this, but also because in more complex discussions it would often mention (useful) things in its train of thought that it dropped from its response - but if I said "when you were thinking, you mentioned this" it would then expand on that point. Taking that away is another thing that has negatively impacted the value I get from Opus 5.0 versus earlier models.

> Of course, it also has no concept of what context the user has and assumes that it must be the same it holds in its memory, which creates this “I didn’t know that you didn’t know” type of communication.

Yes, this is a repeated problem for me. It will drop something in as though we have discussed it before and when I say “hold on, what is this” it realises its error - though on more than one occasion has started to get snotty with me, or actually gaslighted me and pretended we had already discussed it. That was at what I assume must have been the edge of a context window in a very long chat though.

Can you explain what “one of the easiest ways to demonstrate model collapse” means, please?

I think you mean “if it can’t understand its own output this demonstrates model collapse” but I’m not entirely sure what “model collapse” means in this context.

I feel like it’s got significantly worse since opus 4.6 - to the point that it’s almost unusable now for a lot of what I do, which is predominantly textual analysis of documents (“explain this clause and its intersection with this other clause” for example) or summarisation of transcripts. It comes out with such incoherent garbage no matter what rules I ask it to follow.

Genuinely thought you meant you carry out deepfake voice phishing.

My own anecdata, fwiw. I have periods of my year where I spend the majority of my time (like 25-30 hours a week) holding and recording calls, which I then transcribe and review. I’ve never once found an error of this type. I also record and review a lot of quarterly earnings calls, and have never experienced an error of this type.

Depends on your level of interaction in the meeting. I have a lot of meetings where I’m having in-depth “fact finding” interviews with one or two other participants. Taking notes breaks the flow and sometimes the other participants find it rude. So instead record those meetings and generate a local transcript after using whisper.

Quite likely because they are very angry about being shaken and thrown. As you would be too, I imagine.

Didn’t most people keep stick insects (which aren’t quite crickets, but similar) as pets when they were kids?

I thought that was a near universal experience for nerdy academic families, along with hatching frog spawn and having a chemistry set and causing minor explosions in the garage.

Right, I’ve never seen an anime, probably for exactly this reason.

Why did you pick such creepy voices? They are very childlike and weird.

Possibly but the 60s is a very long time ago now… My parents were both adept in Fortran in the 60s when they were in their twenties but now in their mid-eighties I doubt they would be much good at helping someone who wanted to problem solve anything using Fortran!

The other interesting but useless fact is that silk top hats (which as the article points out supersede beaver as the fibre of choice) are no longer made - and the technique for making them is effectively lost because they need specialised looms which are also no longer made.

But if you want to go somewhere super fancy like Ascot and look the part, a silk top hat is de rigueur.

There are now a handful of shops that basically have the entire remaining (good) stock and rent them out for an arm and a leg, and take care of them like rare precious things. Which I guess they are.

Please do this yes! A well-meant blog post written by a human - however "amateurish" you might feel it is - is probably going to be better than clunky AI nonsense. And remember, whatever you write doesn't need to be long winded.

Also, there's no reason not to use AI to help you get your thoughts straight, create some bullets you can write from, give you some structure - and then help you proof your finished writing.

The problem is that when you get AI to write something from scratch like this, it decides what the story is, and how to tell it. If you decide the key storylines, work out what matters to you, work with the AI to brainstorm some ideas and then write that up in your own words and then get the AI to help you polish it but not rewrite it you'll have far better outputs.

I posted a really long response to the parent comment, highlighting the stuff that made it stink of AI. Watch out for those if you do this sort of process.

It’s a blog post written by AI. AI makes everything ponderously dramatic.

I don’t really understand the point of this. Scrivener has a revision history. Even google docs have versioning.

And if you’re writing seriously you’ll know that your first draft often bears no resemblance whatsoever to your final draft. Different character names, different structure, different style, different tone.

You may well have a whole bunch of things that seem completely different and a whole load of fragments that are written revised, scrapped, unscrapped, re-revised, stitched together, pulled apart and then finally assembled into a working draft.

I’m not sure how soemthing that basically counts your words and (maybe) proves you’re not using AI helps with a typical sprawling novel plan.

It wasn’t the em-dashed that sent my spirit sense tingling. It was all the obvious AI tics and tropes. But maybe it’s satirical.

Did you use AI to write the blog post?

"Coordinated attack" might raise more than eyebrows with the legal departments of the companies you've called out.

>Over the next 27 hours, in four distinct waves, all of them tried to use my VoIP (SIP) server to place phone calls on my dime.

As you've clarified in your comment here the companies did not do this. Compromised machines on their networks, controlled by third parties did it.

Bending Spoons also bought Eventbrite.

I’m not if this is satire. If so, well done because you’ve written something about a “business” that is quite literally based on crap.

It’s not a “real business” by any stretch of the imagination.

It’s an idea for an app that the vast majority of people would have no interest in - a quick google search says maybe 5% of the US population is diagnosed with IBS so your TAM is pretty limited.

Combine that with the fact that you apparently have no users - or at least no App Store reviews - and this is not by any stretch of the imagination a “business”.

Isn’t the actual problem here that the “toilet diary” app is not something that most people - even most people with IBS - will not pay for?

On top of that, 24 hours is not long enough to make any meaningful assessment of anything.

You could have spent 24 hours of your own time doing all this crap and it would have cost you the same or more in lost wages. Plus sleep deprivation.

Nonsense app, nonsense experiment. Half way amusing write up. But why on earth did you waste the time?

It definitely has the stench of AI about it, yes. But I really do think that it's gibberish. I have to read each sentence several times - and hop about in the structure - to work out what it's actually trying to say.

> Starting from where each of the 33 boroughs’ residents actually live, and arriving by 7pm on an ordinary September Thursday by public transport, journeys to it span 47 minutes between the nearest borough and the furthest, and no borough needs more than 70. No pub in London does better on both measures.

I had to spell my way through this to work out what it means.

"Starting from where each of the 33 boroughs' residents actually live"

Presumably they live all over each of those 33 boroughs - so this is just nonsense. Or has the "author" carried out a study of the distribution of residential property vs commercial property in each borough, and then picked starting points for each borough? Who knows.

"and arriving by 7pm on an ordinary September Thursday by public transport"

"An ordinary September Thursday" is the sort of ponderous nonsense that AI loves to insert to make writing feel important, but it means nothing at all.

"journeys to it span 47 minutes between the nearest borough and the furthest, and no borough needs more than 70."

"journeys to it" - to what? The pub? Just say so.

"span 47 minutes between the nearest borough and the furthest"

The more I look at this the less I know what it means. The nearest borough is the borough the pub is in - and so someone living next door will take about two minutes to get there. From the "furthest" borough does it take 47 minutes to get there? And by what measure? If you're getting the tube from the "furthest" borough what if you live 20 minutes walk from the nearest tube station?

But then we find that, actually, it doesn't take 47 minutes from the furthest borough because "and no borough needs more than 70". So what's the "span" about? I think it means "if you live in the borough furthest away from this pub, it will take you 47 minutes longer to get there than if you live in the nearest borough". But I suspect the "nearest borough" means "neighbouring borough" rather than the borough in which the pub itself is located.

"No pub in London does better on both measures"

What are the two "measures"? I thought we were talking about one, which is the longest travel time. What's the other measure? I think it means that the two measures are "travel time from the nearest borough" and "travel time from the furthest borough". But this reintroduces the problem of what happens if you live next door to the pub?

Imagine you're looking for a pub in London to visit with friends from around the city, and you want to find one that is a fairly equal journey for everyone involved no matter where in London they live. It's a trickier problem than you might think, and having crunched the numbers there are a surprisingly small number of pubs that fit the criteria. In fact, by my calculations, the best placed pub is The Greene Man on Euston Road because the longest journey someone will need to make, no matter where in London's 33 boroughs they live, is 70 minutes. For those who live closer, in neighbouring boroughs (but not within the borough itself) it will take around 33 minutes to get there - so the span between nearest and furthest is just 47 minutes.

>The bigger finding is what nobody manages: not one of London’s 3,170 pubs is within an hour of all 33 boroughs. Keep the worst-off borough to 60 minutes and the list of qualifying pubs is empty. It stays empty at 65. The first pubs that everyone can reach appear at 70 minutes, and there are exactly two of them, less than a hundred metres apart by Great Portland Street station.

This is even worse.

"The bigger finding is what nobody manages" - this is absolute gibberish.

I think it means "The more interesting data point that this study uncovered is that there are no pubs in London that can be reached in 60 minutes or less from all 33 of London's boroughs."

"Keep the worst-off borough to 60 minutes and the list of qualifying pubs is empty." More gibberish.

Worst-off borough? The poorest? Or do we mean "worst off" in terms of travel time to a randomly selected pub? But that in itself makes no sense, because there are going to be plenty of pubs that you can travel to in under 60 minutes from the "worst off borough". So it must mean something different.

"It stays empty at 65."

No idea what this means on first read. But then, looking back, it means "If you increase the time from 60 to 65 minutes, there are still no pubs that qualify". But there's still the same problem from the previous clause - what are the qualification criteria for this "empty list".

"The first pubs that everyone can reach appear at 70 minutes, and there are exactly two of them, less than a hundred metres apart by Great Portland Street station."

You need to increase the maximum travel time to 70 minutes, and then no matter what borough someone lives in, there are two pubs that anyone can reach.

"Exactly two of them". Exactly two? Or just two? Or slightly less than two? Can you have slightly less than two when you're talking about pubs?

"Less than a hundred metres apart by Great Portland Street station".

Are the pubs "by Great Portland Street station" and one hundred metres from each other? Or are the pubs each one hundred metres from Great Portland Street station? Or are they one hundred metres from each other, and the quickest route from wherever you're coming is via Great Portland Street station?

We then veer into utter gobbledegook.

>"Ranked by keeping the worst-off borough’s journey as short as possible, then by the spread between the luckiest borough and the unluckiest. Every number is a TfL Journey Planner arrive-by-7pm journey from a borough’s population-weighted centre.

This does actually give a little more clarity on the earlier question, of what the "where each of the 33 boroughs residents live" means. It turns out that yes, the author has done some sort of population distribution analysis.

But we're introducing yet more opaque terminology. Earlier we talked about "poorest borough" but now we are talking about "luckiest" and "unluckiest". Is "unluckiest" the same as "poorest"?

‘Worst case’ is the longest journey any borough faces. ‘Spread’ is the gap between the nearest borough’s journey and the furthest’s.

"Furthests's" is 100% actual gibberish.

I could go on.

The whole thing is definitely written by AI, and what's worse is that author either hasn't realised, or hasn't cared that the output is effectively unreadable.

> The maths has no opinion on beer.

Slop has no opinion on good writing either. My heart sank when I opened your website. I genuinely thought it was going to be interesting and it was just a screed of AI gibberish.

Badoom-tish

“Gypsies” are a race of people. “Junkies” and homeless people are, at an individual level, not.

Homeless people and drug addicts may well be victims of circumstance or victims of society depending on your point view - or even victims of their own choices or actions - but there’s a difference between saying “this race of people is problematic” and “this class of people is problematic”.