davidclark

>If you were trying and failing to use an LLM for code 6 months ago †, you’re not doing what most serious LLM-assisted coders are doing.

Here’s the thing from the skeptic perspective: This statement keeps getting made on a rolling basis. 6 months ago if I wasn’t using the life-changing, newest LLM at the time, I was also doing it wrong and being a luddite.

It creates a never ending treadmill of boy-who-cried-LLM. Why should I believe anything outlined in the article is transformative now when all the same vague claims about productivity increases were being made about the LLMs from 6 months ago which we now all agree are bad?

I don’t really know what would actually unseat this epistemic prior at this point for me.

In six months, I predict the author will again think the LLM products of 6 month ago (now) were actually not very useful and didn’t live up to the hype.

We’re now finding that sounding helpful and constructive does not equal being helpful and constructive. I wonder what an updated comic would say.

> The overall average ability of people being able to get from Point A to Point B safely and reliably, especially in areas they are unfamiliar with, has certainly increased dramatically.

Is there evidence for this?

Last time this happened to someone I know, I pointed out they seemed to be picking the first choice every time.

They said, “Certainly! You’re right I’ve been picking the first choice every time due to biased thinking. I should’ve picked the first choice instead.”

The same legal rule applies to both for determining whether something is a derivative work.

No one is stopping you from using similar proportions or colors as Miyazaki to draw a character. You are also allowed to draw your own interpretation of an electric mouse-like monster.

Copyright infringement occurs if that character looks exactly like say Totoro or Pikachu. That is not “in the style of”, that is copying.

A problem with LLMs is that since their corpus is so large, it is difficult to identify when any given output is crossing that line because a single observer’s knowledge of the works influencing the output is limited. You might feed it a picture of your grandfather and it returns an almost exact copy of a grandfather character from a Miyazaki film you haven’t seen. If you don’t share the output with others, it might never be noticed that the infringement occurred.

The given argument conflates the slightest influence with direct copying. It is a reductive take that, personally, I’ve found emblematic of pro-LLM arguments.

Is this a thing? Why would it be? Look at my username - how many people with that name exist in the world?

Only one of us can have a blue check on Twitter? Which one?

How would it be trivial? Can you describe in a more specific way?

The data I can find says it was last updated 9:02 PM Jan. 5, 2025 (presumably America/Chicago from my browser). That’s a >2 day window as of writing this comment.

Not throwing any accusation, just trying to understand the technicals.

If there was any manipulation of community notes in the last 2 days, how would we know?

If there’s manipulation of this data before it is published, such as ratings or notes never hitting these data files, how would we know?

Maybe, an individual could check to see their own contributions are included in updates to the published data. Is that sufficiently common such that it would get caught?

Community note data I can find (log in required): https://x.com/i/communitynotes/download-data

Have you written a computer programming language? Calling every single one “verbose and stupid” seems like a Chesterton’s Fence issue.

You’re making fun, but I’d send my kid to this school.

Absolutely prefer nothing here.

I know it’s just a spicy take on a forum, but this sounds like a terrible public policy.

I’d like the same article topic but from the person who did Day 1 pt1 in 4s and pt2 in 5s (9s total time).

Is this article AI-generated? This website appears to do a lot of “diving in”.

This is a really nitpicky thread when in my top comment I allowed for the possibility of private sources:

> If it truly cannot be linked because it is private, more context is still needed to understand what this data means.

“based on communications with Moody’s analysts”

“based on internal data from Moody’s”

“data from Moody’s” with no qualifier indicates the reader should be able to reasonably find the information themselves (which they can’t in this case)

Do you think the reporter at FT accessed this information on a paper report which was mailed to them?

If not, then it is on the internet somewhere. Whether it is on the “public” or “free” internet is different. If it is not freely available, then they could still give a real citation, so someone else with access to Moody’s private data could find it.

Yep! And, since I provided the basis for my commentary, you don’t have to trust my interpretation.

My focus was critiquing their phrasing, which turns that 25% into 38%.

Like I said, I’m not actually an expert on this to know if the trend is what matters.

Moody’s report could be aggregating the Fed data, but we’ll never know without a real citation.

> Credit card delinquency rates, which are seen as a precursor to write-offs, peaked in July, according to data from Moody’s, but have only fallen slightly and remain nearly a percentage point higher than they were on average in the year before the pandemic.

This is a prime example of a style of reporting that really grinds my gears.

The citation is clearly to another internet source, so a link should be provided. If it truly cannot be linked because it is private, more context is still needed to understand what this data means.

I actually can’t find the source myself, but I can find “Delinquency Rate on Credit Card Loans, All Commercial Banks” from the Federal Reserve. [1]

The percents from that source somewhat match those referenced in the FT quote. “Peaked in July”

- 2024Q1 3.15%

- 2024Q2 3.24%

- 2024Q3 3.23%

Using 2019 as “the year before the pandemic”, the average was 2.5825. Is +0.6475 “nearly a percentage point”? I guess it technically would round up.

Seemingly important context that the quote doesn’t give is that 3.23% is lower than any time 1991Q3 to 2011Q4. But, maybe the trend matters more for this metric.

[1] https://fred.stlouisfed.org/series/DRCCLACBS

Someone being deceived is a victim, yes.

> If some ChatGPT text can trick you then your process is broken anyway

This is pretty unfair and seems like victim-blaming when we have companies spending billions of dollars to create these programs with the specific intent of trying to pass the Turing test.

Might just be me, but I also read in a condescending tone to these types of responses akin to “let me google that for you”

Seems like you’re trying to make a counterpoint that’s just reductio ad absurdum

> As a dev, I shouldn't have to care about responsive designs or tech stacks or accessibility or versions of node libraries, all to provide a website.

This sounds like, “As a carpenter, I shouldn’t have to care about types of wood, or saws, or cuts, or ergonomics, all to make a chair.”

Wordle was famously made and run by one person. How many are needed to keep it running now?

> We also found good levels of accuracy: the generated documents were 70% accurate, and the generated code was at 60%.

How is accuracy measured here? Is a document a single file? Is the LLM generating code and some separate kind of “document” such that “code” accuracy can be 60% while “document” accuracy can be 70%?

> Ask a question in a more natural, conversational way

I think this might actually be my main pain point with LLMs. Personally, I don’t want this.

I understand it might be helpful for other people. But, I prefer highly specific, advanced search functionality, such as site: or filetype: in google/ddg searches.

scryfall.com for magic the gathering cards is a great example. I’d much prefer typing a few brief flags such as “id=r” instead of “Get me all red identity cards.” And I know I’m getting all red identity cards with scryfall’s current search functionality.

They are also composable, so I can add/drop ones easily instead of perfectly rephrasing a whole sentence because I wanted to change one clause.

I’d need the same level of trust in the LLM’s filtering capabilities as I do in those boolean or regex matching field filters. An escape hatch to hard filters probably would be best for my experience searching things.

This counterpoint feels like it needs just as much scrutiny as the position it’s refuting.

Isn’t “but everyone else has one” the appeal kids make to their parents about most everything? (I know I was guilty of that as a kid myself)

Why is this a new level of “denying them a social life”?

If I tab complete my function and variable symbols, does my lsp write 80%+ of my lines of code?

This is so cool! Flame graphs are super helpful for analyzing bottlenecks. The eflambe library for elixir has let us catch some tricky issues.

https://github.com/Stratus3D/eflambe/blob/master/README.adoc