DrewADesign

Plot twist: they were both sold on early investment in companies that survived the .com bust. Now they’re VCs that everybody worships as business geniuses even though they’re just lucky idiots, and the sycophantic chatbots finally let them feel as smart as everyone says they are, and a whole bunch of hype-drunk fans are feeding into it.

If you’re just talking about learning a programming language I think you need to be quite judicious in your AI usage.

In my experience, people learn programming languages best by overcoming frustrating roadblocks. You often end up learning something important, even if it’s just about your mindset or approach, that landed you there. This is the difference between someone with a wet signature on their comp sci diploma and someone with a few years under their belt.

A lot of people start with tutorials and cargo-cult their way through solving their first problems, but eventually need to learn how to do things the tutorial code can’t. It seems like the AI coding tools can could perpetually make things that could be bashed together well enough to sorta solve a problem and think “oh I’ll just learn about that later,” and then never learn about it at all. If your goal is to make some quick tool to help you with something at work in a different field, well, touchdown. If you’re trying to learn the language, fail.

I’m very close with several current police dispatchers, and even have an expired state telecommunicator certification. Calling 911 can absolutely be a coercive force, and people use 911 to harass people all the time because in many instances the dispatcher has no choice but to send uniformed people over there to investigate. Police are equipped and trained to deal with certain kinds of problems, and they’re constantly looking for evidence to support a criminal charge. It’s their job. All they know walking into that situation— as human beings that aren’t immune to the power of suggestion— is that the caller, who is the complainant, says that the other person is doing something wrong. That is not a neutral action. If it was, many departments wouldn’t be adopting non-uniformed, unarmed social worker responses to non-violent situations.

You can try to say that I’m arguing whatever you like. If you’re claiming that the underlying structure of digital so-called neural networks is comparable to biological neural networks— which we’ve studied for far longer without really understanding— no amount of jargon will obviate the ‘citation needed’ requirement for that claim.

Well, when you’re on the clock, you’re an agent of that company. And when you’re installing an indiscriminate mass-surveillance camera, calling the police on someone that’s video recording you is kind of bananas. Perhaps you don’t find that to be an interesting event, but a lot of people who are rightfully angry at the non-consensual deployment of a mass surveillance network disagree.

Personally, I don’t think it’s different from any other faith-based motivation.

Yeah I like Cal’s take on it, though in this context I’d argue LLMs have even less agency, and are even less deserving of anthropomorphization than a dog is.

> I don't think knowing that will make me rich.

As someone who’s not really sure that any of this is sustainable, I’d implore you to not sell yourself short. I reckon there’s a ton of dogma and nearly religious zeal among these companies, which among some people is earnest, and among others is cynical hype farming. I’ll bet someone objective enough to focus on using available tooling to solve real problems in practical ways that mitigate actual risks and are honest about actual limitations will be eBay here while the others are going to be somewhere between lucent and pets.com.

::golf clap::

It sucks because I think analogies can be useful in helping people make a mental model of complex things, which is meaningfully beneficial. The problems happen when people aren’t honest about the limits of the analogies, which is damned-near guaranteed to happen with this stuff.

Situations have lots of independent variables, Doctor, and Anthropomorphism is one problematic facet of many in the way this industry is pushing LLM products.

If there was a collision at an intersection with a stop sign partially obscured by a tree, that had traffic volume that would have better been served by a traffic light, on a foggy night, where one person was texting while driving, none of those things would diminish the fact that the other driver was drunk.

> We have no better model for how human decision making works than LLMs

This is a claim that requires a lot of citations.

> biologically inspired

Nature inspires a lot of creation, but superficial similarities don’t mean other aspects are similar. Making an extremely realistic sculpture of a soufflé, even using a foam medium, doesn’t bring me any closer to being a chef, doesn’t mean I know anything about albumen foams, sauces, and heat transfer, and it doesn’t bring me any closer to having dinner ready. Browning on top of a soufflé is evidence of maillardization. You could pull up some studies on that and claim the brown on top of the soufflé sculpture, which I applied with an airbrush, proved that the Maillard reaction was occurring, and if another person didn’t know anything about cooking, they might even believe you. It would, of course, be completely wrong. And the other person, of course, could loudly exclaim that I can’t prove that there was no maillardization.

Can’t agree with you here.

> I love how we all just collectively decided that LLM decisionmaking cannot possibly be like human decisionmaking - because if it were, the consequences would be just too awkward.

In love how people get salty about people not going along with a superficial supposition just because they can’t definitively prove it wrong.

> All that while still not knowing how either kind actually works.

We do know that zero parts of human decision making are based on predicting the next most likely letter based on a giant internet-based database. We do know that’s what LLMs do. We do know exactly how each part of an LLM works even if the combined behavior is too cryptic to feasibly analyze at the moment. We do not understand all of the functions of an actual neuron. Openworm isn’t even close to accurately simulating the 302 neurons of a roundworm and you’d need over 200 million roundworms working in conjunction to equal the number of neurons in one human brain.

My dog seems convinced that the malevolent invader in a mailman uniform would break in and attack us if she didn’t fiercely bark at him, six days per week. I certainly can’t prove the mailman doesn’t want to kill us, and that the mailman wasn’t solely deterred by her barking. Empirically, the mailman goes away soon after she starts barking, and we’ve sustained zero mailman assaults after hundreds of purported attempts. Maybe I should just run with it? Her model is too simple to come up with the obviously correct answer, but it’s not even directionally accurate.

The burden of proof is on the person making the claim, which in this case, is that these comparatively simple logical constructs are remotely comparable to the complexity of biological systems.

I do get their usage intent. If something is at all automated, in English, we often refer to as having some amount of agency. If I started up a riding lawnmower, put a brick on the gas and pointed t it towards a field, many might say I “let it run rampant.” But since nobody is at risk of anthropomorphizing riding lawnmowers, it’s not problematic.

Anthropomorphizing LLMs is a huge fucking problem though and I, personally, think we should expunge all of these casual inadvertent linguistic agency affordances with great prejudice.

OpenAI didn’t ‘let’ these bots do this any more than someone ‘let’ Claude Code make them a website.

You’re not interested because you challenged my definition of negligence by inventing some glib pseudo technicality and then got flustered when I asked you to explain how it was rooted in the real world. Sometimes we get caught taking out of our asses. It happens. But I understand you’re just trying to save face so I’ll leave you here to finish that up yourself. Have a good one.

I asked some pretty straightforward questions about the specific legal issues involved— the gun case being an important point in our legal system’s handling of negligence — and your answer basically translates into you not knowing. Negligence is not exclusively criminal in the US legal system, but the underlying concept of negligence is pretty consistent. If you’re not simply toeing the line for OpenAI and want to move beyond your vibes-based understanding of the legal concept, there’s a lot of info out there, including the full test, the reasonable person concept, and very relevant examples of how this worked in real trials. I’m not interested in debating whatever your personal take on negligence means here because it’s largely irrelevant.

Specifically what legal type of negligence are you referring to and how does that known or obvious rule fit into the standard legal tests? Does that known of obvious rule have to be a law? There were no ‘rules’ about attaching strings from your doorknob to the trigger of a shotgun and pointing the gun at your door, but when someone died from just that, it was pretty famously determined to be negligent.

It’s one of design’s key responsibilities. Developers need a sophisticated mental model of how it works, and how that gets displayed. Designers need a sophisticated mental model of what problems various users are trying to solve, and the best ways to communicate how the software can solve those problems, as well as state, etc. They’re both deep topics, and even as someone with significant professional experience in both design and development, I find it difficult to consider both things at once. Many things that are very important to design could be a serious PITA for developers. Many tools, such as front-end frameworks, that are important time savers in the dev environment are a serious PITA for designers to work around. Many designers don’t understand why seemingly small changes, maybe something that would require hand-rolling a common library component, could triple development time and cause huge maintenance issues. I’d say most developers don’t realize that their understanding of software architecture and construction is nearly useless when designing interfaces, workflows, etc., and that many of the folk ‘best practices’ they’ve learned about design are no more authoritative than any other folk knowledge computer users pick up. They sound like completely nontechnical gamers confidently wielding technical jargon to mis-explain phenomena people experience, and since none of the other gamers in the conversation know any better, it only bolsters their false confidence. Unfortunately, that alone explains why FOSS developers in independent projects repeatedly try and fail to make software that appeals to anybody other than software developers.

> We have a word for attack with no intent. It's accident.

And we have a word for an accident caused by people that failed to implement proper risk mitigation, were not paying attention, and should have known better. It’s negligence.

Don’t worry it’ll be gone pretty soon.

I’ve only heard the term flimflam as a name for a scam that involves tricking a cashier into giving you a bunch of money with confusing overlapping talk.

> As if generative AI needed more publicity for its “we are literally just making stuff up” approach to getting stuff done.

To be fair, there are technical considerations. Like, technically, nobody in charge of these companies gives a shit about misleading people.

In my experience, there’s no better way to learn more complex programming techniques or tools than running into a brick wall with the simple tools you’re familiar with, spending a ton of time on an unreasonably irritating solution, only to realize that you’re running into a classic [n] problem and some clever person figured out a simple logical trick to address it. I mean, it can be enraging but that knowledge tends to stick.

Though I doubt that happens much anymore.

> I think a lot of HN commenters are way too disconnected from the average person

This is basically an old-man-and-the-starfish situation for me.

> You're making a big assumption that a user will even connect the dots

If they do, it’s because their wildly inaccurate mental model happened to guess the right answer. What most will think is “I was just browsing the internet and my computer froze.” Maybe they’ll connect it with that notification they just got about renewing their antivirus software subscription. Maybe it will confirm their (probably mistaken) impression that their computer has been “acting weird” since something arbitrary and unrelated happened. And similarly, some people with an accurate mental model will mistakenly assume that they deduced the cause because they’re smarter, rather than having a different focus with corresponding lacking mental models in other areas.

Sorry, no. Explaining why someone would have taken them at their word is definitely not stupider than blaming people who could have been lied to for trusting a company that lied to them.

Personally, I wouldn’t assume it was lying. To me, dark patterns (like manipulative wording) imply that:

1) someone in a governing body, or someone in the organization, e.g a designer, ethicist, lawyer, developer, etc. has successfully argued that users should be able to avoid something that they determine is not in their best interest.

And also:

2) someone in the c-suite or marketing has decided to mitigate that through some dark pattern.

If they never intended to give the user that control, they’d probably just not give the user the control in the first place. Giving them the option and not honoring it would either imply they were that incompetent or careless with fate protection, which seems most likely if this is all true, or an even more cynical approach to tricking users into thinking it’s not used for training to get them to share better shit. But if that was the case, why bother with the sleazy dark pattern? That seems a little cartoon villain-y to me. I suppose the in-between option would be that they decided at some point they were no longer willing to honor it and didn’t want to deal with the inevitable PR shitstorm of removing it. I could definitely see that happening in this industry, these days.

Ok

I mean it sounds like a retroactively contrived explanation of events. If they released a statement on it, they clearly feel the need to give an explanation of their behavior. People criticize OpenAI every day and they rarely address it. They addressed this. That right there says that, at the very least, they’re concerned about the optics.

I don’t have a dog in this fight, but it sounds a bit “I was just punching the air— it’s not my fault someone was in the way.” It’s not like it’s impossible, but it doesn’t immediately pass my smell test. I’ll wait until someone close enough to be knowledgeable but has a lesser stake weighs in.

The idea is that if I have $200,000 to give to a charity and I give $200,000, my $200,000 yields $200,000 for the charity.

If I say I’m going to match other people’s money 2:1 and that convinces people donate $90,000, my $180,000 yields $270,000 for the charity. I’d probably have written off the 200k anyway and just toss in the rest after, or throw it at another good cause.