DrewADesign

Pouring 100% of your tech optimism, and probably portfolio, into a product that some say is about to get Wile E Coyote flattened by market dynamics would probably inspire serious market skepticism.

I don’t buy them being surprised. This perfectly plays into their pattern of using fear to make their models seem more powerful than they are. They obviously had the technical expertise… there isn’t a damn thing a bunch of randos on some HN thread knew about that model that they didn’t. It gave them an excuse to delay their IPO when their books seem like they’re going to be pretty shit compared to Anthropic. It helps them reposition themselves as being more safety-forward which the market is clearly more interested in. All that is to say they had motive out the ass, easily had the knowledge and capability to avoid the problem, knew better than anybody else what the models were capable of, set up the environment, gave it the prompt, and then did not even monitor the output.

Negligence is carelessness. Recklessness, is disregard for a known, substantial risk.

I absolutely believe this was recklessness.

As a commercial artist seeing some pretty serious market disruption based on models that used my art and writing to create, without permission, credit, or compensation, I’d have a hard time not punching anyone in the AI business complaining about other people using their work uncredited. Then I’d probably have a real hard time not doing it a few more times. I’m confident I’d settle for just dressing them the fuck down, but it would take some real restraint.

You can be unsympathetic to the corporation’s bottom line while being sympathetic to the human beings that had their work trivialized by some cocky blog post.

> that does not make it any less dangerous.

But, critically, it does change how and why that event was dangerous. As a deliberately extreme analogy to illustrate my point, consider that pure botulism toxin and home cleaning chemicals could both potentially kill a few hundred people if mishandled.

The toxin requires extreme diligence, knowledge, equipment, etc. to avoid a mass casualty event. The deaths might trigger new policy for handling or access. The company would probably receive a fine and would be on the hook for civil damages.

Home cleaning chemicals would require extreme levels of negligence to accidentally create a mass casualty event. The deaths would likely not change the availability of any of the chemicals, but the people themselves would probably be imprisoned.

With the toxin, the most dangerous part was the chemical itself. With the cleaning chemicals, the most dangerous pet was the idiots handling the chemicals.

OpenAI had to really work to get it as dangerous as it was, deliberately ignored the flaws in the security environment, and didn’t monitor it as it ran. To me, LLMs seem to be a lot more like the cleaning chemicals than the botulism toxin.

I also have ADHD-driven blah blah blah. I find achieving brevity laborious, but find LLM results to be unacceptable. A journalism class taught by a well-known professional food critic helped most, but consistently using the free “Hemingway App” editor was almost as beneficial. Blindly following every suggestion pares your writing down to a nub, but it’s great at highlighting sentences you need to rethink without blurring your voice and ideas with bland suggestions.

That’s what happened in US manufacturing. It’s now 20-30 years down the line and the economically important US manufacturing sectors, like defense, are scrambling to hold onto institutional process and trade knowledge as the final remnants of the old industrial system retire. We all know how useful someone with a fresh comp sci degree and no experience is in a real dev environment is. Imagine the leads showing them the ropes graduated two years prior?

And to dismiss criticism of the report’s obvious slant is even more foolish. Just because the report isn’t outright lying about the facts doesn’t mean it’s an objective, definitive, and unimpeachable analysis of what happened. It definitely shouldn’t be the only thing anybody should read on the topic. This article also has a slant. It’s not an objective, definitive, and unimpeachable analysis of what happened, either. It also, definitely, shouldn’t be the only thing anybody should read on the topic.

I did read the report. Yes, this article does criticize the presentation of the events in that report. This article doesn’t say that the report is lying, but it does say that it deliberately sensationalized aspects in unhelpful ways, and de-emphasizes important considerations. That is important when it’s pretty obvious these companies are using fear to signal their products are more powerful than they are.

If you haven’t read the article, it might present some useful counterpoints. It criticizes that particular report’s portrayal of the incident.

You’re comparing the sub-minimum-wage offshore template jockeys to average designers, not “high end” designers. A high-end designer making a one-off poster for a local fair is about as likely as a senior engineer at a FAANG company making a website for it.

It would be like someone paying for a simple website for their small business saying off-shore fiverrrr Wordpress hacks are average coders, and anyone making anything close to a reasonable dev salary in this job market was a high-falutin’ luxury option.

An average graphic designer would ask what unique, non-informational things about the event the poster should portray, do a little bit of research and exploration, and then figure out how to communicate that in pure visual terms. It draws people into the vibe of your event, and hints at the visual impact of experiencing it. You get the nice type work, information hierarchy, attractive layout and color scheme, etc. for free because those are the tools average graphic designers use to visually communicate. This is not expensive compared to the expertise required. With both AI and the Fiverrrr designers, at best, you only get nice type work, information hierarchy, attractive layout and color scheme, etc. You’re not paying for design — you’re paying a graphics software technician for a graphic, like you would at a sign shop.

An average designer is who you’d want to make your posters and maybe a few other types of advertisements, such as postcards, or bus shelters or billboards or banner ads. Someone in a country with a minimum wage of less than a dollar is who you get on fiverrrr. A high-end graphic designer is who you’d contract to do a complete branding and identity overhaul.

I think that’s an exaggeration. Oversimplification? Poor analogy? Sure. Deliberately misleading? I don’t see it.

But there’s rarely one correct answer. Conveying the same approximate message to multiple types of readers is extremely difficult in a handful of words. People with subject matter knowledge are annoyed by headlines written for laypeople that don’t know the jargon or have a mental model of the important concepts that jargon represents. They always seem to miss important distinctions, make poor analogies, or the like. Headline-length blurbs that successfully convey those things to the knowledgeable are often utterly opaque to laypeople causing them to ignore articles about topics they’d find interesting. The only option is trying to estimate the subject matter expertise of the audience, and despite many people in the tech world assuming their software knowledge gives them technical expertise in many non-software fields, the layperson headline is absolutely the best approach here.

> Anyone applying for a job should be doing this with their cover letters

Sadly people lost the skill to write a standard form concise 3-paragraphs-of-3-sentences cover letter even before keyword-based ATS initiated the beginning of the end. It wasn’t supposed to be a summary of a CV and an essay about your lifelong passion for automated ad-bidding quality checks or some other shit employers delude themselves into believing someone might organically be really into.

Same

Look at that dev productivity go! The continually softening job market means c-suites everywhere care less than ever about their company’s image among developers. They’re surely thrilled to stop pretending to give a shit about anything beyond the next few quarters’ stock prices and their bonuses.

Plot twist: they were both sold on early investment in companies that survived the .com bust. Now they’re VCs that everybody worships as business geniuses even though they’re just lucky idiots, and the sycophantic chatbots finally let them feel as smart as everyone says they are, and a whole bunch of hype-drunk fans are feeding into it.

If you’re just talking about learning a programming language I think you need to be quite judicious in your AI usage.

In my experience, people learn programming languages best by overcoming frustrating roadblocks. You often end up learning something important, even if it’s just about your mindset or approach, that landed you there. This is the difference between someone with a wet signature on their comp sci diploma and someone with a few years under their belt.

A lot of people start with tutorials and cargo-cult their way through solving their first problems, but eventually need to learn how to do things the tutorial code can’t. It seems like the AI coding tools can could perpetually make things that could be bashed together well enough to sorta solve a problem and think “oh I’ll just learn about that later,” and then never learn about it at all. If your goal is to make some quick tool to help you with something at work in a different field, well, touchdown. If you’re trying to learn the language, fail.

I’m very close with several current police dispatchers, and even have an expired state telecommunicator certification. Calling 911 can absolutely be a coercive force, and people use 911 to harass people all the time because in many instances the dispatcher has no choice but to send uniformed people over there to investigate. Police are equipped and trained to deal with certain kinds of problems, and they’re constantly looking for evidence to support a criminal charge. It’s their job. All they know walking into that situation— as human beings that aren’t immune to the power of suggestion— is that the caller, who is the complainant, says that the other person is doing something wrong. That is not a neutral action. If it was, many departments wouldn’t be adopting non-uniformed, unarmed social worker responses to non-violent situations.

You can try to say that I’m arguing whatever you like. If you’re claiming that the underlying structure of digital so-called neural networks is comparable to biological neural networks— which we’ve studied for far longer without really understanding— no amount of jargon will obviate the ‘citation needed’ requirement for that claim.

Well, when you’re on the clock, you’re an agent of that company. And when you’re installing an indiscriminate mass-surveillance camera, calling the police on someone that’s video recording you is kind of bananas. Perhaps you don’t find that to be an interesting event, but a lot of people who are rightfully angry at the non-consensual deployment of a mass surveillance network disagree.

Personally, I don’t think it’s different from any other faith-based motivation.

Yeah I like Cal’s take on it, though in this context I’d argue LLMs have even less agency, and are even less deserving of anthropomorphization than a dog is.

> I don't think knowing that will make me rich.

As someone who’s not really sure that any of this is sustainable, I’d implore you to not sell yourself short. I reckon there’s a ton of dogma and nearly religious zeal among these companies, which among some people is earnest, and among others is cynical hype farming. I’ll bet someone objective enough to focus on using available tooling to solve real problems in practical ways that mitigate actual risks and are honest about actual limitations will be eBay here while the others are going to be somewhere between lucent and pets.com.

::golf clap::

It sucks because I think analogies can be useful in helping people make a mental model of complex things, which is meaningfully beneficial. The problems happen when people aren’t honest about the limits of the analogies, which is damned-near guaranteed to happen with this stuff.

Situations have lots of independent variables, Doctor, and Anthropomorphism is one problematic facet of many in the way this industry is pushing LLM products.

If there was a collision at an intersection with a stop sign partially obscured by a tree, that had traffic volume that would have better been served by a traffic light, on a foggy night, where one person was texting while driving, none of those things would diminish the fact that the other driver was drunk.

> We have no better model for how human decision making works than LLMs

This is a claim that requires a lot of citations.

> biologically inspired

Nature inspires a lot of creation, but superficial similarities don’t mean other aspects are similar. Making an extremely realistic sculpture of a soufflé, even using a foam medium, doesn’t bring me any closer to being a chef, doesn’t mean I know anything about albumen foams, sauces, and heat transfer, and it doesn’t bring me any closer to having dinner ready. Browning on top of a soufflé is evidence of maillardization. You could pull up some studies on that and claim the brown on top of the soufflé sculpture, which I applied with an airbrush, proved that the Maillard reaction was occurring, and if another person didn’t know anything about cooking, they might even believe you. It would, of course, be completely wrong. And the other person, of course, could loudly exclaim that I can’t prove that there was no maillardization.

Can’t agree with you here.

> I love how we all just collectively decided that LLM decisionmaking cannot possibly be like human decisionmaking - because if it were, the consequences would be just too awkward.

In love how people get salty about people not going along with a superficial supposition just because they can’t definitively prove it wrong.

> All that while still not knowing how either kind actually works.

We do know that zero parts of human decision making are based on predicting the next most likely letter based on a giant internet-based database. We do know that’s what LLMs do. We do know exactly how each part of an LLM works even if the combined behavior is too cryptic to feasibly analyze at the moment. We do not understand all of the functions of an actual neuron. Openworm isn’t even close to accurately simulating the 302 neurons of a roundworm and you’d need over 200 million roundworms working in conjunction to equal the number of neurons in one human brain.

My dog seems convinced that the malevolent invader in a mailman uniform would break in and attack us if she didn’t fiercely bark at him, six days per week. I certainly can’t prove the mailman doesn’t want to kill us, and that the mailman wasn’t solely deterred by her barking. Empirically, the mailman goes away soon after she starts barking, and we’ve sustained zero mailman assaults after hundreds of purported attempts. Maybe I should just run with it? Her model is too simple to come up with the obviously correct answer, but it’s not even directionally accurate.

The burden of proof is on the person making the claim, which in this case, is that these comparatively simple logical constructs are remotely comparable to the complexity of biological systems.

I do get their usage intent. If something is at all automated, in English, we often refer to as having some amount of agency. If I started up a riding lawnmower, put a brick on the gas and pointed t it towards a field, many might say I “let it run rampant.” But since nobody is at risk of anthropomorphizing riding lawnmowers, it’s not problematic.

Anthropomorphizing LLMs is a huge fucking problem though and I, personally, think we should expunge all of these casual inadvertent linguistic agency affordances with great prejudice.

OpenAI didn’t ‘let’ these bots do this any more than someone ‘let’ Claude Code make them a website.