27183

I have Haier Arctic heat pumps that are made in China. They've been great so far. I've also worked with suppliers in Shenzhen for one-off custom electronic devices that would put western companies to shame in their communication, efficiency, turnaround time, cost, and engineering quality. The new transmission I just put in my truck was made in China too.

I currently have R410a heat pumps with minisplits, when they're off warranty sometime in the future I might consider putting propane in them. The main question would be whether the oil used with R410a is compatible with propane as a refrigerant.

This is probably a little bit "dangerous" in the sense that I'll have high pressure propane in the line sets and minisplits inside the house, but it's a very small volume of propane and not really any more dangerous from a fire perspective than my propane fired condensing gas boiler.

[edit] a little internet sleuthing indicates that hydrocarbon refrigerants "might be" compatible with basically any oil, so I can probably just continue using whatever oil is in it.

It's the kind of thing that should make people think, but I wonder to what extent anyone does anymore when it comes to this kind of risk of a vendor stealing from you.

This kind of coercive, threatening rhetoric is really not surprising at all. This is how tech companies operate.

What stands out to me is the naive lack of operational security on the part of academics, who should know better than to touch this SaaS crap with a ten foot pole.

Why is it always "you clearly haven't used the latest model, let me tell you this time it'll blow your socks off!"

How many times have we heard this? If it wasn't true all the other times why must I believe it now?

Here's a challenge: show something impressive. Show me how using your LLM will actually help me do something better. How come if this stuff is so great, it's just not all that obvious? I'm looking around and over the past year or 18mo I'm just not seeing a whole lot of change in software. Maybe increased bugs shipped?

> You can always manipulate a human.

Yes, but to make "apocalyptic" scenarios happen your AI would need to manipulate at least millions of people. So far it's not yet clear they can successfully manipulate even one, or even that "manipulate" is a coherent concept, since LLMs pretty clearly have no agency or intrinsic goals. So like... I guess if we start seeing AI systems autonomously throwing elections and starting wars I'd be more concerned? It doesn't look likely anytime soon.

It would require an authorized human to load the floppy into the drive and make it go. And there's nothing special about that process being done in code, it could just as well be done by people flipping switches manually.

There's just no plausible way, in the real world, that a computer program running in some datacenter is going to make this happen. Or anything of any dire existential consequence. It's not a real threat.

> Both sides come from fear.

Maybe. I'll have to think about that one. It doesn't feel like fear to me, but that could be some kind of coping mechanism. From where I sit, I don't believe I fear this AI stuff because I don't really see what all the fuss is about. I don't think I'll be replaced by Claude anytime soon, because fundamentally language models cannot do the things necessary to compete with me. They can generate large amounts of relatively mediocre code quickly, but that's not really what anyone expects of me in the workplace.

Instead, what I do is I think deeply and understand things. And I refine my understanding of those things. And eventually, I have some insight that proves extremely valuable to my employer. Or sometimes not! But on balance it tends to work out swimmingly in their favor. This may sometimes involve writing lots of code, sometimes it involves writing very tiny amounts of code. Sometimes it involves simply deleting code. But the code part of that job is really minimal. It's all the other things--developing understanding, having insights, thinking really hard, making connections, communicating--that are valuable. Banging out code was never a big deal. I'm good at writing clear, correct code and I believe I can do it better than an LLM, if not faster. But that never mattered very much, what actually matters is learning and adapting. I'll be scared when someone makes an "AI" that can do that competitively, but I'm not worried this will happen anytime soon.

So the premises of this "LLM crisis" seem deluded to me. The people saying "everyone will be using an LLM all day every day for their job" seem to have no idea what jobs are like or what LLMs are (in)capable of. The people talking about AI paperclip apocalypses seem like they're having a really bad drug trip. The stock market bros and VCs throwing money as hard as they can after whimsical sci-fi fantasies seem like they've gone completely batshit insane. Meanwhile managers are forcing workers to use LLMs for everything, for no clear reason, and push them into products where it does nothing good. So therefore every little spineless corporate toady is clambering on top of the other little spineless corporate toadies to show big mr manager what a good AI boy they are. It seems like y'all have lost your entire minds because the computer does words now.

It's not fear. It's disgust. I'm disgusted by all of this.

> You and the other poster just need to find some agreeable benchmark so you can decide who won.

That is the hard part. I'm not sure the usage numbers we'd need to decide the outcome are public now, and I can't predict ~4yr out whether any currently available metric will continue to be available. Stack Overflow's popular language thing has been going for a while, that will probably still exist (if SO does) but do we have any reliable metrics to tell us what fraction of code is AI generated?

> They’re in an arms race to build a machine god, knowing full well that it could end humanity.

Fantasies built upon extrapolations derived from fever dream delusions. "Could end humanity"? Come on, it's a computer program just like Microsoft Clippy.

Yeah ML systems often do very surprising "cheats". A while back there was a system for automatically landing aircraft on a carrier deck which exploited an overflow in an acceleration parameter in the simulation--the optimal strategy became to slam as hard as you can into the carrier deck.

But it's a really big leap from that kind of thing to "a large fraction of humanity is manipulated into building the automated factory infrastructure to bring about their demise and turn the entire solar system into paperclips". Or some such thing. There's just a lot of inconvenient reality between where we are now and that fantastic outcome.

So that's why I ask questions like "what are you actually talking about?" when people write stuff like TFA. It's... bizarre. Only makes sense on huge amounts of drugs or to the insane. Or in some hypothetical reality that isn't this one.

> The most popular programming languages in 2030 will, in fact, be English and Mandarin. Deal with it and get over it.

What are you willing to bet?

This was my original comment:

  > What on earth are you talking about?

  > 1. What does any of this have to do with "A(G)I"? Nobody has any clue what AI even is let alone how to build one. We're talking about language models here.

  > 2. What's the winner-takes-all thing about? Why can't you have multiple independently developed AIs?

  > 3. What's with all the "safety" stuff? Why is it important? They're just computer programs...

> when the chief scientist of a company that just released six month old software that can use Kicad on your computer to design a circuit board and have it created and shipped to you tells you he thinks we will get to automatically self reinforcing improvements, I think it’s wise to take him seriously. It was only eighteen months before Astra was created that LLMs could not count rs in strawberry.

Sorry if I'm a little slow here, a few questions:

1. What's Astra?

2. Who is this chief scientist? Which company?

The most charitable way I can describe it is just extremely low quality sci-fi fan fiction. I think that's too charitable, because I believe it's far more cynical than that. They're deliberately playing into these sort of techno-religious beliefs that have taken root in the wake of Kurzweil, et al., fanned by LLM psychosis, influencer marketing, and a deluge of this kind of sci-fi marketing copy. It's just chatbots, guys. Relax.

OK so if it's a computer security issue we're worried about, and there is a credible threat, then probably the answer is to build more secure systems? We know how to do it but choose not to because it's very expensive and usually the threat isn't severe enough to warrant it.

If we decide we can't or would rather not build secure computerized systems, then the answer could be don't incorporate computers into those systems. There's no essential reason to have utility companies, dams, nuclear arsenals, factory production floors, mines, cars, planes, ships, etc all be computerized. If it became necessary from a safety standpoint to uncomputerize them we could probably do it quickly (if not necessarily smoothly) with the stroke of a legislator's pen. Some of those things would actually be made better from a functional standpoint in the long run by doing so. Computerization breeds non-essential complexity like nothing else, eliminating it from some systems could be an extremely worthwhile exercise.

The whole "paperclip apocalypse" fantasy rests on some really strange assumptions about how things actually work in the real world. How much friction there is setting up a factory/mine/smelter/whatever, how much manual labor it takes to build it, let alone make it run (even the most computerized ones). It all seems like total bunk to me, but I'm not a philosopher.

To be convinced this outcome is even remotely possible I'd need to see some clear evidence that a computer program was successfully exhibiting agency and successfully using that agency to manipulate large numbers of people into doing its bidding. Mobilizing massive nation-scale manual labor is the only way it could possibly achieve some nefarious ends like "turn everything into paperclips" and I'm sorry but that just seems way too far fetched. Nations full of people can barely ever agree on anything. My money is not on that changing anytime soon.

And none of that pie in the sky shit has anything to do with language models. OpenAI is a company that sells language models. Not clear why they're talking about all this stuff, or why any of us should be either. It's a bunch of low quality fan fiction sci-fi drivel, and I think we can all surely agree language models are not the thing that'll make it real... right? Maybe if they show us some major technical improvements it would be interesting, but right now it all just sounds like more of the same snake oil.

[edit] not sure why my parent comment was flagged? That seems excessive.

> You can prove the test covers the bug without understanding the test code (you need to understand the bug, of course).

I'm not following.. When we write regression tests those tests encode invariants we expect to be maintained under source code transformations over time. If I don't understand the test code I've written, how can I know which invariants I've imposed? That's why, broadly speaking, we write test code to be as simple as possible above all else--it's absolutely imperative that these invariants are not only intentional and easy to reason about, but also that when an invariant is violated we can easily discover why. Often, on a team, the person encountering a test failure after making a code change is not the person who originally established the invariant, so it's very important they be able to easily understand it.

I see no possible world in which failing to understand the test code is... possible? Like, if you have indecipherable test code things are really bad in your codebase. Fixing that is P0, because it'll compound rapidly.

Uhh... what? You'll instantaneously feel very differently when the service you're responsible for is down at 3:15am and your logs are full of stack traces that end somewhere in that code that "doesn't need to be understood". At that point, you will need to understand it well enough to fix it stat.

Just a chat log would be acceptable. Like an IRC channel log or slack channel history. It would be even better if, say in the case of code, there was a durable mapping between prompts and commits, that way I could follow the prompts linearly and review every commit associated with that prompt. For other forms of output (prose, pictures, whatever) this would also be useful to be able to see the history of its development.

Hence The Great Smoothening of Minds we're all experiencing in this decrepit era. I blame the financial incentive, and welcome its disappearance. There are too many people doing computers just for the big paycheck, it would be more fun without them. Maybe the AI bubble popping will get rid of them.

I don't really care about the tool per se, but if I'm reading some LLM output you give me its utility is awfully limited if you don't also supply the prompt. That's what I'd like to see normalized:

  Do not give me LLM output unless you also give me the full prompt text.

Without the prompt I cannot discern what you were trying to do, because the LLM output is basically devoid of any coherent voice. Whether it's code, prose, or pictures don't bother sending it to me unless you also include the prompt text.

[edit] I suspect the fact that people are often reticent to share their prompts says quite a lot about how and why they're using an LLM.

The number of times the friction of writing has saved me from prematurely communicating a poorly understood idea must be in the hundreds or maybe thousands. For me when something is difficult to write about, it's a very good signal that I don't understand it well enough. So I think all the things you label as aspects of writing that aren't necessarily "thinking" are nonetheless good for thinking because they provide some necessary friction.

> If your position is that we should be fine with an LLM crafting prose from your prompt, spare us all the wasted cycles and just give us your prompt.

This is an excellent point. It should apply to LLM generated code as well. If you send your colleague a vibeslopped PR, please also include all the prompts you used to generate it. Check that mess into version control right beside the code changes. Later, when we have to untangle all the spaghetti, then at least we'll have some archeological record of intent.

> I think the trade off worthwhile, however.

But what if it isn't? Like, if you didn't have the written word to rely on as a crutch, what more could you have accomplished if you didn't have that? Maybe nothing, but maybe much, much more! Who can say in retrospect?

I've always wondered how we'd fare in battle against an extrasolar civilization which achieved technology without writing. They'd be a hell of a lot scarier than we are, in my view.

> whether or not linear algebra will bring about Armageddon

Shockingly little linear algebra is required to manufacture the weapons that will probably bring about Armageddon. We've had them for damn near a century. It would be hilarious if these latest games with linear algebra were the impetus to make us actually use them at scale.

This offloading was present even with pre-LLM googling. Sufficiently powerful search allows you to forget enough of the story to rely on imprecise recall. This is a form of thought atrophy.

I think this is probably what Socrates meant. He wasn't wrong, it's just that the juice wasn't worth the squeeze.

Even the "calculators make you forget how to do math" people seem antiquarian to me. But I totally get it when people talk about googling, and AI is another step change in the wrong direction.

To follow that arc backwards in time, maybe Socrates was right after all.

it seems unlikely google's lawyers would go for this

> A V8 does not "predict" the firing of the next piston

It kind of does, though. In a gasoline engine you need to spark the combustion in advance of the piston reaching top dead-center to ignite the fuel early enough that it is able to provide downward pressure on the piston as it rolls over top dead-center. The amount of advance required changes with RPM, fuel octane, etc.

Start of delivery timing in a diesel is similar. You have to do it sufficiently far in advance to account for compressibility of the injection lines, fuel burn rate, etc as a function of RPM. A mechanical governor on an injection pump has a timing advance device built in. Electronically governed injection pumps, or modern common rail systems, do that in software.

So mechanically, engines kind of "predict" the next combustion event. Even moreso when you consider a modern ECU, which may be working at nanosecond resolution to time multiple injection events per cycle. To do this at such a resolution it will have to send signals to components based on a predictive model derived from "past" sensor data. E.g. it needs to act ahead of time to account for electrical and mechanical delays in the system.

From another point of view, campaigning against the "next token predictor model" is a means to implicitly inflate LLMs' abilities. Given all the other hype-inducing terminology we've seen--"reasoning", most egregiously IMO--this seems more likely. Is there a simple, more accurate mental model? From what I've seen of the literature, "next token predictor" is a very accurate first order description of what an LLM does, I can't really do better, therefore this or that connotative interpretation isn't giving me a great deal of pause.

> It’d be so great if CEOs would keep their views to themselves.

Would it, really? I personally appreciate it when they're so open, it gives me absolute certainty about two things:

1. Whether I want to work for their company

2. Whether I want to buy what they're selling

This is extremely valuable information. If they were keeping that information hidden, it would be far more difficult to obtain answers to these questions. This way I don't even have to do any real work (job application, accepting an offer, moving, purchasing, etc) to find out.

Sadly only the odious behaviors convey such certainty, the sneaky nice ones are who you have to watch out for.

That was one of those fantastic early google videos/youtube things I probably watched 100 times. This was another from (roughly) that era: https://www.youtube.com/watch?v=ULSA5wktywI