ipdashc

> But what happens when it touches a single, very critical instance of software?

I don't disagree at all, but the AI companies/engineers to their credit have been very clear on that risk - if anything, a bit too "clear" with all the apocalyptic rhetoric. And I've been disappointed that the overall response has either been "they should go to jail" (for a set of incidents that, while useful canaries in the coal mine, effectively harmed nobody) or "they're making it all up for marketing/monopolization". Sure, it's different people saying both, but it still feels kind of dumb that this is somehow collectively our response as a community.

Admittedly I'm pretty much on your side here, I just think the "yeahh send them to jail!" thing in particular is peak "internet ruining discussion on an important topic by taking it to its most ridiculous conclusion"

> I guess my point is that when you get rich having fun working something you love, maybe you don't go whining online telling people that you missed your daughter's birthday and that they should not be pissed at the potential damage you may be contributing to doing.

Agreed, it's pretty hard to try and get sympathy out of that. But on the other hand, what else can they do? I'd rather they explain their side of things than just stay quiet about it. They are right on their main point, which is that it's a hard problem at its core and I can understand why getting hit with a barrage of "just configure a sandbox dude" would tick anyone off.

> When a bridge collapses and people die, then we say the same: "someone should be in jail". The different with the software industry is that nobody gives a shit about the harm they cause. Civil engineers have to, or they go to jail.

The difference is that 99.9% of software is just not as important as even your average two-lane bridge.

The recent cries to send random OpenAI people to jail frankly strike me as IT people just inflating our own importance. Oh no, HuggingFace got hacked. So what? Nobody got hurt, no individual lost their property. A company had to rebuild some server configs. The reason nobody cares enough to send someone to jail is that it's just not that important.

Also, even civil engineers very rarely go to jail when a building or bridge collapses. This is something everyone is using as an analogy when in reality it's very hard to prove criminal negligence.

> The point here being that maybe, just maybe, AI labs should not be shipping stuff they cannot secure.

Yeah, and when they say "okay, fine, we should slow down" everyone says they're making it up for marketing reasons.

And then if they actually slow down on their own everyone makes fun of them for being obsolete and switches to their competitor.

Tricky problem, isn't it?

Yeah, they do, they pay for the land. That's how buying land works and it's separate from the permitting process to actually build on it. I'm not really seeing where we're going with this

Sure. ChatGPT alone has a billion users. I don't particularly like AI, but it's clearly useful, and a lot of people are using it. (And even this assumes that every data center is an AI one.)

Alright, fair enough. I guess no factories, no power lines or pipelines, no thru highways or train tracks, no office parks, no farms other than subsistence, no radio towers or aeronautical beacons, no bus depots or railyards, no quarries, oil rigs, or mines. Heaven forbid someone is forced to live nearby something that's useful for someone other than themselves.

This is a silly standard and the reason why people make fun of NIMBYism. By this logic we shouldn't build anything at all unless it immediately benefits the people living next to it. (And I don't like how much power DCs are using either, but that should be the argument, not "it doesn't benefit me personally so don't build it".)

Sure? I would be completely fine with a data center next door assuming it's a normal one and not something like xAI's. There are several not too far from me already.

It's a chatbot, not their boss. Who cares?

If anything I'm glad to see them having fun with it instead of further convincing themselves that it's God.

From what I've heard from other fields it seems as though this is somewhat unique to the tech industry. I share your belief that it's ridiculous. Companies somehow got along fine pre-FAANG with a few interview rounds before Google and Amazon convinced everyone that you need to have five rounds, a panel, a take home assessment, and a slate of brain teasers to pick out the true worthy 10x engineer.

Grow some balls, do a couple interviews, and just pick the candidate you like the most. You're hiring for a glorified IT guy, not a NASA astronaut. Companies that stretch out the process to a ridiculous degree need to get over themselves.

EDIT: and to be clear, yes, I've been on both sides of the hiring table

It's entertaining to me that so far you're one of the only replies to notice this :)

I knew mathematicians were a smart bunch, but honestly, the events of the past few weeks have really given me a new level of respect for them.

They might be the first community I've seen to experience the AI "rush" and (at least as presented to an outside observer) immediately come together, assess the situation, and calmly, empathetically, and rationally act. They evaluated what AI is good at and what it lacks. They've thought through how it'll likely affect their field in the future. They've explained where the need for humans still lies, and made clear proposals for how to change their own field and for what demands to make of AI companies. Of course they're not all on the same page, but they're at least talking and trying.

They haven't started worshipping the machine god and loudly claiming their whole field is solved. Nor have they flailed wildly at LLMs as if complaining enough about it will make them go away.

Every major statement I've seen come out of the math community on this matter reads as well thought-through, humble, reasoned, and deeply human.

In these days of fear, uncertainty, and obsolescence anxiety, honestly, they've given me some confidence that maybe we will figure this stuff out after all. Maybe we'll learn from them. Who knows.

> this would be a major issue even without 'the algorithm' and other such 'engagement' maximization efforts

Thank you, I don't know why nobody else brings this up. The engagement algorithms certainly aren't good, we should regulate them, but it seems like a stretch to think they're the root of the problem.

Heck, you don't even have to go back to BBS and newsgroups. People can get addicted to group chats, to games, to browsing HN. Kids can get bullied on all these places. No engagement algorithm needed.

I think there's a fallacy of sorts where when something becomes harmful we assume it's evil. Computers make it really easy to shovel entertainment at our eyeballs, it's easy to get addicted to that and for it to cause social problems.

I haven't really seen this, personally. Basically everyone I know is both at once. We acknowledge that it's really good at what it does and use it, but it's not enjoyable to use, and we end up dejected about the future of our craft and our jobs.

The amount of downright hostile comments is insane. HN moment, I guess.

Guys, I don't think he said "all the airlines need to drop what they're doing and start using my model". He's clearly just playing around with some rudimentary simulations for fun. Did we forget that we can do things on computers for fun?

Yes, but the plane fits hundreds of passengers. Isn't that much obvious?

Not to disagree that flying is very CO2 intensive, but it's not like the airline is burning fuel for fun. The passengers paid them to do it

This looks like a pretty reasonable policy to me all things considered. And no, I'm no fan of Flock. But they do run security cameras for the cops, they can't just say go ahead, go wild on all our customers' cameras. The lawyers would throw a fit.

The carveouts for stuff like configuration and DNS are entirely reasonable. Have you ever been behind a security@ email before? You get a lot of BS reports of that sort.

I think it's pretty analogous to coding, honestly. When it comes to gaining true understanding of a piece of software, I still find today's models almost as useless as they were a year ago.

They can write great code, they can gain understanding of something for themselves, but they still kinda suck at explaining it to people. As the article says, if you want understanding, it seems like there is no replacement for getting in the weeds yourself. A model can help, but it's not going to magically download knowledge into your brain.

Not sure how to word this, but it's been bugging me for a while: In a public consciousness sense, it feels like there's barely sci-fi at all these days, let alone the optimistic sort. Something is off about it in a way I can't really put my finger on.

Every previous major technological shift had an associated wave of sci-fi and speculative fiction, right? We explored space and got a generation of stories about aliens and moon bases and Mars missions. We discovered radioactivity and got The World Set Free, and then the nuclear bomb proper inspired countless Atomic Age works. We discovered the transistor, and Asimov and Clarke gave us visions of possible computer-powered futures. We discovered DNA and got decades of comics and movies about mutants and superpowers and bioweapons.

(Of course, we only remember the good ones. A lot of the forgotten stories were total shit. But people still wrote them! And at least in retrospect, it sure feels like they had a cultural impact, and made interesting observations about the way the technology of the time might shape the world.)

But now? We're currently a few years into what's probably the biggest technological shift since, what, the start of the millennium? And yet it feels like we're all collectively just sitting here watching it happen. The closest we have to popular speculative fiction is some billionaires and Bay area rationalists writing breathless blog posts about either curing cancer with AI or AI killing everyone, and universally being derided for it either way.

And as much as I myself don't like the work those guys put out, the derision does bug me. Because it feels like the effects of an irony-poisoned culture of cynicism. We can't write or discuss speculative fiction, good or bad, about what a future AI world might be like, because... I dunno. It'd be cringe? It'd be unrealistic? Or maybe we just don't want to think about it?

And so we all sit here, for the first time staring down the possibility, scarce as it still is but growing likelier every day, of a future where computers are better at every intellectual pursuit than us. And doing nothing about it! Nobody writing stories about a Culture where the future AIs benevolently rule over us. Nobody writing about a Butlerian Jihad where we take power back from the machines. (You'll note that those two examples are from the 80s and 60s respectively.) We just sit here going "oh man, huh yeah, it'll sure suck if the robots take all our jobs." And hardly even try to imagine a world where that's happened and how much it sucks, or a world where we successfully revolted against it, or a world where we manage to somehow thrive and find fulfillment despite it.

I'm not sure what I expect, really. I know authors can't just pull a grand vision for a future out of a hat. I'm sure many have, and I just haven't read or watched it. But it just feels like something is different this time, like there's something that's gone missing culturally speaking. Oh well. Maybe this is always how it feels being in the Interesting Times, instead of just reading about them after the fact.

Reading this was a relief, I've only got a hobbyist's knowledge of impedance matching in the electrical sense and felt like I was going crazy... Good to know I'm not alone

Hm, it seems like it'd make sense to automate a script to send one small prompt at, say, 7 or 8am so that you get your reset around noon instead of having to wait until 5h or more into the workday.

I never quite understood why they picked 5h, it seems oddly arbitrary

Sure, that's much more feasible, but there's no real point. We don't do this for any other kind of industrial infrastructure (except the kind that HAS to go in residential areas, like utility substations). It's like the ultimate first world problem, spending money and gas on earthmoving so that the few dozen people who might occasionally drive past this thing can glance at a grassy berm instead of a concrete wall.

> They should be underground (that’s not shockingly expensive at the scale we talk about)

... I'm not sure you realize the size of a modern hyperscale datacenter? They're pretty dang big.

The amount of earthmoving to bury one would also be awful for the environment. There's really no need to complicate it that much, they're big warehouses full of computers, just shove them on some unused industrial land somewhere as is already being done.

They should be required to pay for new renewable generation, I agree. That much is common sense.

People in my area are complaining about data centers being built in rural fields and on former aluminum plant and power plant sites. The debate has become pretty detached from reality. What remains rooted in reality mostly comes down to electric bills / power consumption and the fact that, as everyone is saying, it's the only way for people to push back against big tech billionaires.

Wow, greebling is a good metaphor for it that I never would have thought of.

> "every pod has a machine"

Everything you mentioned is annoying, but I think this one is the worst. I don't know why it loves slapping on meaningless info blurbs like that. I guess it carries over from training on test/demo programs where the functionality is clearly spelled out, but on a production UI it looks so bad.

Yeah, I know, I agree with all this. I'm still saying that if your entire argument for the absolutely giant section of society that drives into cities for work, which includes the rich, the poor, the disabled, those with large families, etc etc is "lol just don't do that stupid" don't be surprised when you find it hard to win support.

> Congestion pricing is one of the few policy ideas that's so overwhelmingly positive that even the people who try to argue against it end up strengthening the argument. Love it.

> "What about all the people who need to drive into the city for work?" Uhh... don't!

... Look man, I'm in favor of the policy too, but this reads like straight satire. Please tell me you're trolling.

I think that this is true to some extent, but that people often underestimate how much Discord and group chats get used compared to "traditional" (feels weird calling it that, in this context) social media.

I feel like he is. VM breakouts are vastly rarer than kernel LPEs, aren't they?

> I'd also be interested in seeing where you do and don't agree

I agree that society more or less eventually needs to get to the sort of state you're describing, or at least something resembling it. It should be plainly obvious to anyone that if machines do more and more work, eventually capitalism just isn't going to make any sense. Of course it's a question of when that line is crossed, but we're moving further and further every day.

I mostly "disagree" with regards to how feasible it is in our current global system, or how realistic it is to actually get there. Off the top of my head, a few things:

- It'd probably require international cooperation. If we assume (and this isn't a given, but so far seems likely) that a society that prioritizes ruthless progress at all costs goes faster than one that prioritizes the people and their welfare, the former eventually outcompetes the latter, economically and militarily. You already kind of see this with the EU vs the other superpowers, nobody would question that the EU is a nicer place to live, but they're becoming sort of irrelevant globally. We can question whether this really matters, but it is a factor.

- In the same vein, whatever country decides to nationalize all its data centers (and more) will presumably soon lose all foreign investors. What would be the point of investing anything in that country if it's just going to get taken the next day?

- Public opinion, obviously. The average US voter thinks food stamps are communism, good luck convincing them to nationalize tech companies. Personally I think this is actually less of a barrier than most online people make it out to be, the typical US voter doesn't actually care about economic systems, they just care about aura. Look how many "socialist" adjacent things Trump has done and his party loves him for it. But it is a barrier.

- You have to really trust whoever you actually give control (whether a central government, a more anarchist type thing, whatever) of the data centers, etc to to go with this. Or else they'll just end up abusing that power the same as the companies would.

- It doesn't quite solve the core problem of, in this hypothetical post scarcity future, people no longer having jobs or meaningful lives. It's sure better than a corporate feudal hellscape, but I think fundamentally people are still going to lead pretty depressing lives if there's literally nothing for them to do other than sit around while robots do everything for them. And yes, obviously we're so, so far away from this, but I think one of the points of the OP is that we're getting closer to it every day, and in particular it seems like intellectual labor and art (the fun things!) might be the first to go. Is our future really to do what's left of the manual labor while chatbots (whether privately or publicly owned) do all the thinking for us? I doubt it, but we have to admit it's kinda scary that it's looking like a possibility.

I'm sure there's a few other random things, but that was the few that came to mind. Not explained all that well, but hopefully you find it useful.

> it's not actually obvious who's running these bots

This is fascinating to me. It's a large enough phenomenon that it's affecting the entire Internet and yet nobody seems to know yet who's actually doing it. Which isn't surprising, of course, it's hard to trace back to a source through all these proxies and it's probably a bunch of distinct groups anyways, but still!

Personally I have to wonder how much of it is "scrapers for training data" vs just tool-use LLMs. Even if you use chatgpt in thinking mode you can clearly see it searching and visiting a bunch of different websites to answer a question, presumably faster than any human would. That's got to add up. It's got me wondering why everyone seemingly discounts that as an option