roenxi

It isn't quite a lobotomy, I'm going to butcher the research a bit, but basically what they're finding is that these models create a sort of "bad stuff that I should refuse to engage with" axis in the statistical vector space they operate in. So usually there to be some sort of vector that ranges from 0 for a puppy snoozing peacefully and 1 for writing a virus that exterminates humanity pornographically while broadcasting racist and homophobic slurs (which would be quite something to see I have to say). If the vector is closer to 1 the model generates a refusal.

So what you can do is feed the model a small number of reasonable and likely refused prompts to map out that vector in the model's vector space, then do a fairly surgical weight modification that just hits that vector. The end result is the model is more or less the same as it was before, just with no guardrail refusals. It is quite a clever technique that doesn't even require many assumptions about the specific model being used.

"Propaganda campaign" is the wrong term, but there is still an interesting effect - in the 20th century we tried a bunch of things and liberal capitalism underpinned by democracy won the battle of ideas so badly that the alternatives were basically discredited.

But that didn't convince many people to actually change their ideologies. Instead, the authoritarians rebranded themselves as a type of liberalism and the communists rebranded themselves as a type of capitalism. The net result is there is a lot of intent that tarnishes liberal capitalism, but it is mainly people who aren't remotely liberal capitalists trying to rebrand their bad ideas as compatible with liberal capitalism.

In effect it looks a bit like a coordinated effort because there are a bunch of people working hard to associated prosperous ideologies with bad outcomes.

I don't mind the AI tropes that much, the issue is more that the amount of information is exactly what was in the prompt but the AI manages to add many words saying nothing.

> The screen we shipped in January 2016 made two statements at once: pick intent said your plan matters, while the queue attached to it said what a ladder measures is negotiable. Players parsed the two separately, with total precision. They kept the first and burned the second down.

This is a classic AI construction to me that reeks of Markov chain thinking. The first half is probably the prompt to some extent. The second half ... doesn't actually say anything if you read it closely. Humans can write this sort of content-free fluff but usually they do it for a strategic reasons and not when they're trying to explain something. "Players parsed the two separately, with total precision" resolves a question of clarity that was never raised and for which there is no evidence of problem or solution (it is probably trying to say they understood and disregarded ... something that was basically elided). Then we have a tortured metaphor that doesn't tell us anything about what the players did.

It is quite tiring to read because you have to re-read things a few times to confirm that it didn't say anything and it isn't pretty wordwork worth reading on its own. Reading this is an exercise in trying to reverse-engineer the prompt.

The gap seems to be so small I'm not sure how much we should care. If we assume that the trend in the graph does continue as a rough linear improvement, the open models will have reached the same level as the present closed models in 6 months and likely saturated the benchmark by mid next year. That seems more important than where we are now and the exact level of measurement accuracy in July 2026. There is a difference between US and Chinese models but it doesn't look like it is going to be strategically significant.

Things like the Knesset, attempts to knock out the entire military leadership simultaneously, maybe some sort of attack designed to wipe out a significant % of Israelis although I don't know what that would look like (maybe some sort of hit to take out food distribution or water).

Attacks trying to topple the Israeli state, in essence. Knocking out profitable capital is quite unlikely to do it.

That is similar to pointing to someone on the street and claiming they want to commit suicide because you see no indication that they want to live. Unless there is strong evidence we can safely assume people don't want to be nuked. "I see no indication" is not the standard. The Iranians didn't start this, they don't seem to want to continue it and it is obvious that their planning will involve basic considerations like not doing things that will get them nuked.

Although hitting the AWS data centre in Tel Aviv seems like it would be a great tit-for-tat strike to really annoy the US and Israelis without threatening them in a way that will provoke a nuclear response or doing real damage to civilians. So I don't see why they'd purposefully hold back on that.

> So you’re in agreement with me that “no politics” isn’t really a thing in a space?

Yeah, its a meaningless term. The people calling for it are clearly being inarticulately and are trying to say something else.

But the trans bathroom thing has an obvious resolution. The people fighting over it are more about wanting to pick a fight than that there is an actual issue; if the people around me wanted me to use the female bathroom or the male bathroom either is cool. It's a bathroom. They meet a unisex need. If segregation is an issue in the abstract then stop doing that until there is an objective standard to use instead of a subjective one.

> So they go into "soft skills" that their more talented peers aren't interested in, and sandbag themselves with authority once they get it.

Maybe their more talented peers should learn some of these soft skills and take an interest in authority positions instead of acting like losers? Is there some smart-person conspiracy to put idiots in charge that nobody has clued me in to?

I don't want to be going too far in telling my talented peers their business, but if capable people aren't interested in doing anything to take a leadership position then the leadership is going to be incapable by definition.

> ...the issue is Chinese models, whether open or not, and that's what's looking to be banned...

Is the US planning to invade China and take away their models? Because otherwise China is still going to have the models even if the US bans them. I don't think they're going to be going with the US position on this one.

The paradox can be resolved by assuming they are fallible and limited human beings - they're laying a bunch of people off, they understand at some level that this is evidence that their poor hiring and management decisions have caused some amount of unnecessary suffering, so they feel bad.

But they aren't foresighted enough to have handled the situation appropriately in advance. Or they might have run into the limits of their ability to empathise and only understand that they feel bad but not that the fired people probably feel worse. I'd bet more on the former.

In many ways it is a triumph of modern society how ordinary the people are who head these large and complicated world-spanning corporate entities.

Bunch of software engineers are worried about their future employment prospects.

I did just write "that is an inescapable problem that has dogged humanity since before we were a species" you know. But nonetheless the trans bathroom one has a simple and nonpolitical resolution until the people who want segregation start making an argument about why some people need a special bathroom. At which point the gender thing probably doesn't matter.

This is a part of why people are hopeful that they can keep spaces focused. Many of the popular "political issues" are more theatre around identity than any real substance.

> So forcing everyone to change their meeting place to a venue with a unisex bathroom only is not political, but a trans woman using a woman’s bathroom is political?

Nobody is calling for unisex bathrooms as far as I know, so it can't really be called a political choice. It is just practical - there isn't a consensus about who should get a special bathroom. How can there be a special bathroom if we don't know who is supposed to be using it?

> Also, Helen still wants to be called Helen and still wants to be referred to as a she. By this logic, everyone must go by they/them? Or what?

I don't see how to get away from that one, there is a fundamental rift between the people who think individuals control their identity and the people who think that identity is constructed by someone's peers (see also, shaking off an unwelcome nickname).

But that is an inescapable problem that has dogged humanity since before we were a species so good luck excluding it from any space. If it flares up and diplomacy can't smooth it out then some organiser is just going to have to make a call.

That actually sounds like a relatively simple one to resolve although the practicality might be a bit poor. The nonpolitical resolution is to use a space with 1 small unisex bathroom and insist that everyone keep it clean. If there isn't a consensus on who gets a special bathroom then there shouldn't be one.

It would be more useful to say that communities should be organised around a set of principles and be ruthless about sticking to them.

The issue isn't so much that groups become "political" (which is a hard standard to define) as much as they lost focus on a goal (like being a book club) and morph into an in-group, an out-group and a set of memes used to identify the in-groupers and ostracise the out-groupers.

The point of most political memes is to identify a small group who are coordinating to try and control a larger mass of people. When the local book club becomes a conduit for them that is obviously toxic for a general member joining because of the joy of reading. A book club community should prioritise books and including people if and when they enjoy books.

I think it would be hard to make the label "ethnostate" stick to the US. It is too diverse.

As I follow the stats, somewhere between 20-40% of the US public seems to be open to the idea that Israel is a genocidal ethnostate. I have a low opinion of the US State on the best of days but it actually seems quite remarkable that they would be buying from them given where public opinion seems to be at. The US public is tolerant of a lot of things but the political cost-benefits surely can't make sense for that.

Open sourcing is a complex decision so who knows what their calculations are.

But I'd assume that they're preparing for some sort of winner-take-all market in model quality where if they don't do anything the winner will be aggressive, hostile and American. Likely trying to push the Chinese economy back to the year 2000. If that is the starting point either the Chinese have to win the market (unlikely) or squeeze the profit out of it to make winning the market meaningless.

Publishing high quality open models is a well known tactic for profit squeezing. Being 2nd place with the same business model as the front-runner is a losing strategy in a winner-take-all market so they aren't going to bother with that. But if they can commoditize the model, their superior energy costs and likely coming chip manufacturing wave will hopefully give them a big advantage.

That's how nearly everything works. If people don't think action is in their best interest they don't act. And you can substitute any country for China - although if anythign the Chinese have been unusually competent compared to everyone else. the US has been adopting a foreign policy with a deliberateness of action that could be likened to someone having a seizure.

I expect the Chinese will do something wildly stupid at some point, but they've been behaving quite responsibly so far.

> I have no source for this but I feel like the environment thing is too frequently repeated (also without any evidence).

> They're pumping out solar like mad and places like the Xinjiang deserts actually have nowhere to put their excess electricity from what I've read previously.

You've found a pretty compelling piece of evidence right there. There is a reason they're the ones pumping out solar like mad - because they have relatively lax environmental laws. If they had western-style environmental laws the panels would have to be manufactured in India or Africa or something.

Obviously it isn't the only criteria because I'm sure there are places with lower environmental requirements than China; but the big selling point for the last 20-odd years seems to have been more that you're allowed to manufacture there and they have a competent workforce rather than that the wages were amazingly low.

Yep. https://ourworldindata.org/grapher/median-daily-per-capita-e... for example. Although note the log-log axis which undersells the gaps between countries.

I'm not really sure that China's labour costs can be described as "low" these days, going by GDP per capita [0] they're above the global average. They're a little lower than the Europe median I think but the gap probably isn't strategically important.

The Chinese edge tends to come from their environmental laws; they allow their industry to do things that the West would ban.

[0] https://en.wikipedia.org/wiki/List_of_countries_by_GDP_(nomi...

> America pays workers just 27% of what its wealth allows – the worst in the OECD

The US consistently scores among the best countries in the world for paying people [0]. Is there some way I can lodge an application to be exploited in a similar manner, without having to move there? Being wealthy and having to live among really wealthy people sounds better than being poor and living among equals.

They're inventing a metric there that just doesn't matter.

[0] https://en.wikipedia.org/wiki/Median_income

The CSIRO aren't the group who have to cost, pay for, take on the financial risk of or be operationally involved in running a nuclear power plant. Their opinions are welcome but not in any way final. The Chinese might easily be willing to come in and build something as their industry matures, for example, at costs not currently anticipated.

The more pressing issue is the Australian government is a coven of luddites who have taken a bold and consistent stance against any form of advanced industrial progress taking root in Australia whatever and have identified nuclear power as high technology that therefore must be categorically banned. Leaving the whole question an academic one and leaving the CSIRO the only group taking an interest.

We might suspect the hardware used for the LLM is very similar to a graphics card and so the wattage would be extremely similar to using a graphics card to display a game's graphics in the first place. Which puts a certain ironic perspective on tokarf's original comment.

> but that isn't because they are reasoning better, it would because they included millions of Rubik's Cube states with next moves as text in the training data, I presume.

Isn't it far more likely that the LLM has memorised the well known algorithms for solving a Rubik's Cube and has become intelligent enough to execute them? That seems like it'd be a lot easier than memorising millions of cube states. It doesn't even seem obvious that it could memorise next moves, it seems [0] there are more possible states of the cube than these models have parameters. It'd need to be a Large Rubik's Cube Model (LRCM? LRM?) rather than an LLM.

[0] https://cube.alen.is/

Yeah but that is likely the normal state of things - high status people are human too, we're all shocked. And on Twitter comment - mass media has been a tool of propaganda since shortly after it was invented. Twitter is substantially better than something like a newspaper or TV channel.

The difference in the modern era is the internet is such a robust communication channel that the ultra-wealthy can't pay money to have the negative stories suppressed any more. If anything the current crop seem to be unusually well behaved by historical standards because there are so many eyes on them. Scandals like the whole Epstein thing or #metoo would simply have vanished into silence before the 2000s unless they were being used to lever out someone uncooperative for political purposes.

People can write what they like on their blog, but this article seems to be missing key details - we don't find out what the model is or the coefficients he's using and the model apparently immediately produces unrealistic results. Hopefully it is made clear in the book, but unless someone has read Abundance it is hard to follow.

If the model is just a rough rule of thumb then spot checking 4 cities in completely different legal frameworks, cultures and economic situations doesn't give us a lot of data. Economies are very complicated and any effect can be overruled by something unrelated but more impactful happening.

Then they don't need to tax you and you can just give the money to educational institutions. The argument for taxes is either "I don't think it is fair for me to pay for this alone since others are benefiting" or "this absolutely positively must be a group effort" (eg, the military).

"I want my taxes to..." is a generally a bad start, because the point of taxes is it doesn't matter what an individual taxpayers wants to happen to them. Most taxpayers want the taxes to not be spent the way they will be. That is why they are taxes. If people had an actual choice then in all likelihood they wouldn't pay them.

It isn't my pension, and if it was my pension the major issue would be that I was relying on people I think are untrustable with money to fund my retirement. That is one of those ideas so strategically bad that no tactical success or failure matters. I personally wouldn't choose to let more than a fraction of my money get pushed into that vortex. It looks like a disaster waiting to happen (hopefully I'm wrong and everyone goes home happy).

As a rule of thumb in life, if someone is managing your money then you should by and large agree with their judgement of the markets.