impossiblefork

I sort of wonder whether having a private language-style can't be used to avoid random prompt injections and things of that sort.

Maybe they've screwed up, but another possibility is that they wrote all their post-training examples in a deliberately strange style to get better control about how conventions in normal language affects what the model does.

They become very non-public datasets though, and it does not preserve the culture the books represent.

It's culture. These scans are of course not distributed, so the value doesn't quite continue to exist.

You say "culture", but is is culture, and culture that was interesting enough to write down. Writing a book in the 70s isn't a trivial thing, there are gatekeepers, editors, publishers etc., even for your little book about local history, and the information is valuable to the community it's about. It affects their internal cultural transmission.

In order to read them, presumably.

Just because they are for sale doesn't mean that no one else would have bought them. Destroying them obviously destroys them, which obviously destroys culture. After all, if the books is destroyed, it will not be read.

Think of it like this. If I buy an island, and there are bunch of people who live on it, maybe they're day laborers or whatever, is it right for me to expel them, if I don't want them? Can't that be straight up genocide, if they have some culture indigenous to the island?

The same is true for books. If you destroy culture, you destroy culture. There is no magic which triggered "but I owned the physical book" which means you didn't.

It doesn't matter what property relationship you have to a thing: whatever property relationship you may have to it, you still do what you do, i.e. you take whatever action with respect to it as you take with respect to it.

Another good example is food. If there's a shortage of it, and you still have a great deal, you may think "eh, what does it matter if I accidentally burn some, I save time through my carelessness" but if there are others who aren't getting any, you may be killing people, and may be despised for how you use "your property". The same is true here. Why should one not despise one who deprives others of rare texts, that may even be lost, and thus cause cultural destruction of some culture that may actually be rare, by treating the carelessly. That rare book on model building or whittling from the 70s may actually be important.

Of course they shouldn't.

If the books are rare enough there's a shared cultural value that is being destroyed.

So, for OpenAI 60-100 instead of 20-200. It seems pretty much the same.

Yes, I suppose I wrote this a year late. But again, Euclyd and VSORA. Probably basically the same thing but better.

Yeah, I think a crisis in unavoidable.

Great technology, but $20-$200 a month technology, with a lot of people going for free options. Like the telecom industry. Average is probably going to be like a phone bill. Basically: you can become Verizon or AT&T or T-Mobile. $450 billion a year in revenue. A superb business, but merely a superb business.

Yes, and I think you would get much more effect by going for much smaller and cheaper firms.

Cerebras is only 700 people. I wouldn't buy it, but since it's only 700 people, you can probably build something like it with 1400 people in a couple of years if you set the goals right and hire the right people to lead it.

Euclyd, VSORA etc. exist. If they were buying American, why not Groq.

Then once one has good hardware without an enormous markup, then one can start thinking about building LLMs with it.

Yes, but that doesn't prove that the determinant is actually the real signed volume multiplier.

I think this is what led me to feel unhappy about determinants when I was a first-year university student. You need to actually prove that the determinant is the volume of the N-dimensional parallelepiped, and the axiomatic proof doesn't do that.

So you need basically two extra lines after proving those things so that people can say "okay, the determinant eats ignores all input vector non-orthogonality so that it gives volume".

Lorenzo Sadun's Linear Algebra: The Decoupling Principle would probably be enough too if you added something about determinants.

Correction: One person had quit cybersecurity due to exploding workload due to a changing threat situation.

This really doesn't sound like a talent exodus, since the people leaving aren't technical experts.

I think LLMs are best for ideation, experiments, small things.

They've probably already settled on most of the architecture and the big ideas, so they're details in big things instead of how to make complete small things.

The thing LLMs really speed up is how some ordinary person-- a PhD student, or similar, can whip up a miniature synthetic experiment that turns out to be horrid and needs to be fixed by hand, but which at least gave him a plot on the same day he had the idea. That's, I think, where LLMs shine: prototypes. Anthropic probably doesn't need that to the same degree as the small experimenter.

So, concerning ethical approaches to model development, there are many things worth looking into: copyright is one, environmental cost is another, and whether there are any additional concerns worth taking into account. The list of these concerns can then be used for decision making, and we can ask things like "given what we want to build, can does picking a location that gives us lower environmental cost add so much time or cost that we can't go for that path?", things like that are't crrazy.

When it comes to how humans interact with AI we can ask things like "would it be better if people could detect AI output?" "how can we alleviate skill loss from excessive AI reliance?" Things like this can decide how the models behave. Imagine if taking concerns like this seriously leads to models that are easy to learn from. At the moment people seem to be moving away from that so as to make distillation difficult, worsening this concern.

I'm not going to try to talk about machine consciousness, but these other things are definitely important questions where someone digging into them could make models interact better with society.

I don't think these did. They'd also quit because they felt it was too much work, now that the cybersecurity situation had gone mad.

I have heard of zero murders of NATO and EU agency people, and I think you're quite wrong.

There are of course people like this, recruited on the internet in the many criminal networks that exist, but I don't think they've had any notable successes.

I like the argument and it seems valid, but I have heard the opposite from people working in commercial cybersecurity.

Oh, in that case it is actually flipped.

Top US universities actually have higher teacher/student ratios than top EU schools, and I wouldn't be surprised if this pattern holds in general, so it would be easier to do in the US than it is here.

Well, that's the deal, I assume-- that they weren't allowed to buy the Japanese light source outright, so they bought the American one, even though it required giving the Americans some sort of veto or control.

I guess it sucks if one wants to expert broadly, but if you're big on vertical integration and the Japanese won't sell I guess you take what you can get.

It isn't legally possible for them to do this at the EU level. The EU parliament would never vote for it.

For pressure at the country level leading to this kind of thing I think it's very unlikely. Here in Sweden it wouldn't just require a vote in the Swedish parliament and before this there'd have to be förarbeten and you can't just brazenly push things through with insane arguments, Swedish social convention goes against it-- and there's just no way to get it through.

It also might not even be legal. "We aren't at war with China and I'm a communist, and the US LLMs are so aligned with values inimical to my political ideology that this is interference with opinion formation" might be an actual legal argument that the ECHR or CJEU might actually have to accept.

Yes, but surely the problem isn't that the law requires anything in particular, but rather that a bunch of books are getting bought by an organization that is likely to systematically destroy them and that there among those books are a niche books or unusual books that would end up disappearing, and thus that cultural artifacts and knowledge might be disappearing due to this.

I think what you want is a welfare state, not socialism.

Socialism is not necessarily cuddly. "To each according to his contribution" is the one sentence description of it after all. I want socialism, because it's "to each according to his contribution" but "to each according to his contribution" isn't obviously this nice cuddly we-take-care-of-everybody type thing.

I'd call the New Deal an attempt to bring discipline to go from capitalism to free markets, to control excesses, prevent monopolies and otherwise preventing people from locking down the economy so that there can be competition, and that's a goal compatible with socialism, because it does bring us closer to "to each according to his contribution" but it isn't an attempt at creating socialism, but of breaking capitalism down to create a market economy with free markets (capitalism is not a market economy, capitalism is the accumulation of capital by some to the exclusion of others, so capitalism and a market economy with free markets are actually things that are extremely incompatible).

There's no stopping bioweapons. Bioweapons are easy. The reason bioweapons aren't built is because very few biology nerds with sufficient lab skills are evil; and just having an LLM won't give you the lab skills to do it.

Anyone who can publish a gene technology/biomedicine paper can make a bioweapon. If you wrote a paper about how to make a bioweapon easily, it would be unpublishable not because of any danger, but because there wasn't enough novelty.

This is actually illegal here in Sweden though. Lawsuits are ongoing.

I'm kind of surprised that there haven't been criminal cases.

How would that be justified, because as I see it would necessarily treat model outputs as more protected than actual human output?

Surely Anthropic/OpenAI ToS isn't something which can have more weight than, let's say, the ToS of a website expressed through robots.txt, considering that for the length, actual human effort has gone into writing the text, at a cost much greater than anything output by an LLM. So so long as the AI firms are allowed to crawl and then train on human-generated websites that don't allow crawling, but are crawled anyway, I think this kind of thing is really hard to justify.

It's probably actually good, but after this stuff with Qwen-Image-3 and it's ability to things like render images that look like they were rendered with LaTeX or compose many things side by side or in depth, maybe it's perhaps not totally in the right direction to only focus on visual stuff.

Mm.

Still, maybe less money could be enough.

If Euclyd's chip costs 40k per chip and we want to substitute all worldwide H200 inference capacity, so let's say 700,000 H200eds and Euclyd are right about how good their chip is, then we'd need 28,000 Euclyd cards, this is only 1.120 billion.

So maybe even 1 billion for VSORA chips this year and then two billion for VSORA and Euclyd chips next year, with the split based on how well the chips perform and some money for the lagging firm so they can keep designing things too.

I am actually not quite sure.

Here in the EU Rhea 1 was delayed and thus maybe a bit of a failure, but Rhea 2 might be good and seems to be on time. OpenChip seems to be on time too. The university work, people like Hochreiter etc. is clearly good.

What are the high-profile failures?