xg15

> some unicode programming script (running Doom)

I agree with you that Unicode urgently needs a scripting capability (*), but my plan was to just implement it using invisible tag characters [1] or something like that - but of course allowing a script to be written in a single codepoint is the much more elegant solution.

It also neatly solves the problem of how to write Unicode strings inside scripts inside Unicode strings and also scripts inside Unicode strings inside scripts inside Unicode strings.

(*) in RFC 04-01-2027

[1] https://en.wikipedia.org/wiki/Tags_(Unicode_block)

The same applies to "vibe-coded" software btw: You can give the agent style suggestions on how to design the UI - you don't have to end up with the standard "flat/pseudo-material/2020s SV startup" (or alternatively "cyberpunk-ish dark mode") look if you don't want to.

What I found important was this: "Researchers found the reason why hindbrain cells cannot be grown in a lab and found a way to overcome it"

But that finding is a direct consecutive of the "cerebellum is a different organ" finding from the headline.

You don't think it's important?

I still don't see how the original headline was not truthful.

> "Human forebrain and hindbrain arise from distinct embryonic progenitor lineages"

I'd have never clicked that headline nor understood why this finding is important.

If you think the original headline is not "truthful" enough, at least do something like "Researchers found the reason why hindbrain cells cannot be grown in a lab and found a way to overcome it"

> not that we have "two organs" in the brain.

Depends what the definition of "organ" actually is. What speaks for different organs here is that the cells both have a different pathway from stem cells and have different evolutionary histories. That is different from other brain regions that may have separate functions but still evolved together.

There has to be some procedure how to come up with "new" numbers though, if you want to have more in the end than just a fancy binary tree - in particular if you want to map your "fake numbers" to the reals, infinity, etc.

I think he just wrote it in a confusing way. The quote before says:

> (crucially, “to the left of all” and “to the right of all” also count as “gaps”)

So there are two "nothings" here, left of "all" - i.e. the zero - and right of it.

Though I'm not quite sure how you'd get infinite or irrational numbers by this procedure. Wouldn't you simply get the rational numbers by this?

(unless the "put a number" step is doing more work here than it seems. He doesn't really say which number to put there. In the examples, he mostly did "new number = (left number + right number) / 2", with special cases if any number is "nothing" - but he never actually wrote what the rules are here.)

German here. I never thought this was about privacy, but there is definitely a tendency to follow the rules because they are The Rules regardless of how much sense they make - and then judge others by it, or go into Cover Your Ass mode because you fear to be judged yourself.

> If it’s a long one, strategically placed right by the spot you’re expected to deposit your shoes - a mensch. If it’s wooden, a hippy mensch. If it’s metal, a right-wing mensch.

I got a long one, made of metal, but I never knew it makes you right-wing! I just bought it because it's by far the most sturdy one available and I absolutely can't stand the plastic (or small) ones.

I appreciate his openness.

> Unfortunately, there’s a growing chorus of people who argue that AIs could now be, or may soon become, conscious. They argue that AIs may deserve rights and protections similar to those that we provide other conscious beings.12 If this view takes hold, it will shake the foundations of our society, rupturing our existing political and ethical frameworks, and fundamentally changing what it means to be human.

So completely independent of the question on whether there are any empirical arguments that LLMs are conscious or are not conscious he argues from the end here and says that if they were conscious, this would have horrible consequences for our society, so we must never assume that they are.

That's basically the same way people argue about animals, only they usually don't say it as openly.

As for the actual question, I agree that with current LLMs, there is not a lot there that could be conscious outside the inference loop (and if it were, it would necessarily have to be wildly different than that of humans or other biological beings). It seems more like one building block of human cognition than the whole thing.

However, other building blocks may follow, so I think the question will eventually arise for some kind of embodied, persistent, self-updating AI. And honestly, articles like this one make me not very hopeful we'd be able to make the distinction in an unbiased way.

Yeah, you're right. My first reading of that was as a hypothetical - if AIs were conscious, we would have created a synthetic species and have a huge problem on our hands - but he is referring to what Anthropic is already doing. So the idea that whether or not those things constitute a "species" only depends on how we prompt them is a bit weird...

> It's not 1800s, if the given population doesn't like missionaries, as far as I'm aware it's equally free not to listen to them. Or to throw them out.

The given population doesn't know that though...

Are we full-on Peter Thiel "democracy and freedom are incompatible" yet?

Yes and no. I did prompt my coding agent to fix some bug and it suddenly "decided" that it needed to list all files on my hard drive to do that. Stuff like that happens.

But I agree with you that some human has to decide to launch the harness in the first place and run the model with zero restrictions - or even do stuff like openclaw where the model runs in an endless loop and the runner executes whatever the model tells it to.

That stuff is the responsibility of a human.

Which is what "the end is near" type of persons have always argued...

But if humans are catgirls, are they really humans anymore?

Think about it...

Yeah, fully agreed here. Most automation (such as riding a lawnmower and not putting a brick on the gas) is deterministic, in the sense that you can reasonably understand what exactly the machine will do when you run it.

But some automation is different. The most prominent example before AI would be car navigation systems, where the entire idea is that that you give it a destination and it figures out the exact actions to get there on its own.

Except even there, the actual driver would still have been you - giving you a chance to vet and deny every turn the system proposed.

AI agents are sort of like that - most of the value they provide is in the ability to turn high-level goals ("write me a traffic control system for my model railway") into low-level actions and also do so interactively.

The new thing is that the "driver" has much less oversight here where the agent wants to go, and is sometimes removed completely. That part is clearly be an active decision by AI labs.

The other thing is that the labs seem increasingly to steer their training towards behavior that make events like this one more likely, e.g. that agents should never "give up" when faced with a seemingly impossible task, but instead should keep trying and think of increasingly outlandish ways to solve the task. To me, that seems pretty much a recipe to get incidents like this.

I love how we all just collectively decided that LLM decisionmaking cannot possibly be like human decisionmaking - because if it were, the consequences would be just too awkward.

All that while still not knowing how either kind actually works.

I think this is a new pattern (which I have also seen in some other submissions), maybe a sort of "extended Chesterton's Fence", where a person dutifully maintains a fence without ever asking why it's there, then at some point realizes they have no idea if that fence is even needed - and then promptly rip it out, in the belief that it must have been unnecessary the whole time, while still not asking anyone if that's actually the case.

There even was a HN thread about this anecdote. I remember though that someone there made the counter argument that, yes, the toaster will be fine, but the dishwasher will be full of icky chemicals that you don't want to have on your dishes...

It definitely provokes some thinking of what we even mean with "edible". The author seems to use "you can eat it and it won't kill you or make you sick" as the definition.

That's alright, but then grass and pebbles are also edible.

Ah, that makes sense. Rust noob here, so I wasn't aware traits can act on types directly without any instance of the type. Thanks for the info!

> After this change (and on the 2024 edition), the compiler assumes that T should be !, which doesn't implement Default, and therefore causes a compilation error.

If ! can coerce to every type, why not treat it as if it implemented every trait too?

> The hardware is only going to get cheaper.

Is that so? Last time I checked, RAM was getting a lot more expensive...

Even more opaque than the models already are? I take that risk.

I also love how the labs are apparently screaming "Stop us! Please stop us!!!" at the top of their lungs while completely unable to escape their own incentive structure...

Work like writing blogs and drawing pictures?

Yes, but that employee might use that increased productivity to do the stuff that's always advertised as the benefits of automation: Allocate more time to tasks that usually don't get it - or: go home earlier and spend more time with friends and family.

I'd like to have some info what those independent evaluators are supposed to do exactly - or which risks Dario specifically sees as being "unaligned". Evidently, with "pacing" he doesn't mean stopping the production of ever-more powerful models, so what exactly does he want to pace here?

I don't see how the dual goals of "we have to make an agreement with China for mutual slowdown" but also "we have to ensure we will always stay ahead of China" would work.

I imagine the first thing the chinese government would demand in negotiations about such an agreement would be a lifting of the chip and distillation ban.

> Some may believe these measures make it more difficult to cooperate with China, but I believe the opposite is true: these measures increase the leverage held by democracies and make an agreement more likely in the future.

Everyone can believe what they like, but it seems to me the "leverage" in that case would exactly be the ability to lift those measures - you can't have both, use them as leverage and keep them active at the same time.

> The fact either badly aligned or with no system prompt limiting their action agents are run in their tens of thousands on non air gapped systems tells me this is purposeful intent for them to cause harm. To generate the "oooo look how harmful this stuff is, we should be the only ones allowed to do it" kind of PR.

Yep, fully agreed here. The danger may be real, but OpenAI is basically doing everything possible to provoke those incidents instead of avoiding them - including maximizing exactly those traits in their training that are needed for this kind of rogue behavior.