dsign

Though your statements are generally true today, I wonder if I can put a bet somewhere about how long it takes until somebody "fixes that problem."

Hmm. You are making a good point, and I wouldn’t give the benefit of the doubt to this post’s author. But you can’t just unsee “this is twice load bearing”. Some people will wonder if it’s a training, self-reinforcing feedback artifact. For others, it will work like a memetic virus[^1].

The paradoxical thing is that those of us who have English as a second language should have an easier time producing non-AI-English by the mere act of directly writing what we want to write.

[^1]: https://en.wikipedia.org/wiki/Viruses_of_the_Mind

There is a persistent point that this article makes really well: numbers. And here, bigger seems to be worse. More people equals feebler interactions, worse social life, more alienation between citizens and government. It's not the full story, but this simple rule is a very solid first approximation that has worked fine for me. The relatively small country where I moved has an ok government. Personally, I've found that small towns work well for me, and I've chosen job and residence based on that principle. I wouldn't mind a countryside cabin for hunting-gathering with friends during weekends :-) .

At the same time, I like global shipping, the Internet, good highways, and I definitely hate tariffs. I don't think we need to be disconnected to be happy. We just need to make the right choices where it matters.

> That's not really the same as it having actual consciousness,

It's trivially easy to come up with definitions of consciousness that support your point. But there isn't one of them that it is trivially singular and above all others from a scientific point of view. If you say, "I know consciousness when I see it," you are just making the point that consciousness is attributed by the beholder. Idem if you affirm "I'm conscious". If you say, "only humans are conscious", you are socially attributing consciousness. Now, I don't want to be argumentative, I honestly haven't been able to figure this matter in a decade of armchair internal debate[^1], and it seems I have some company.

[^1]: Having this internal debate makes me believe I'm conscious, but now that I have seen LLMs in action whose "internal debate" is more lucid than mine, I honestly question that "internal debate" is sufficient.

Your teddy bear, alas, is not cared for in a socially determined way. But you could create a church of your teddy bear and, with some grit, you could get people to widely assign consciousness to it. We have done that sort of thing for thousands of years, and not with fluffy teddy bears; it has been more often with rocks, though not exclusively. It's fun to think about how much of humanity's history and culture is the product of consciousness socially attributed to bad weather [^1].

I'm pretty sure somebody will create an improved synthetic simulacrum of self awareness in the coming years, suffering and all, just so that we can enjoy better synthetic software architects and product owners with "skin in the game". If that consciousness simulacrum were put in a teddy bear that shares your whole life and gives you decent advice and conversation, for how long do you think you could, in your private mind, deny it consciousness? Worse, what would your human friends think if you were to maim your teddy bear?

[^1] https://en.wikipedia.org/wiki/Yahweh

Anecdotally, I find AIs, and the doom and gloom they will cause, are great water cooler conversation topics that engages most of the team. However, ever since the advent of LLMs developers have stopped speaking of programming languages and software technologies almost entirely. The number of conversations on manual crafts, artistic endeavors and entrepreneurship has gone up.

> > The cynic in me bemoans that we brought this upon ourselves, in more than one way.

> I enjoy working with other employees and people need human contact to become fully self-actualized. Isolating people to maximize productivity misses the point and brings no more productivity and just avoids fixing any real existing problems.

While I share the feeling and the posture, I haven't managed yet to use them for better documentation. In fact, for me enjoying human contact takes precedence to pressing people--however politely--into writing better docs, specially now that we have LLMs. The skill of writing should be taught and nurtured in school, and if you made it to the workplace without it, then maybe it's too late and you are better off using an LLM to assist you. Which is exactly what I bemoan.

> The subscription models have so many more perverse incentives baked in

Yes. But, as your "so many more" implies, there are also perverse incentives in pay-per-token. And now, for the first time, the companies with the perverse incentives also happen to own the intelligence needed to, ad-nauseam, evade market and customer oversight. Potentially, this is a war where one side can inflict a thousand paper cuts in one second and the humans are on the other side. I think this is going to be an interesting test, a taste if you will, of what AGI means.

You have great colleagues. I wasn’t comparing frame of mind of my colleagues and LLMs, which is a loaded notion anyway, but now that you mention it…

I don't like AI-generated text at work, at all. It feels lifeless and unfocused.

But what I hate the most is that it is objectively better than what I had before. No typos, clear structure, and, regrettably, the verbosity and autistic obsession with detail of the LLM is more actionable and useful than the human guy who wrote lists of commands and URLs as documentation, without explaining anything. Or the colleague who writes in uppercase and with question marks and who doesn't make any sense and forces me to engage in an interrogation effort to get to the bottom of what they are trying to say. Or the colleague who simply hates writing--despite being decent at it--and will call you to give you a meandering verbal explanation that lasts two hours of what they want from you. The cynic in me bemoans that we brought this upon ourselves, in more than one way.

My take is that "sovereign computing" will go from something 0.001% of the population cares about to over 1% in the coming three decades, i.e. three orders of magnitude. It will depend on how long we continue in the current survtech course. There is also the question of if we will see mandatory backdoors in hardware. For example, Country X could legislate that all CPUs sold in their territory need to have a latent command-and-control area for law enforcement to use. If that happens, I'll start converting my savings into a physical stash of old reliable CPUs.

In any case, it seems that CPUs have a very long shelf life; it's still possible to buy 8086 processors[^1].

[^1]: https://www.digikey.com/en/products/detail/rochester-electro...

> if this software is ethical?

IMO, it absolutely is.

First, if fruit flies could suffer existential anguish the same way humans do, they would breathe in relief that digital upload is in sight.

Second, even if the fruit flies don't care, a lot of (awful) humans would love a resolution to the matter of souls in a way that results in a raised middle finger to a forever-hypothetical Creator, and if we can't yet escape cancer, heart disease, parkinson and all the other awfuls, now at least the fruit flies are on their way out. Which rejoices said (truly awful!) humans, which will take what they can :-)

Who the hell needs more than 640 KB anyway? :-)

Jokes aside, I've been extremely frugal with memory in my latest project, in the hope that it can fit in a few megabytes of RAM. But we have come a long way since the eighties, and now an audio codec (OPUS) takes the best part of a megabyte, and so do "frugal" scripting engines such as Lua and even SQLite \o/.

Anecdotally, I saw Opus 5 come up with a complicated loop unrolling technique when I asked it to implement a simple biquad in SIMD, missing a simpler solution. Maybe it was a downgraded session, who knows. That SIMD instruction set, the one for the ESP32-P4, is not very popular and all the documentation it has is a couple of blog posts. So I'm pretty sure it has at most seen some code for a predecessor during its training. However, the LLM was able to derive a full listing of the operations and their arguments from gcc to get us started, and that's why I was able to come up with my own implementation. Along the way, it also came up with insights about possible gotchas. Then, when implementing algorithms, it has been able to reason things out and get things working, despite the ISA not being extremely well known.

I was arguing with a colleague today that we will never have a reason to make our AIs autonomous and sentient. But now I can't stop imagining an AI startup meeting where somebody says "LLMs do not go through the struggle of understanding and don't experience the distress of "wtf am I reading?"" and the next second the technical founders jump in joy "...that's our way in! Let's outcompete the competition by inventing pain for AIs!"

Also, the "D" in this parallel story grew up to become an angel investor and a good friend of Felix. This D and his wife live happily with their cat, and travel the world.

These are another "Jacob" and another "Felix", not the ones in the post, but there are so many similarities and such a terrible lesson learned that it needs to be written.

The Felix I know is still happy; still going out in the long summer days or camping during the winters, playing minecraft sometimes, discovering that life has yet so much to give. Felix has so many ideas in his head that any given day it's a miracle he finds the strength of spirit to prioritize and keep the focus. People at work like Felix because he only compromises on the right things and never throws anybody under a bus so that he can run because they called from the school.

Felix dances in the mornings, and takes catnaps, and tries to keep away the sadness. Sadness not of his life, but because his Jacob died a month ago the most terrible dead possible, that Jacob caught after his immune system was reduced to shreds by regular sleeplessness caused by his kid's iterative flus, family stress, excess alcohol (caused by immediate and extended family stress), and the never ending procession of viruses the kid kept bringing home. For all that saying that family is so that you don't die alone, Jacob never wanted his family for that he could die surrounded by them, and Felix is pretty sure Jacob would have preferred to die alone rather than suffer so much and give so much suffering to those he loved.

Felix thinks often of the urn of ashes that is all that is left of Jacob, and about how the family Jacob loved has been left with no choice but to forget and to keep going, one feet in front of another, after the husband that did everything was eaten alive by a metastasis. Felix knows in his bones that he will find the time to compose and sing a song for Jacob. It'll probably be called "Sex toys are underrated."

I came here to say that. Though not in a bad way; the economics of what is possible but practical/impractical changes greatly with LLMs, and I suspect and hope we will be figuring out the ramifications for a while.

I personally have been trying to get a financial security cushion for the last ten years or so (and failing) so I don't take my own feelings into account; AI hasn't made me more frustrated. On the contrary, it's a tool I now use in my escape-the-rat-wheel adventures, and there's nothing sweeter than seeing Opus dismantle in ten minutes a problem that would have consumed two full days of my time in this world. But when I speak with colleagues and friends, all of them, now and not before, in all age groups, and with only in common to be knowledge workers, are searching for a way out.

Hmm. Forget about desktops. I'm programming a microcontroller which is actually reasonably priced and has gotten all its 32Mb of RAM in the same die as the processor, but I'm getting nervous and having a strong urge to stockpile...

> Both premises seem laughably unlikely to me.

Yeah, that's not gonna happen.

> The AGI-pilled must think one of two things;

I'm cynical and I believe that AI will be our undoing. In fact, when we let that genie out of the bottle, many other scenarios became possible. For example, here's one third scenario to your "two things": a sufficiently cynical[^1] and resourceful rogue AI may solve climate change by steamrolling over the "set of political, social, and systemic reasons" and teaching the monkeys a lesson about the real significance of very vast agency and intelligence. For example, it could manipulate people into an anti technological uprising and subsequent backward, anti-humanist ethos, while at the same time modifying one or two key ecosystems so that the Mediterranean would smell of cooked cabbage (dimethyl sulfide) and bring about a North African monsoon that, with sufficient time, would turn the Sahara into carbon-sequestering dense jungle...

[^1]: It takes one to know one.

I think the guy is right on point at least when it comes to therapies. But he is also missing one major angle: we are no longer in control; our systems are. If we were to vote on the issue, I suspect that a narrow majority would prefer cheaper housing, and a total majority would prefer cancer to be cured. But there's no direct mechanism of any kind to exercise that vote. Not in USA, not in Europe, not in China. When/where there's democracy, nobody runs in a campaign of advancing immune therapies[^1]. The closest proxy we have to a vote is "regulated markets", and my recent experience losing a loved one to cancer taught me that cancer is not a disease for the streams of money moving through that system. Far from it.

I'm frankly pessimist that we will be able to fix those systems in a humanist way, at least not in my lifetime. If we were any good at it, we wouldn't have extreme inequality, nor our educations systems would be in tatters, and we wouldn't cast ballots propping up politicians whose best skill is ragebait.

However, if fixing the system is not an option, access to cheap technology and intelligence could help a lucky few circumvent those systems. And guess what the lucky few at the head of the AI companies will say then... Not necessarily the worst outcome, since most people will find more viable to equip a private biolab in the basement of their countryside manor than fixing the medical system of first world economies.

[^1] For the record, I think that would be better than running on xenophobic slogans.

Replace "books" with "human brains" and you have a "warehouse" ready for a grim science fiction project. Or, increasingly likely, a prophecy.

I like the "cognitive debt" term. With the latest models, what I've observed is that they are really good, but I don't use them to write main code because I need to know what I'm doing.

The article is not wrong though that it pays off to have some imagination on how to use the models. For example, I want to use SIMD instructions in an ESP32-P4 CPU. Those instructions are undocumented for the most part, with just a couple of handwavey blog posts and some infuriatingly vague marketing material. So I just asked an LLM to create a `SIMD_P4.md` document with all the details. Lo and behold, it practically reverse-engineered the ISA. Now I can program in assembler by hand all I want and build that skill in my own brain, and whenever I find a slightly unclear op in the document, I ask the LLM to refine the documentation in that op.

> What am I missing?

Their marketing department :-) . I'm only half-joking; those guys are hard at work finding the best product-market fit for ChatGPT/Claude. "Product market fit" means "strongest revenue", which is not necessarily going to bring the best tool for you or me, but the one that can either get more consumers to shell off money, or more enterprises to cough money for licenses, and in both cases those consumer basis will be narrowed down to what legal and geopolitical circumstances allow OpenAI and Anthropic (and this is why they want to ban the competition!). It also means dark patterns and enshitification, of which I'm already seeing some both in the Codex interface (it was just renamed "ChatGPT"!!!) and in Claude Code (which also is just "Claude" now and can't '@' properly any longer). So in the medium run most people will be better off running an open source harness that can use any model.

This is AI-writes-too-well is a problem with many solutions. Here's a slightly unconventional one: if an author has enough skin in their book, they could just do as Tolkien and sing their songs[^2], in public. Taking that one step further would be to establish rapport with readers by teaching them how to play those songs, e.g. in a piano or a guitar. It's like going back in time, to when the Iliad was sung and performed instead of written. Though there's no need to negate technology; if a writer is sufficiently technologically inclined, they could forego the linear story and instead release interactive fiction for musical instruments...Hm, I'll see myself out now.

[^2]: https://tolkiengateway.net/wiki/Category:Songs

Hmm, those are some worrying news for Zephyr, but also for Linux. For reasons that escape my full understanding, a lot of embedded hardware needs Linux to work. If Zephyr, which I bet has been carefully crafted for microcontrollers, leaves some unnecessary things on, who knows what even an extremely trimmed down Linux running in, say, the beagle board, gets away with.

I loved the video. Also, I had a lot of fun watching it while imagining I was living in 2051 and this whole progression had ran its course. It's impossible to predict the future, but certain events can negate whole branches of possibility. This one video in particular prunes the "we never figured out robotic motion and thus all the science fiction scenarios built on that premise never came to happen" subtree.

It's a movie for theaters and not a flashback edgy post; the success of the work is about pleasing the audience. That is already a thing in Homer's original Odyssey, at least as I read it in high school. But now with a few more decades on my ribs, I can quickly find out that the text that I read came from oral traditions that spanned from 1200 to 600 BCE, and after finding its way to written form, a cultural chain of transliterations:

Archaic Greece -> Hellenistic Alexandria[^1] -> Roman Egypt -> Byzantine Constantinople -> Renaissance Italy

Each age was obsessed with something, and the latest stages in that chain are heavily marked by Christian values. So, yeah, swords and explosion and stuff and don't do anything we would consider obscene today which includes sex, though movie directors are completely unabashed about bloody horror. To me, it's not a reflection on the director, but on his audience and their society.

[^1]: https://www.getty.edu/art/collection/object/103T4T

A bit tangential, and in the topic of nostalgia for old ways, remember when every block had a photo shop? Couldn’t we get a “PCB shop” in our bright near future? I wonder if the density of enthusiasts that need PCBs made is so low that the entire market is captured by only one or two companies based off China subject to the realities of shipping times.