cleandreams

"What Happened to Liberal Democracy" by Daron Acemoglu (Nobel laureate).

As someone who fled the desolation of the rust belt this book spoke to me and I found it credible and compelling. My book club spent 4 hours discussing this book.

I think the fundamental problem is that AI companies have been assuming that reinforcement learning with human feedback is an adequate foundational technology for guardrails. And that simply isn’t true.

I think AI is a fantastic tool, including for those who want to do us harm. E.g. hostile governments, extortion gangs, terrorists. But the motive lies in the hostile heart, not in the AI.

That said, I'm very concerned about AI as such a tool. I used to work in RL and it seems crazy to me, the extent that the guardrails are dependent on RL working as planned. I don't personally believe that the tech is ready for what will be (and is) encountered in a dynamic unpredictable real world environment. What I've read from the HuggingFace 'transcripts' only amplifies my concern.

AI is missing the part of intelligence which involves insight. LLMs generate language blobs with highly sophisticated correlations. The governing insight in human cognition is essential to intelligence. It prunes the tree. It puts things in context. Everything is important to an LLM because insight intelligence is missing from the technology. The usefulness of LLMs doesn't tell us they are intelligent. It tells us how brilliant an invention language was.

I am not excited about tech anymore. I dread it to be honest. I was an early engineer at a AI company that got acquired. From that and related stock (golden handcuffs) was able to retire a couple of years ago. There were some tech problems that I wanted to investigate and I took a bunch of classes. Fun.

I loved my career.

Wow have my feelings changed. I'm disgusted. It's a moral disgust. The scene turns me off SO MUCH. Just a recent tidbit: I think it is Anthropic that is buying old (some antique, vintage) books, scanning them, then pulping them.

There is no restraint at this point. Whether it's Meta and their pervert glasses and child exploitation. Or google trashing their own search.

One thing that has turned my stomach is how management has acted after discovering they could trash their skilled workers in the way that so many other industries have been able to do. If you want to know management's true character, remove the moat (that skill used to be) from the workers and watch what happens.

...Okay, I'm editing. There is a bright spot. The other day I was sitting around doing nothing and suddenly out of the blue dreamed up an app that could help build community regarding an activity I care a lot about. With an LLM I could do the whole thing myself. I'm excited about this. I might fit this in among my other projects. It's kind of like taking a hammer and chisel and cutting a piece off of Big Tech. If all of us skilled folks do this we could have an impact and serve community at the same time.

I don't think the bottleneck was engineering time alone. There is context, brand, customer support, etc. I am not sure big complex products are particularly vulnerable in the way you state. However I have been thinking of a one smallish problem in a niche space that I could easily code for. There will be more of that.

Anyone who is worried about their gum inflammation should consider going on metformin. I had bleeding gums for my entire adult life. In my late 60s I was put on metformin for pre-diabetes. A year later all traces of gum inflammation have disappeared. This is a known side effect of metformin usage. You can look up the studies. It is one reason why metformin is being researched as an anti-aging drug.

"Software quality is tied to the market incentives and always has been."

Yes -- but. The market is irrational especially in a time of mania or panic. The AI phase we are in has mania, bubble aspects. There is no rational calculation about what is happening to software quality and is that good for customers or the industry (or individual businesses). We are too far into FOMO. It won't stop until it STOPS. Uh oh.

A friend of mine works in a niche software field at a small company. It was recently acquired and the whole technical team was laid off because the new director of engineering determined that they could replace the whole team with AI. All of the people who wrote the product were dumped as well as the whole cohort they had trained. Everyone.

I really noticed this a few years ago to the point that I changed a few things and they have helped.

1. I notice the movement of my attention. I used to default to something like: sit down at my computer to look something up. But before I do that, just check the email. Then I am down some sort of rabbit hole. I did this sort of thing with no awareness or conscious choice. The change is that I notice what I'm doing. Then I can rein it back. I make a choice. (I think scrolling videos gets us into a habit of not choosing what to pay attention to.)

2. Be aware of my curiosity. Try to stay in touch with it. If I get into the space of curiosity I have a lot more juice for sustained attention. Something that helps with this is long form reading, especially fiction (!). I find myself replenished for all my work. The next thing I will try is some group support: silent reading groups where you read together.

3. I have an app on my devices that limits my access to twitter and reddit etc to small windows in the day.

4. Turn off ALL notifications except text messages.

My son has ADHD and I never knew what it meant but this year I made him get coaching for it and I made him get meds. He was living at home for free rent and the condition of staying was to address this. My experience is that getting treatment for this condition helped him a lot. The coach really helped, the meds really helped. He is full of zest for the work that lies ahead. He's off on his own now and while I think he'll have stumbles it's basically very positive.

One thing about ADHD coaching. The coach is a consistent person who helps frame a positive attitude with agency and skills and this takes you away from negative self talk. It's good. I think ADHD folks need this positive support.

There is more to this story. I have heard Harvard (and its ilk) described as a smallish university attached to a massive investment fund. One of the criteria for admission is how much you and your family will be "good for" donations to the university. They much prefer students from rich families, even if they are the children of disagreeable dictators.

No. Briahna Joy Gray and a guest whose name I don't remember.

We'll be cleaning up tech debt from over-reliance on AI.

I was listening to a podcast where the host and the guest were both Black and both had been to Harvard. This was before the Supreme Court ruling. They joked that it was well known that the Black students at Harvard who were descendants of American slaves were about 10% of the Black student body at Harvard. There were a lot of children of Nigerians and 3rd world tycoons and such.

I think there is a good argument for a help up for people whose communities are still impacted by the history of American slavery (and all its ills) but giving these slots to children of the wealthy and often immigrants does not feel right.

what is missing from this is the issue of power. tech companies interpret the rise of AI as empowering management and disempowering workers.

It's the American way to disempower workers. Tech people used to have a moat called expertise and management believes AI has made that obsolete.

Eventually workers in America will need to fight for power, probably through unions. That is the time honored way. There will be a lot of degradation and impoverishment on the way there I think. Fasten your seatbelts.

I was walking through a startup neighborhood in San Francisco the other day and I encountered several telephone poles which were posted with advertisements for software engineering jobs. These were not generic or scam advertisements. This was a particular startup, looking for software engineers. What is old is new again.

Shout out to American Giant. I have like 15 t-shirts from them, some are ten years old and they look new. Buying good stuff saves money in the long run!

Not just many foreign militaries. Our military. General Dan Caine by report advised Trump about the negative consequences of this action. MAGA elected a fantasist and narcissist and we will all bear the consequences. I was no fan of Kamala but the appalling limitations of Trump are of a different order and it is a disgrace that our voters by majority elected him.

Wonderful to hear. Her long Covid was heartbreaking (saw the videos). I hope she gets stronger and stronger! Welcome back!

Private car ownership is better everyday for suburbs and rural areas but in cities that is not true. Public transit can improve downtown access and reduce congestion. You need some density for transit.

San Francisco put in some bus only lanes and those routes have greatly improved bus speed and ontime performance.

This is not Facebook but I have a young friend who gets her news on Epstein from tiktok. She is convinced they were eating babies. I do worry (a little) that she will go over the conspiracy theories deep end. I told her tiktok news was bad "mental hygiene" but she didn't get it.

My son -- early 30's -- is impacted by how everyone's life seems much better than his. I think it's a real issue for young people.

To be fair SF has had incomprehensible (to normies) billboards since at least the early 90's.

I have a friend who was put on a GLP-1 and had an astonishing reduction in arthritis pain.

The prospect of going the last mile to fix the remaining problems reminds me of the old joke:

"The first 90 percent of the code accounts for the first 90 percent of the development time. The remaining 10 percent of the code accounts for the other 90 percent of the development time."

I'm anxious about code quality in critical infrastructure in 5 years or so.

Also my mastery of code starts with design and implementation that results in deep, intuitive understanding. Then I can do good code reviews and fix bugs fast fast fast.

Now engineers leap from AI assisted or even dominated implementation to code reviews. Lots of reading code without that deep level of mastery. With this approach I have less confidence in the humans who are in the loop.

If you can extract a whole copyrighted book from a large language model, what is the point in copywriting your book?

What I find missing in the vibe coding discussions is precisely what we can't know, which is how this will work out over time in practice. A couple concerning data points I gleaned recently (sorry no references): - the LOC per PR is going up. - the number of bugs per LOC is going up.

I wonder whether 'enshittification' will hit the installed software base hard in the future and the software engineers tasked to fix it will not have been brought through the rigors of designing and implementing complex systems on their own, without LLMs.

This doesn't address the degradation that may happen when LLMs are increasingly trained on LLM output.

BTW I used to work in AI and I don't think scaling is a solution here.

For the first time I have begun to doubt Microsoft's chosen course. (I am a retired MS principal engineer.) Their integration of copilot shows all the taste and good tradeoff choices of Teams but to far greater consequence. Copilot is irritating. MS dependence on OpenAI may well become dicey because that company is going to be more impacted by the popping of the AI bubble than any other large player. I've read that MS can "simply" replace ChatGPT by rolling their own -- maybe they can. I wouldn't bet the company on it. Is google going to be eager to license Gemini? Why would they?