peterlk

Yep. I’ve found that having an endpoint that serves a well, documented openapi.yaml is very effective for agentic usage. The biggest difference is that you can break down a REST API into RPC-like chunks and save on some tokens if you break up the tools well. But pragmatically, I think saying “tell your agent to hit /api/v3/openapi.yaml” is quite useful

I wonder how similar the 2000s-era napster fight will look when we get a high quality music diffusion model with 0 copyright guardrails. Then I can listen to unlimited, radio of my favorite artists serenading me with songs I’d wished they’d written

It’s been a while, but when I was in high school, I was up at 6:30 so that I could run through the shower, get to 7AM jazz band late, be at school until 4ish, then off to my minimum-ish wage job until it was time to go to evening jazz/symphony/other bands. Home at 8 or 9; bullshit some homework for a passable grade; and then do it again (I was often doing homework during band practices). Weekends were music events.

I did not go to a top tier school, and I did not live in a top tier place. The stories I’ve heard from others indicates to me that my experience was not unique. I wasn’t even the most busy of the people I knew.

This wasn’t every single day, but it was more on days than off days. Contrarily, I had a great time in college because the pressure of classes actually felt like less than I was doing in high school.

Preach! I spent several hours this weekend trying to move my library between computers. So many popups and warnings telling me that I need to do something with no explanation of why. The docs are a mess. The library compatibility is a mess. Every few months they do something that fucks up libraries or makes them incompatible with some hardware. I just refuse updates now.

Rather than trying to make a new, shiny library, they should just release rekordbox 2.0, and then everyone would have a common lexicon to ask whether something is “rekordbox 2 compatible”. But instead we have this questionably compatible library plus as part of old, busted software that tries to brow best you into wrecking your old libraries.

I just use Claude to explain rekordbox to me now, and it is a bit more helpful than forums, though it also often gets confused from the conflicting advice/docs on the internet

I’ve been wondeing if this is a similar phenomenon to plane crashes being reported more when there is a plane crash, thus leading people to believe, incorrectly, that there are more plane crashes. Is it maybe common that recalls happen with relative frequency and it’s just a cost of doing business and they’re routine so not reported on? Or is this company really screwing up badly?

This is what the presumption of innocence is supposed to do. If you have an alibi, then big brother doesn’t get involved. These are cultural problems that we do not need big brother’s benevolence to solve for us. We need better cultural systems of accountability; the problem is that those are _way_ harder to build

I don’t think it matters that much for Google. They need to not fall hopelessly behind, but I don’t think there’s a strong economic reason for Google to burn the kind of capex that the frontier labs are burning. Strategically, I think they’re probably doing better than OpenAI and Anthropic. The Gemini models are open, and they are what researchers are working with (see neuronpedia as an example). Over time, this will give them a strategic advantage for the same reasons that open source wins over proprietary. Meanwhile, OpenAI and Anthropic have massive capex that needs to be returned to investors while their margins are being undercut by Kimi/Deepseek/Qwen. Google can wait around for the coming frontier lab profitability crisis and cruise right on by with their Apple contract and owned data centers to pick up the pieces and exceed the existing frontier labs.

Can you be more specific about what “unbelievably terrible” means?

Nothing is stopping people from doing that. It’s just more inconvenient than other available options. If I were a politician trying to remove private communication, I would first pass something that allows scanning communications, then pass something that allows e2ee messages to be scanned, then make it illegal to use non-scannable e2ee messaging. People could still do it, but now they could be punished if they’re ever caught

Over Christmas, I spent several minutes trying to debug my beeping dashboard - it only seemed to happen sometimes while driving, so stopping didn’t let me figure it out. Eventually I discovered that it was beeping at me because my eyes weren’t on the road enough. Of course, figuring that out required me to take my eyes off the road to figure out which blinking signal was associated with this particular alarm.

Also, being constantly warned that I was speeding in rural areas where the car missed a speed limit sign caused me to start ignoring the speeding alarm within a few hours of driving the car.

I feel like there’s some lesson here in building to the lowest common denominator, and giving people products rather than tools (tools are more dangerous, but more useful), but maybe I’m just grumpy.

Yeah. I would expect that Samsung or Apple’s bundled AI things (whatever those end up being) will be good enough for consumers, and open models will be the things that actually get used in the products of the future because they will allow technical people to alter models in ways that specialize them for whatever use case they’re building for. They will also be the onramp for engineers/data scientists of the future, and thus will have more available, specialized knowledge over time.

The bitter lesson would claim that we only need one model with more data thrown at it, but I’m a bit skeptical that we’ll end up with “the one true way” of building a model, and I think there will be model tradeoffs that we can pick from as the industry matures.

Over the long term, it seems like open models must win out. This feels like it rhymes with the story of operating systems. Despite the enormous financial contributions of Microsoft and Apple, linux still won because control matters over the long term.

I predict that mech interp and things like Neuronpedia will matter more and more over time, and the frontier providers are disincentivized from providing those tools

The chain of “Why?”s here is really baffling to me.

Why do we have measles outbreaks? Because people aren’t vaccinating their kids.

Why aren’t people vaccinating their kids? Because there seems to be a generalized fear of vaccinations and their side effects

Why does this fear exist? I’m at a loss. Is it because people aren’t less educated now than they were 30 years ago? Is it because deadly diseases have fallen out of public consciousness? Was there a coordinated external effort to psyop the American people into believing such things (now I’m starting to feel like a conspiracy theorist)?

None of these reasons seem super compelling to me. The “diseases fell out of public consciousness” feels compelling at first, but why would there not have been some admonishment from grandparents who knew someone who died from polio? That seems like family lore that wouldn’t get lost.

Why not run a publicly funded ad campaign with images that look like cigarette packaging? Remind people what these now-preventable diseases look like.

It just seems wild to me that this issue seems to be getting worse. It’s like watching car crashes rise while people are cutting their own brake lines and then continuing to cut brake lines (and encourage others to di so) after their friends die. Wtf is even happening?

Yes to diffusion models! Combo pipelines of generative and diffusion models have super interesting potential

AI output reads like homepage marketing content (e.g. the text that fades in when you scroll down an apple product page) expanded to fill some context window size (the paragraphs tend to be about the same size).

What you get is vacuous, choppy, wordy, and hyperbolic. I have found that adding secondary passes for tone and style improve the readability dramatically.

You need to have an Apple developer account. Then you need to submit your app to Apple for review. Then you need to comply with a list of sometimes arbitrary corrections/requirements that they send back (there is a document that specifies what you need to do, but it is not uniformly enforced in my experience). Then, eventually, you can list your app on the app store.

It’s not super onerous, but it is much more annoying than the theoretical alternative of allowing people to install software of their choosing on their hardware (i.e. download the binary and run it)

Any sufficiently specified prompt is indistinguishable from code

I am also not in the market for a Ferrari. The problem is that it looks so pedestrian. Personally, I think the Ioniq has more personality. For a 600k car, it should have some appeal. This just looks like every other EV; it’s generic and boring.

I think they might have had much more success with a strategy like the R32 EV. Take something classic (like the Testarossa) and electrify it. Remind people that EVs are an evolution rather than a capitulation to generic boringness.

Yes they do…? Who cares if they just predict the next token? The outcome is that they can invent new abstractions. You could claim that the invention of this new idea is a combination of an LLM and a harness, but that combination can solve logic puzzles and invent abstractions. If a really large spinning wheel could invent proofs that were previously unsolved, that would be a wildly amazing spinning wheel. I view LLMs similarly. It is just fancy autocomplete, but look what we can do with it!

Said differently, what is prediction but composition projected forward through time/ideas?

This is not a very compelling argument. Things already cost money. We wouldn’t oppose a water tax because we were worried people might refuse to hydrate themselves once water was marginally more expensive. It might marginally exacerbate an existing problem, but the benefit of solving the target problem (funding roads fairly), even if imperfectly, is a much greater good

Bluesky, threads, mastodon, and everything else built on activitypub AT, etc. are still there. You can leave X behind; the only thing stopping you is the other people who could also leave X but are still there because you’re there. There are real problems with the fediverse, but they are solvable and the biggest problem is the social connections/stickiness. So start with that!

I think everyone should avoid talking about consciousness unless someone in the conversation provides a clear definition of it. If no one provides a definition, we can replace the word “consciousness” with the word “spirit”, and basically nothing about the conversation would change. Without a definition, every conversation about AI consciousness devolves into one camp saying that humans are special and consciousness is unique to them, and another camp that waves their hands about consciousness “duck typing”.

For example, we could define consciousness as the ability to communicate claimed internal states. Perhaps there could be a complexity metric that gives us a metric of consciousness.

We could define consciousness as the ability to respond to stimuli in complex ways. This would make a supermarket’s automatic doors slightly conscious.

Personally, I don’t really care how it is defined in any particular conversation, so long as it is defined. Otherwise we’re just flailing at each other in the dark.

Thanks for numberwang today. Probably my favorite meme for board meetings and all-hands presentations

Reminds me a bit of the coolest talk I ever got to see in person: https://youtu.be/FITJMJjASUs?si=Fx4hmo77A62zHqzy

It’s a derivation of the Y combinator from ruby lambdas

It’s because they’re playing with monopoly money. If you leverage all the money on itself repeatedly, you can make the numbers look insane. Then, you hand pieces of the leveraged money (stock options) to a bunch of executives who will be watched very closely. You don’t sell stock that’s going to go up in value, so they have to be very careful about when and how much they sell. If they sell too much, then the facade can break, and the leverage evaporates.

To be clear, there is real, underlying value & revenue. But there’s a lot of froth right now

My favorites are the micrograd series by Andrej Karpathy on youtube [0], and “Why Deep Learning Works Unreasonably Well” [1]

The greats on youtube are also worth watching: 3B1B, numberphile, etc.

[0] https://youtube.com/playlist?list=PLAqhIrjkxbuWI23v9cThsA9Gv... [1] https://youtu.be/qx7hirqgfuU?si=8zmrbazuvnz379gk

Attention is all you need…?

The short answer, as far as I’m aware, is that no one really knows. The longer answer is that we have a lot of partial answers that, in my mind, basically boil down to: model architectures draw a walk through the high dimensional vector space of concepts, and we’ve tuned them to land on the right answer. The fact that they do so consistently says something about how we encode logic in language and the effectiveness of these embedding/latent spaces.

Modern AI is a miracle. The math that makes it work is beautiful and really impressive. For example, if you wanted to map all knowledge on earth, how would you do it? AI answers that question by building a high dimensional vector space of embeddings, and traversing that space moves you through a topology of basically every concept that humans have.

Or another thought; why is it that a stochastic parrot can solve logic puzzles consistently and accurately? It might not be 100%, but it’s still much better than what you might expect from a markov model of ngrams.

Openclaw is only sort of interesting. How to vibe code your first product is uninteresting. Claims about productivity increase from model usage are speculative and uninteresting. Endless think pieces on the effects of AI slop are uninteresting. There’s a lot of hype and grift and bullshit that is downstream of this very interesting technology, and basically none of that is interesting. The cool parts are when you actually open the models up and try to figure out what’s going on.

So no, I’m not bored of talking about AI. I’m not sure I ever will be. My suspicion is that those who are bored of it aren’t digging deep enough. With that said, that will likely only be interesting to people who think math is fun and cool. On the whole, AI is unlikely to affect our lives in proportion to the ink spilled by influencers.

I don’t think this problem is that hard to solve, it just requires political will that doesn’t exist. The solution is to make it the platform’s problem. If the platform doesn’t want to deal with fraud, they don’t get to operate in that jurisdiction. Sue them into submission. If they don’t care about that geography, then there is now a gap in the market for a more local business to fill.