dilap

The fundamental nature of the model is that it consumes tokens as input and produces token probabilities as output, but there's nothing inherently "predictive" about it -- that's just perspective hangover from the historical development of how LLMs were trained. It is, fundamentally, I think, a general-purpose thinking machine, operating over the inputs and outputs of tokens.

(With this perspective, I can feel my own brain subtly oferring up a panoply of possible responses in a similar way. I can even turn up the temperature on my own brain, making it more likely to decide to say the less-obvious words in response, by having a drink or two.)

(Similarly, mimicry is in humans too a very good learning technique to get started -- kids learning to speak are little parrots, artists just starting out will often copy existing works, etc. Before going on to develop further into their own style.)

That description is really only fair for base models†. Something like Opus 4.6 has all kinds of other training on top of that which teach it behaviors beyond "predict most probable token," like problem-solving and being a good chatbot.

(†And even then is kind of overly-dismissive and underspecified. The "most probable word" is defined over some training data set. So imagine if you train on e.g. mathematicians solving problems... To do a good job at predicting [w/o overfitting] your model will have to in fact get good at thinking like a mathematician. In general "to be able to predict what is likely to happen next" is probably one pretty good definition of intelligence.)

MIT (https://gist.github.com/karpathy/8627fe009c40f57531cb1836010...)

I think you'd be much better served by writing something rough that maintains your own voice!

> The problems aren't bugs; they're consequences of design decisions that may have made sense a decade ago, but don't align with how JavaScript developers write code today.

> I'm not here to disparage the work that came before — I'm here to start a conversation about what can potentially come next.

Terrible LLM-slop style. Is Mr Snell letting an LLM write the article for him or has he just appropriated the style?

Leadership matters!

Well sure, but lots of big companies have all the resources in the world and can't execute. Google really did turn things around in an impressive way.

Haven't used Linux in forever, but middle-click to paste was like the one thing that consistently worked everywhere. So it makes some sense that someone want to break it somewhere. With enough fingers meddling all conventions are broken. Wisdom of the crowds. Democracy.

(Actually I have been playing with Omarchy recently a tiny bit, inspired by the absolutely devestatingly bad macOS update. Initial tire-kicking was very positive. They had a "universal copy-paste" feature that still seemed WIP at the time...i.e. didn't work everywhere...)

It's neat!

Suggestionms:

A version where you actually need to find sets -- "have two, find third" is way too easy.

Monospaced font -- it's too hard visually to see difference in word length right now, w/o counting letters.

I read about 4 paragraphs of the blog post, it does not at all read like it was written by ChatGPT!

Some people are perhaps overly focussed on superficial things like em-dashes. The real tells for ChatGPT writing are more subtle -- a tendency towards hyperboly (it's not A, it's [florid restatment of essentially A] B!), a certain kind of rhythym, and frequently a kind of hard to describe "emptiness" of claims.

(LLMs can write in mang styles, but this is the sort of "kid filling out the essay word count" style you get in chatgpt etc by default.)

Yeah, by design, of course, but I still think it's bad (& there are plenty of ways to scare grandma without lying to her, if you really need to do that).

In general I'd contend that the mindest which leads you to believe "we need to lie to our users because they are dumb" isn't conducive to making good software.

It's basically a "when to rip the band-aid off" type of situation.

Briefly poked around w/ linux again for the first time in years (Omarchy, DHH's tune of Arch + hyprland), and hoo boy, it's come a long way! Nothing like the KDE/Gnome+X jankery of the olden times. Very polished, very slick, very nice.

Saying "Foo.app is damaged" is lying to the user though, which is not nice, and not a good sign, in general, for the health of a company / its culture.

If you find it useful no harm in enjoying it! The main problem with AI content is it's just not good enough...yet. It'll get there. The LLMs just need more real-world feedback incorporated, rather than being the ultimate has-read-everything,-actually-knows-nothing dweeb (a lot of humans are like this too). (You can see the first signs of overcoming this w/ latest models coding skills, which are stronger via RL, I believe.) (Not first hand knowledge tho -- pot kettle black situation there.)

I read the first few paragraphs. Very much reads like LLM slop to me...

E.g., "Zig takes a different path. It reveals complexity—and then gives you the tools to master it."

If we had a reliable oracle, I would happily bet a $K on significant LLM authorship.

I'm a huge K2 fan, it has a personality that feels very distinct from other models (not syccophantic at all), and is quite smart. Also pretty good at creative writing (tho not 100% slop free).

K2 hosted on groq is pretty crazy for intellgence/second. (Low rate limits still, tho.)

They crossed it definitively, and still unbelievably, to me, when they started showing ads as the first result in App Store search. For a long time searching "ChatGPT" in the AppStore would surface a rip-off clone w/ a lookalike icon as the first result. How many thousands of users inadvertently downloaded the clone, paid for it, and were, basically, victims of a scam, facilitated by Apple? (Now the first result for ChatGPT, Claude, Grok is at least the correct first party ad, though this almost seems like extortion on the part of Apple.)

(Software quality has also fallen off a cliff, though that's more a loss of instutional competence, I think, than active anti-user behavior motivated by avarice.)

X has everything, and you can pick what you follow (there's a "For You" tab, but also a strictly chronological following tab). I like it for variety of political views (e.g. super-lefty @caitoz, super-righty @L0m3z), following interesting LLM stuff (@elder_plinius is a great follow), lots of devs (e.g. carmack...), art accounts (@yumenohajime, @neurocolor), nutrition/health stuff, so much good stuff!

(The FYP, alas, sucks, and has since forever...)

Huge fan of X, but it's pissing in the face of your fans to tell such obvious lies.

Light-weight wrapper over C, so as fast as C when you want it to be. Message passing isn't as fast as, say, vtables, but is still quite snappy and flexible for loosely binding objects together. No generics avoids code bloat.

In practice obj-c apps were snappy, e.g., good perf on extremely limited hardware of original iPhone. SwiftUI (I assume) of MacOS settings app much slower than the old version it replaced -- too much heavy programmer framework magic resulting in slower final code? That's my diagnosis/guess from afar (I might be wrong ofc), a pitfall that objc did not tend to lead developers into.

Taste & trade-offs aside, you've gotta make it compile reasonably fast! I do get that Objective-C is not the pinacle of language development, but you shouldn't give your main language the rough edges of a research project.

(And while the past shouldn't necessarily be a shackle on the future, it is striking that such a radically different set of trade-offs was picked for Swift vs Obj-C.)

I think both Go and C# are pretty nice languages, to give you an idea of where I'm coming from. And Rust is very interesting -- as a user you see software that gets written in it exceed the previous state-of-the-art (e.g., ripgrep).

I don't see that w/ Swift. It seems like the opposite. E.g., the terrible Settings rewrite that rolled out a couple releases ago...

Confession, though, while I did a lot of ojbc back in the day, I've never done more than kick the tires on Swift, so I'm not critiquing from a position of deep knowledge -- more like talking shit from the sidelines. But I do think I'm right. ;-)

Swift is an early example of Apple losing its way. Such a stark contrast to Objective-c -- which was a simple, fast compiling language that hit way above its weight for expressivity and runtime speed. A great language for its day. Swift is "a C++ hacker's first attempt at language design".

There is a website forvo.com which has a bunch of community-generated pronunciations of words in a ton of languages. I used to use it a lot when I was playing around w/ learning languages.

There's also a paid API. I made a very basic command-line client which might still work: https://github.com/erinok/forvosay

Forget privacy, imagine what someone like @elder_plinius could get up to if you invited them over to dinner. All of the "AI Safety" issues get a lot more real once the AI's have bodies.

Nah, I don't think so -- it really was a big deal to have a bug back then, and software quality used to be a lot higher. We could go back and run some VMs to try to objectively quantify this (and it would be interesting to do so), but I'm personally confident my memory isn't being tinted by nostalgia.

The main reason is the ability to do constant updates now -- it changes the competitive calculus. Ship fast and fix bugs constantly wins out vs. going slower and having fewer bugs (both in the market & w/in a company "who ships faster?").

When you were shipping software on physical media having a critical bug was a very big deal. Not so anymore.

Besides the bad design, the implementation is awful as well. Slow and flickery.

I think having a world (3d maybe, or maybe just 2d) you could talk about in a really simple way might be useful here. Imagine something like "el gato quiere la pelota roja" and you have to carry the red ball to the cat to pass to the next lesson, and there's a cat, and a dog, and capibara and various shapes; something like that...

There's probably the opportunity to have simple stories and personalities come into play too, early on, to add interest. Think about e.g. the Frog and Toad books for children learning to read.

Playing briefly, looks pretty good! -- though I wonder if there's a way to move away from using a source language (or maybe it does this in later lessons?). You really want to try to get your head 100% inside the target language as quickly as possible, and not be translating back and forth.

Duolingo is terrible†, but proper gamification combined w/ LLMs for real conversations could be an incredible learning tool. (I might build this if no one else does.)

†It can be useful for going from absolute 0 to epsilon, just to kind of get familiar with the language, but if you're using it more than like 2 weeks, you're seriously wasting your time (vs. reading material in the target language, watching TV in target language, trying to talk w/ people in target language). Anki, too, can be a trap that feels like learning but isn't, really, in my experience.

YouTube is incredible, YouTube is poorly run. If I were making the laws, I'd do something similar to mandatory licensing of songs for radio: mandate that YouTube, as a sort de-facto content monopilist, provide third-party access to its database (upload, discovery, view counts, recommendations, etc). Devil is in the details, but well-done it would strictly improve the world.

Independent competitive companies are great, but things tend to devolve into de-facto mini-governments once things stabilize, and from there I think the (real) government using its power to force a little more competition could really improve things.