gwern

One thing to note here is that the closure is a costly, credible, verifiable signal of commitment to their principles. Unusual or questionable principles, perhaps - few other Christians feel the need to close on Sunday - but principles nevertheless.

Suppose B&H or Chick-Fil-A suddenly changed their policy to be open, which is easy to verify (just see if you can buy something on that day). What do you think the reaction of their customers would be to this greater convenience and no loss of quality? Would they celebrate? I think the reaction would be highly negative - everyone would panic that they had sold out and the beancounters were now in charge and the quality was going to go downhill while the prices go up.

Precisely because the policy is so economically foolish and is not a marketing stunt (notice no one is clamoring to imitate them even though it's trivial: just close on some day) and would be one of the first things to go under new (ie. normal) management, it proves that they don't care that much about maximizing their profits but other things.

In the time it takes you to do one good vote, the low-taste slopmaxxers can do 10 (or 20...?).

> That came out in 2009, correct? I wonder how much was spent on LLMs up to that point.

Quite a lot. Look back at the size of the teams working on language models at IBM, Microsoft, Google, etc, and think about all the decades of language model research going back to Shannon and quantifying the entropy of English. Or the costs to produce the datasets like the Brown Corpus which were so critical. And keep in mind that a lot of the research and work is not public for language models; stuff like NSA interest is obvious, but do you know what Bob Mercer did before he vanished into the black hole of Renaissance Capital? I recently learned from a great talk (spotted by OP, as it happens) https://gwern.net/doc/psychology/linguistics/bilingual/2013-... that it was language modeling!

I can't give you an exact number, of course, but when you consider the fully-loaded costs of researchers at somewhere like IBM/MS/G is usually at least several hundred thousand dollars a year and how many decades and how many authors there are on papers and how many man-years must've been spent on now-forgotten projects in the 80s and 90s to scale to billions of word corpuses to train the n-gram language models (sometimes requiring clusters), I would have to guess it's at least hundreds of millions cumulative.

> They're also not humble.

Funnily enough, the more grandiose use-cases of LMs actually were envisioned all the way back at the beginning! In fact, there's an incredible science fiction story you've never heard of which takes language models, quite literally, as the route to a Singularity, from 1943. You really have to read it to believe it: "Fifty Million Monkeys", Jones 1943 https://gwern.net/doc/fiction/science-fiction/1943-jones.pdf

> I don't know why CYC went that way.

If you read the whole OP, which I acknowledge is quite a time investment, I think Yuxi makes a good case for why Lenat culturally aimed for the 'boil the ocean' approach and how they refused to do more incremental small easily-benchmarked applications as distractions and encouraging deeply flawed paradigms and how they could maintain it for so long. (Which shouldn't be too much of a surprise. Look how much traction DL critics on HN get, even now.)

The WWII POW camp paper discussed is a lot more interesting and worth reading than this article winds up being: https://gwern.net/doc/economics/1945-radford.pdf

Language models repeatedly delivered practical, real-world economic value at every step of the way from at least n-grams on. (Remember the original 'unreasonable effectiveness of data'?) The applications were humble and weren't like "write all my code for me and then immanentize the eschaton", but they were real things like spelling error detection & correction, text compressors, voice transcription boosters, embeddings for information retrieval, recommenders, knowledge graph creation (ironically enough), machine translation services, etc. In contrast, Yuxi goes through the handful of described Cyc use-cases from their entire history, and it's not impressive.

It can be easily gamed. The users are self-selected, and they have zero incentive to be honest or rigorous or provide good responses. Some have incentives the opposite way. (There was a report of a prediction market user who said they had won a market on Gemini models by manipulating the votes; LMArena swore furiously there had definitely been no manipulation but was conspicuously silent on any details.) And the release of more LMArena responses has shown that a lot of the user ratings are blatantly wrong: either they're basically fraudulent, or LMArena's current users are people whose ratings you should be optimizing against because they are so ignorant, lazy, and superficial.

At this point, when I look at the output from my Gemini-2.5-pro sessions, they are so high quality, and take so long to read, and check, and have an informed opinion on, I just can't trust the slapdash approach of LMArena in assuming that careless driveby maybe-didn't-even-read-the-responses-ain't-no-one-got-time-for-that-nerd-shit ratings mean much of anything. There have been red flags in the past and I've been taking them ever less seriously even as one of many benchmarks since early last year, but maybe this is the biggest backfire yet. And it's only going to get worse. At this rate, without major overhaul, you should take being #1 on LMArena seriously as useful and important news - as a reason to not use a model.

It's past time for LMArena people to sit down and have some thorough reflection on whether it is still worth running at all, and at what point they are doing more harm than good. No benchmark lives forever and it is normal and healthy to shut them down at some point after having been saturated, but some manage to live a lot longer than they should have...

> Interestingly, for models up to 1 billion parameters, the loss actually starts to increase again after reaching a minimum. This might be because as training continues, the model develops more complex, non-linear representations that our simple linear probe can’t capture as well. Or it might be that the model starts to overfit on the training data and loses its more general concept of nullability.

Double descent?

[0]: ...they're having a "beat the tariff sale". 5% off everything.

Preprint: https://www.celiamoore.com/uploads/9/3/2/1/9321973/wakeman_y...

A useful concept here is the 'incompleteness of contracts' (https://en.wikipedia.org/wiki/Incomplete_contracts): essentially, even for the simplest contract, it is impossible to write down an unambiguous set of rules or contract terms covering every possible outcome or disagreement. Contracts can only be starting points. This is similar to the failure of GOFAI to turn the world into symbolic rules: it may be possible in theory, but not in practice. Anytime a principle hires an agent, they have to leave a lot up to the agent, which is why we have principle-agent problems. So what does that imply for any 'rule'? Well, that it has to be wrong and destructive some of the time. (See also: "work-to-rule strike" or sabotage.) This will be especially true for rules imposed from the outside: if it's impossible for individuals or organizations given a free hand to make perfect rules that should always be followed, how on earth is some third party, like the dead hand of a regulation from a century ago, going to do so? So you want your agents to break the rules... but only, of course, when it's a good thing to break the rules. How do you know that? Well, if it was easy to do so cheaply, you probably wouldn't have the rule in the first place! So in practice, a lot of what happens is just this: if your agent breaks the rules periodically and nothing bad happens, then they probably were doing the right thing; and if your organization encourages the right amount of rulebreaking, it will slightly tend to do better than the ones which don't.

Of course, sometimes your agents weren't doing the right wrong things, just the wrong wrong things, and you just normalized their deviance which is going to explode in someone's face eventually. That is of course true. But if you cherrypick just a few sensational cases and treat them as if that was what supervisors were trying to encourage, you implicitly are denying that the lubricant of the world is often well-chosen rule-breaking.

> the site design was and how clearly communicated the problem they were trying to solve

I don't agree with either. Even after I enabled JS (no warning) and then after reading the whole page, finally realized that the implementation of popins was completely broken on Firefox and switched to Chrome to reread it (it doesn't help that the first 'link' is not a link†, and the link says it's 'broken' but it means broken in a different way from being actually broken so when you click on it and nothing happens, you infer that nothing was supposed to happen, which is why you were told it was broken...), I still couldn't understand WTF the problem was or how any of this could be remotely justified compared to an ordinary ToC and section headers or anchors.

† I'll just note that I have looked at many, many sidenote implementations (https://gwern.net/sidenotes) and the choice to make your sidenote/footnote link look exactly like a regular link is an... interesting choice.

That's unfortunate about Mr Zadunaisky, but it does suggest there's no hope of a recording unless someone should stumble across it in the Borges papers (although given how his estate has been abused, little hope of that being useful, one way or another) or the Argentine library archives...

Anyway, I uploaded the Simon book chapter at https://gwern.net/doc/borges/1996-simon-2.pdf

I see no hint of an audio recording having been made, much less surviving & digitized, in the original article's description: https://www.cambridge.org/core/journals/journal-of-the-histo... It's too detailed to be retroactive notes by Simon or Borges, so I would guess Borges's secretary or a student simply transcribed it as they went.

Yes, this has been done in many forms with other algorithms. You score each generation with a model like CLIP, for example, and then you can evolve 'Mona Lisa made of triangles', say. A constraint like 'exactly 13 circles' will work fine. (And you might experiment with loosening it, like generating a lot of candidates with 5-30 circles each, as a 'library' or 'seeds', before shrinking them all towards 13, to see if you get novel animal designs which are find to find if you simply start the obvious way with 13 circles initialized to random points & sizes.)

Suggestion: change the 'missing teeth' form line to include an ordinal count or something. I was surprised to learn when filing my form that I had "Missing teeth: 30"!

The narrow emphasis on autobiographical content we see today in art is really quite strange. You'll go on Reddit or Twitter and you'll see people insisting that poetry exists only to convey autobiographical content, and so good AI poetry is definitionally impossible. And yet, nothing could be more ahistorical: almost all of the most beloved art throughout history, especially fictional, and especially poetry, lyrics, and novels, is not eyewitness, nor did people find that a flaw or even worth remarking on. How many of Shakespeare's plays are autobiographical? (Is _Romeo and Juliet_ a bad play because Shakespeare did not in fact know anyone in a lover's suicide pact, and simply stole the plot wholesale from a poem? A translated one, at that!) Did Homer ever visit Troy? Did Murasaki Shikibu know a Prince Genji? Did Vince Gilligan have a sideline as a meth cook somewhere in New York during film school before going to Hollywood? (How many Christians are upset that it is, say, the 'Gospel according to Mark', instead of the 'Gospel by Mark'?)

It's amusing how people demand that their fiction be nonfiction. Here's an example I saw the other day, looking up that song which is everywhere, even at the end of random Japanese TV series, because it's so beloved: https://en.wikipedia.org/wiki/September_(Earth,_Wind_%26_Fir...

> Although several theories about the significance of the date have been suggested, the songwriter Maurice White claimed he simply chose the 21st due to how it sounded when sung.[10] Allee Willis adding : "We went through all the dates: 'Do you remember the first, the second, the third, the fourth ... ' and the one that just felt the best was the 21st," Willis explains. "I constantly have people coming up to me and they get so excited to know what the significance was. And there is no significance beyond it just sang better than any of the other dates. So ... sorry!".[11]

Imagine that: writing a fictional song lyric because it... sounded good, rather than because something happened. (Apparently people can't imagine that, and imagine it had to have not been imagined.)

The Gemini models have become notorious for being rather uncensored and unrestricted, with an oddly base-model-like feel, since at least last November.

https://pagedout.institute/download/PagedOut_006.pdf#page=3 to skip to the ToC.

Sounds like they only evaluated GPT-4o and weaker LLMs like mid-last year?

> It should be noted that the other xAI shareholders are/were supportive of the X takeover and all.

No xAI shareholder has any incentive to be honest about that. If they criticize it, Musk will cut them out of all future investments. And if they are an xAI shareholder, criticism may well be grounds for seizing their equity.

Particularly if they are an xAI employee: Musk companies usually have a equity contract & NDA which says that your options can be canceled at any time for $0, they can be bought back even if vested, you are not allowed to sell without their permission while they are private (ie. forever), and you are gagged about all this: https://techcrunch.com/2024/03/15/spacex-employee-stock-sale... (Similar to Altman's OpenAI NDAs and equity agreements that OAers were unpleasantly surprised to discover back around the coup.)

If nothing else, you want to do this merger after the check clears from some new investors, for two reasons: it provides a high valuation you can use to minimize the wasted equity, and because it is highly dilutive of X.ai shareholders to transfer a lot of their equity to a Twitter which is not remotely worth $44b, you want all shareholders in before you soak them in the merger, rather than after, when they can bail or demand revised terms which reflect the actual low value of Twitter now dragging down X.ai.

On the bright side, Google's new Gemini-2.5-pro comes with no Holocaust-denying issues, and is definitely a lot better than Grok, and is also free and fast.

(I've been using it the past 2 days and am pretty impressed - the first high-end Google model I'd willingly use instead of GPT-4.5 or Claude-3.7.)

> Exactly. So then why draw conclusions about how SolarCity was a bad deal for shareholders (see parent comments)?

You can absolutely draw adverse inferences from Tesla's refusal to report certain straightforward, useful numbers. For example, their refusal to publish any real safety statistics, to the point where Elon Musk himself has to invoke rubbish numbers like the 'community tracker'.

If, after all the lawsuits and accusations of failure, and the reporting about the near-zero deliveries and idled factories, the Tesla Energy division is giving you no numbers on the solar roof, and is lumping them together with a totally different product like batteries, there is one thing and only one thing that that means: the numbers are awful.

Yeah, looks like the subreddit has been taken over completely. You can see that the original founder of the subreddit had resigned shortly before... and his farewell-post has been deleted by the new mods. The other mods all seem to be inactive, and going through one of their histories, I found them talking about being harassed and bullied a relatively few comments before ceasing /r/gifted comments

I brought this up in modmail with them, to see if any of the uncorrupted mods would respond, under their names. The responses denied my claims and mocked me, before banning me, but notably, all of the responses were under the group-mod username: so either none of the old mods were willing to respond under their usernames, or it wasn't the new mods responding. (Hint: it was almost certainly the latter.)

I reported it to the Reddit admins, but have heard nothing back. I don't expect too much because if the worst the new mods are doing is spamming affiliate links, that's relatively tame and far down the priority list.

> All user facing LLMs go through Reinforcement Learning. Contrary to popular belief, RL's _primary_ purpose isn't to "align" them to make them "safe." It's to make them actually usable.

Are you claiming that non-myopic token prediction emerges solely from RL, and if Anthropic does this analysis on Claude before RL training (or if one examines other models where no RLHF was done, such as old GPT-2 checkpoints), none of these advance prediction mechanisms will exist?

It sounds like their unit-tests cover abs(), but they weren't covering all of abs(), and were not reliably triggering the optimized codepath:

> When doing the refactoring, they needed to provide new implementations for every opcode. Someone accidentally turned Math.abs() into the identity function for the super-optimized level. But nobody noticed because it almost never ran — and was right half of the time when it did.

If it never was tested, plain and simple as that, then it couldn't matter that it 'almost never ran' or 'was right half the time'.

So the root problem here is that their test-suite neither exercised all optimized levels appropriately, nor flagged the omission as a fatal problem breaking 100% branch coverage (which for a simple primitive like abs you'd definitely want). This meant that they could break lots of other things too without noticing. OP doesn't discuss if the JS team dealt with it appropriately; one hopes they did.

I recently learned about https://en.wikipedia.org/wiki/Self-verifying_theories which gives you most of multiplication while still being decidable, which is pretty crazy.

I think "birds migrate" is a good contender for one of the most startling "humanity discovered that when? and they thought what instead? And it took what to disprove it?" factoids: https://en.wikipedia.org/wiki/Bird_migration#Historical_view... https://en.wikipedia.org/wiki/Pfeilstorch

It seems so simple and common sense and everyday. It doesn't involve microscopic lifeforms or planets or stars, just... birds. "Where do birds go during the winter? Well, birds are good at flying away from hunters or other bad things. That's why they fly. So maybe they just fly somewhere else. I look up in Fall and I see flights of birds flying in a specific direction. And people in other areas say birds arrive from elsewhere for a while. What else would they do? Turn into little fish and hide in the water all winter, or spontaneously regenerate out of random little clams? Anyway, here's a bird with a big old Central African spear stuck through its whole body who managed to fly here alive anyway which proves that."

You have discovered the Law of Fives: https://en.wikipedia.org/wiki/The_Illuminatus!_Trilogy#Numer... ; as it is written:

> In the Erisian Archives is an old memo from Omar to Mal-2: "I find the Law of Fives to be more and more manifest the harder I look."

"Bye Bye Nerdie", S12E16, 2001 https://en.wikipedia.org/wiki/Bye_Bye_Nerdie : https://www.youtube.com/watch?v=u4rbW96R5GM https://www.reddit.com/r/TheSimpsons/comments/qhy6bi/that_ro...

That's a pretty major deal breaker for OP to leave out of his post touting it as something to build everything in your site on (especially for a tech blog)! Does it at least have a polyfill story? I see no mention of how to make it work on, uh, the other 15% of browsers worldwide, CanIUse is telling me.