gwern

Also of interest: https://dbohdan.com/kaur

There's also severe selection effects: what documents have been preserved, printed, and scanned because they turned out to be on the right track towards relativity?

I think my take away is that you are seeing mostly mode-collapse here. There is a high consistency across all of the supposedly different personalities (higher than the naive count would indicate - remember the stochastic nature of responses will inflate the number of 'different' responses, since OP doesn't say anything about sampling a large number of times to get the true response).

Writing or tweaking plugins is great, but it's not a paradigm shift (and risks a lot more toil because now you have to be your own PM or deal with patches/merges, on top of being a reference librarian and copyeditor etc). I feel like if you have a quasi-superintelligence in a box which can run your PKM for you, and you were designing from the ground up with this in mind, that Claude Code is only going to et much better & cheaper, you would not be settling for 'write or modify an Obsidian plugin'. You would get something much different. But 'write a plugin' is basically at 'horseless carriage' level for me.

What I have in mind is something far more radical. There's an idea I am calling 'log-only writing' where you stop editing or rearranging your notes at all, and you switch to pure note taking and stream of conscious braindumping, and you simply have the LLM 'compile' your entire history down into whatever specific artifact you need on demand - whether that's a web of flashcards or a blog post or a long essay or whatever. See https://gwern.net/blog/2024/rss + https://gwern.net/nenex , combined with the LLM reasoning and brainstorming 'offline' using the prompts illustrated by my poems.

No, but I haven't been following the space. (I suspect that with Claude Code-level coding agents, you should be able to do something amazing that thoroughly obsoletes Obsidian/Roam/org-mode, but I don't actually know of anything.)

I've been focused on creative writing, with poetry as my test case, to see what the bottlenecks are to truly amplifying myself through LLMs (as opposed to helping my boss automate away my job or spamming the Internet more efficiently).

I find that frontier LLMs are now there and now I can prompt for genuinely good poetry with LLMs. See https://hollisrobbinsanecdotal.substack.com/p/llm-poetry-and... / https://gwern.net/fiction/lab-animals and https://gwern.net/blog/2025/better-llm-writing

So maybe this year I can turn some attention back to PKMs and Quantified Self stuff...

Yes, and PKMs in general. Like labeling your emails by topic in Gmail. The problem is that the 'toil' keeps piling up, while the value gained is increasingly hard to see.

I have a little rant about it - "‘Tools for thought’ winds up being a lie: there’s tools, but not much additional thought." https://gwern.net/blog/2024/tools-for-thought-failure https://www.lesswrong.com/posts/CoqFpaorNHsWxRzvz/what-comes...

(My answer, of course, is that almost all of this scutwork is well within the capabilities of a frontier LLM today. We just need to apply them.)

The conclusion here seems largely unjustified by the data and indeed is difficult to relate to simple distributions or statistics:

> Increasingly, public institutions seem to exist to manage the obsessions of a tiny number of neurotic—and possibly malicious—complainers.

Why would anyone complain about airport noise when it is ~100% guaranteed to do them no good, and almost all the benefits go to everyone else even if it somehow did anything? Just thinking like an economist here... (Indeed, if a large fraction of locals did complain about something like airport noise, that would itself be highly suspicious to me - as it would indicate an organized campaign or an issue which has become politicized in some way and is now a pretext for something else entirely like a culture war.)

And if there is something I've learned about design and problems, it's that you can have a huge problem, and you are lucky if even 1% will ever tell you.

Your website could be down, and if even 1 person takes the risk of going out of their way to tell you, you should thank your lucky stars that you have such proactive, public-spirited readers!

See also: "Theory of the Nudnik: The Future of Consumer Activism and What We Can Do to Stop It" https://gwern.net/doc/economics/2020-arbel.pdf , Arbel & Shapira 2020; https://pointersgonewild.com/2019/11/02/they-might-never-tel... (commentary: https://gwern.net/ref/chevalier-boisvert-2019); https://en.wikipedia.org/wiki/1%25_rule

> is not related to predicting what is possible with greater scale.

If they can't predict new emergence, then 'explaining' old emergence by post hoc prediction with bizarre newly-invented metrics would seem to be irrelevant and just epicycles. You can always bend a line as you wish in curve-fitting by adding some parameters.

> I thought people recognized that they don't appear out of nowhere.

I don't think that paper is widely accepted. Have you seen the authors of that paper, or anyone else, use it to successfully predict (rather than postdict) anything?

1. You put a lot of time into an intern or a junior too. 2. I didn't say 'paid', I said, 'fully loaded [total] cost'. The total cost of them goes far beyond their mere salary - the search process like all of the interviews for all candidates, onboarding, HR, taxes etc.

> So he's spent $51k on tokens in 3 months to build what exactly? Tools to enable you to spend more money on tokens?

Sounds like that's more than a junior or an intern that would have cost twice as much in fully loaded cost.

Or the cherries could be a delicious pastry or PBJ-like treat: _Collect Horse Buttery Stable_...

(Cerebras is real hardware.)

Could you explain why you think that? I'm looking at the lottery ticket section and it seems like he doesn't disown it; the reason he gives, via Abhinav, for not pursuing it at his commercial job is just that that kind of sparsity is not hardware friendly (except with Cerebras). "It doesn't provide a speedup for normal commercial workloads on normal commercial GPUs and that's why I'm not following it up at my commercial job and don't want to talk about it" seems pretty far from "disowning the lottery ticket hypothesis [as wrong or false]".

> which while describes a curve that shows the same behavior, it's more "accurate" of the actual post creation.

I would say that this graph looks a lot more extreme, actually!

I am guessing you are on mobile and are referring to the togglebar 'demo mode', illustrating its existence.

I wish we didn't have that, but we have to. That demo of the theme togglebar exists to educate users, because we found that a lot of complaints came from people who were blind to the gear icon and were unaware that the control they wanted already existed. (Which is regrettable but understandable, because almost all websites provide useless controls or visual spam, in an example of 'why we can't have nice things'.)

Obviously, it's disabled on subsequent page loads. (One of a number of parts of the UI/UX we streamline using 'demo-mode', as we try to thread the Scylla & Charybdis of a cluttered but explicit UI vs a clean newbie-unfriendly UI.)

To clarify the point here for people who didn't read OP: the oral exams here are customized and tailored to the student's individual unique project, that's the point and why they are not written:

> In our new "AI/ML Product Management" class, the "pre-case" submissions (short assignments meant to prepare students for class discussion) were looking suspiciously good. Not "strong student" good. More like "this reads like a McKinsey memo that went through three rounds of editing," good...Many students who had submitted thoughtful, well-structured work could not explain basic choices in their own submission after two follow-up questions. Some could not participate at all...Oral exams are a natural response. They force real-time reasoning, application to novel prompts, and defense of actual decisions. The problem? Oral exams are a logistical nightmare. You cannot run them for a large class without turning the final exam period into a month-long hostage situation.

Written exams do not do the same thing. You can't say 'just do a written exam'. So sure, the students may prefer them, but so what? That's apples and oranges.

Note: this is different from https://blog.glyphdrawing.club/font-with-built-in-syntax-hig... https://news.ycombinator.com/item?id=41245159 which recently resurfaced, and is cited as prior art.

So if it's not using attention and it processes the entire input into an embedding to process in one go, I guess this is neither a Transformer nor a RNN but just a MLP?

Note: from May 2025. https://gwern.net/doc/genetics/editing/2025-musunuru.pdf

Isn't even thoughtful either.

> The question was never “how do we align AI with human values?” The question was always “which humans get to define those values?” Grok answered that question: the ones with the most money.

Grok is routinely misaligned with Elon, as the article points out in its intro! You don't need to order your engineers to keep fixing what isn't broken...

> I would also welcome someone writing a short takedown where they fix the prompts and get better-than-2022 results from nbp

NBP (and the new ChatGPT generator) are integrated with LLMs to various degrees, so seems like the obvious starting point is a reverse approach: ask them to describe the old images which has the esthetics that Fernando Borretti likes, and start generating from those prompts. If you can recover the old images, then it was just a prompting issue. ("Sampling can show the presence of knowledge but not the absence.") If you can't even with their own 'native' descriptions, then that points to mode-collapse (especially all of the 'esthetic tuning' like DPO everyone does now) as being the biggest problem.

Penn & Teller formulation: https://www.smithsonianmag.com/arts-culture/teller-reveals-h...

There's a vein of research which interprets self-attention as a kind of gradient descent and says that LLMs have essentially pre-solved indefinitely large 'families' or 'classes' of tasks, and the 'learning' they do at runtime is simply gradient descent (possibly Newton) using the 'observations' to figure out which pre-solved instance they are now encountering; this explains why they fail in such strange ways, especially in agentic scenarios - because if the true task is not inside those pre-learned classes, no amount of additional descent can find it after you've found the 'closest' pre-learned task to the true task. (Some links: https://gwern.net/doc/ai/nn/transformer/attention/meta-desce... )

I wonder if this can be interpreted as consistent with that 'meta-learned descent' PoV? If the system is fixed and is just cycling through fixed strategies, that is what you'd expect from that: the descent will thrash around the nearest pre-learned tasks but won't change the overall system or create new solved tasks.

There's also the efficiency argument from new capability: even a tiny bit better weather forecast is highly economically valuable (and saves a lot of wasted energy) if it means that 1 city doesn't have to evacuate because of an erroneous hurricane forecast, say. But how much would it cost to do that with the rivals? I don't know but I would guess quite a lot.

And one of the biggest ironies of AI scaling is that where scaling succeeds the most in improving efficiency, we realize it the least, because we don't even think of it as an option. An example: a Transformer (or RNN) is not the only way to predict text. We have scaling laws for n-grams and text perplexity (most famously, from Jeff Dean et al at Google back in the 2000s), so you can actually ask the question, 'how much would I have to scale up n-grams to achieve the necessary perplexity for a useful code writer competitive with Claude Code, say?' This is a perfectly reasonable, well-defined question, as high-order n-grams could in theory write code without enough data and big enough lookup tables, and so it can be answered. The answer will look something like 'if we turned the whole earth into computronium, it still wouldn't be remotely enough'. The efficiency ratio is not 10:1 or 100:1 but closer to ∞:1. The efficiency gain is so big no one even thinks of it as an efficiency gain, because you just couldn't do it before using AI! You would have humans do it, or not do it at all.

No, the search space is tiny: you can just attack 1 BPE at a time! Stuff like password guessing is almost trivial when you get to do a timing attack on each successive character. So that lets you quickly exfiltrate arbitrary numbers of prompts, especially if you have any idea what you are looking for. (Note that a lot of prompts are already public information, or you can already exfiltrate prompts quite easily from services and start attacking from there...)

While we're at it: my own work 2 years ago in creating an entire workflow for turning Midjourney or DALL-E dropcaps into attractive, lightweight, easy-to-create dropcaps for web pages: https://gwern.net/dropcap We use it for the cat, Gene Wolfe, and holiday pages.

No. I think the need for adversarial losses in order to distill diffusion models into one-step forward passes has provided additional evidence that GANs were much more viable than diffusimaxis loudly insisted.

(Although I'm not really current on where image generation is these days or who is using GAN-like approaches under the hood or what are the current theoretical understandings of GAN vs AR vs diffusion, so if you have some specific reason I should have "caved", feel free to mention it - I may well just be unaware of it.)

This works surprisingly well. If you look into enough dark corners of Unicode, it turns out that you can do a shocking amount of typography, going far beyond the obvious italics and bolds: https://gwern.net/utext

In fact, I found that writing as much math as possible in Unicode makes for the best HTML reading experience: it's fast, simple, and looks more natural (avoids font inconsistency and line-height jagginess, among other things). https://gwern.net/design-graveyard#mathjax

And if you find writing Unicode yourself a pain, you can just ask a LLM to translate from LaTeX to Unicode! https://github.com/gwern/gwern.net/blob/master/build/latex2u...

> but I think fixing the verbal tics is still important and useful.

I don't. I think they're useful for flagging the existence of mode-collapse and also providing convenient tracers for AI-written prose. Erasing only the verbal tics with the equivalent of 's/ - /; /g' (look ma! no more 4o em dashes!) is about the worst solution you could come up with and if adopted would lead to a kind of global gaslighting. The equivalent of a vaccine for COVID which only suppresses coughing but doesn't change R, or fixing a compiler warning by disabling the check.

If you wanted to do useful research here, you'd be doing the opposite. You'd be figuring out how to make the verbal expressions even more sensitive to the underlying mode-collapse, to help research into fixing it and raising awareness. (This would be useful even on the released models, to more precisely quantify their overall mode-collapse, which is poorly captured by existing creative writing benchmarks, I think, and one reason I've had a hard time believing things like Eqbench rankings.)