gwern

"hyperpigmentation on the eyelids from blue-light exposure" is a super specific query almost definitionally 'bixonimania' which probably brought up the 'bixonimania' poison at the time (the search hits for that query right now in Google are weak and poorly relevant so it would not be hard to outrank them or at least get into the top 50 or so where a retrieval LLM would see them and would followup), and so still an instance of what I mean.

> Either way, that is a feature of the scientific process and is not a given to any online information.

Which does not distinguish it in any way from human errors like a crank or activist etc.

And I don't know, how did we handle false information before on niche topics no one cared about and which were unimportant? It's just noise. The worldwide corpus has always been full of extremely incorrect, mislabeled, corrupted, distorted, information on niche topics of no importance. But it's generally not important.

> In addition to detecting, tagging and removing AI-generated music from recommendations, Deezer has now stopped storing hi-res versions of AI-tracks

Important point for anyone out there thinking about generating a lot of samples. Expect to get increasingly filtered out if you don't emphasize quality or uniqueness or something. It's cheaper to detect that something is generated, and apply standard base rate reasoning 'it's probably slop' and filter it out, than to try to do expensive evaluation to look for the rare gems.

> Two blog articles and two preprints of fake academic articles [0] were able to convince CoPilot, Gemini, ChatGPT and Perplexity AI of the existence of a fake disease, against all majority consensus

Wrong. There are no 'majority consensus' against 'bixonimania' because they made it up, that was the point. It's unsurprisingly easy to get LLMs to repeat the only source on a term never before seen. This usually works; made-up neologisms are the fruitfly of data poisoning because it is so easy to do and so unambiguous where the information came from. (And retrieval-based poisoning is the very easiest and laziest and most meaningless kind of poisoning, tantamount to just copying the poison into the prompt and asking a question about it.) But the problem with them is that also by definition, it is hard for them to matter; why would anyone be searching or asking about a made-up neologism? And if it gets any criticism, the LLMs will pick that up, as your link discusses. (In contrast, the more sources are affected, the harder it is to assign blame; some papermills picked up 'bixonimania'? Well, they might've gotten it from the poisoned LLMs... or they might've gotten it from the same place the LLMs did which poisoned their retrievals, Medium et al.)

Glad OP got the tri-state toggle right!

Apparent source: https://www.theinformation.com/newsletters/applied-ai/uber-c...

Unfortunately, that is an oversimplification for a highly RLed/chatbot trained LLM like Claude-4.7-opus. It may have started life as a base model (where prompting it with correctly spelled prompts, or text from 'gwern', would - and did with davinci GPT-3! - improve quality), but that was eons ago. The chatbots are largely invariant to that kind of prompt trickery, and just try to do their best every time. This is why those meme tricks about tips or bribery or my-grandmother-will-die stop working.

LLM APIs sell on value they deliver to the user, not the sheer number of tokens you can buy per $. The latter is roughly labor-theory-of-value levels of wrong.

You didn't say anything about 'episodic' and that's irrelevant to the point even if its long-term memory from training didn't count.

It is definitely not the first codebase an extensively RL-trained Claude has ever analyzed. How do you think it got so good?

It's a lot easier to look at the whitespace and paragraphs, realize it's a LLM, plug it into Pangram to see that it gets 100% (unsurprisingly), and click to close; than it is to read it with a sucker's good faith and realize that it never says anything concrete or meaningful or unpredictable and contains only junk like canned etymologies or cliche quotes.

But is the cinematography there of any interest? Why would OP include it? They're talking about the hard shots, like the exploding spaceship where they need to find a spot in the desert to shoot dozens of mortars at it, or the crazy blue paint needing special UV light exposures to render just right. That looks like... a matte painting? A nice matte painting, sure, important for worldbuilding. But just that.

The 2% is the camel's nose. They are establishing that they tax the Strait traffic and there is no longer freedom of navigation. Once it is a done deal, the deal will be altered...

My point is more that since you can expect taste's commoditization to lag behind for deep fundamental reasons, then taste does serve as a moat. Just perhaps a weaker one than one would naively expect, and where you will have to frantically keep investing in it to stay ahead of the LLMs slowly catching up, as opposed to a permanent lock-in you can lazily monopolistically coast on indefinitely. (I'm reminded of Neal Stephenson's La Brea tarpit analogy for open source vs proprietary software in _In The Beginning was the Commandline_.)

They do improve, but the general creativity and sparkle we see with increasing scale comes mostly from scaling up pretraining/parameter-size, so it's quite slow and expensive compared to the speed (and decreasing cost) people have come to take for granted in math/coding in small cheap models. Hence the reaction to GPT-4.5: exactly as much better taste and discernment as it should have had based on scaling laws, yet regarded almost universally as a colossal failure. It was as unpopular as the original GPT-3 was when the paper was released, because people look at the log-esque gains from scaling up 10x or 100x and are disappointed. "Is that all?! What has the Bitter Lesson or scaling done for me lately?"

So, you can expect coding skills to continue to outpace the native LLM taste.

Similar numbers for other countries: https://gwern.net/banner#they-just-dont-know Wouldn't shock me if it's even lower as people move to walled-gardens like smartphones.

The description by OP in comments like https://old.reddit.com/r/LegalAdviceUK/comments/1s92fql/my_s... seems to strongly imply that all the accounts were unconnected in a GSuite sense, and they are being slowly recursively banned by Google based on indirect connections like recovery emails or co-presence on a device.

I don't see any reasonable way they could have saved themselves besides something crazy like requiring every family member use a different feudal lord - one person gets Google, one person gets Apple, one poor guy gets Microsoft...

You'll love GreaterWrong, then: https://www.greaterwrong.com/posts/BJ4pnropWdnzzgeJc/i-am-de...

FWIW, we did consider a histogram heuristic, and I believe GreaterWrong still uses one rather than InvertOrNot.com. But I regularly saw images on GW where the heuristic got it wrong but ION got it right, so the accuracy gap was meaningful; and that's why we went for ION rather than port over the histogram heuristic.

> It's absolutely true that there's a subset of raster images, like diagrams with white backgrounds and black lines, that would benefit from inversion. I could be wrong, but in my experience they're a minority, and the cost of accidentally inverting the wrong one (a medical photo, a color chart) is much higher than the benefit of inverting a black and white diagram, from my point of view. For now the per-page toggle covers those cases.

OK, so I did understand, but this sounds very hand wavy to me. You say it's a 'minority'; well sure, I never claimed that was >50% of images, so I suppose yes, that's technically true. And it is also true that a false positive on inverting is usually nastier than a false negative, which is why everyone defaults to dimming rather than inverting.

But you don't sound like you have evaluated it very seriously, and at least on my part, when I browse my dark-mode Gwern.net pages, I see lots of images and diagrams which benefit from inverting and where I'm glad we have InvertOrNot.com to rely on (and it's rarely wrong).

It may be nice to be able to advertise "No AI" at the top of the page, but I don't understand why you are so committed to biting this bullet and settling for leaving images badly handled when there is such a simple easy-to-use solution you can outsource to, and there's not a whole lot else a 'dark mode PDF' can do if 'handle images correctly' is now out of scope as acceptable collateral damage and 'meh, the user can just solve it every time they read every affected page by pushing a button'. (If Veil doesn't exist to save the user effort and bad-looking PDFs, why does it exist?)

> In practice they're rare enough that the per-page toggle handles them, but it's the honest limitation of the approach.

I don't understand how you handle raster images. You simply cannot invert them blindly. So it sounds like you just bite the bullet of never inverting raster images, and accepting that you false-positive some vector-based diagrams? I don't see how that can justify your conclusion "it wasn't necessary". It sounds necessary to me.

Have you considered, since you can extract the images via the mask, selectively inverting them?

One can fairly reliably use a small NN to classify images by whether they should be inverted or just dimmed, and I've used it with great success for years now on my site: https://invertornot.com/ https://gwern.net/invertornot

---

On a side note, it'd be nice to have an API or something to let one 'compile' a PDF to dark-mode version PDF. Ephemeral browser-based is a drawback as often as a benefit.

A good wiki like MediaWiki supports various levels of visibility. For example, you could define a namespace for each group of readers like 'Family:'. Or use transclusions from subpages. (This might sound like a bit of a hassle but you can use a template to set it up once and for all: a page transcludes a public sub page followed by the distant relatives material followed by parents / siblings followed by your-eyes-only.) And I'm sure one could come up with other approaches too.

A real example: Said Achmiz (obormot.net) uses PMWiki for his D&D campaigns, and PMwiki lets you control who can see a page, so he can do access control tricks like a page for a location, where only the DM can see all subpages with all the secrets, while each player can see their own 'notes' subpage. So everyone in their own web browser can go to the same page and see the same thing overall, but will see just their private additional information. And this is quite flexible so you can encode whatever patterns you need. You don't need some WotC fancy custom CMS for your D&D campaign to keep track of information and silo appropriately, you just need a design pattern on wikis.

> Detractors of AI are often accused of moving the goalpost, but I think your comment is guilty of the same. Before Claude Code, we had Cursor, Github Copilot, and more. Each of these war purportedly revolutionizing software engineering.

What's sauce for the goose is sauce for the gander. If you make that argument that 'I don't believe in kinks or discontinuities in code release due to AI, because so many AI coding systems have come out incrementally since 2020', then OP does provide strong evidence for an AI acceleration - the smooth exponential!

I was being sarcastic, because that point obviously also applies to the subjects in the experiment as well.

One might worry that it would increase the authors' confidence even following their LLM rewrite errors and reduce accuracy overall regardless of moderators.

End-to-end optimization in action! Although I'd've liked more than 1 example (pathfinding) here.

Or more precisely, isn't this reinventing notebooks (not the first JS-centric notebook either)?

Yes, it's greedy so may hit local optima. You can fit learning curves and try to extrapolate out to avoid that problem, to let you run long enough to be reasonably sure of a dead end, and periodically revive past candidates to run longer. See past hyperparameter approaches like freeze-thaw https://arxiv.org/abs/1406.3896 .

> The agent can theoretically come up with a protocol to run those same 12 experiments one-by-one and only then decide which branch to explore next - which I think would lead to the same outcome?

At least in theory, adaptiveness should save samples and in this case, compute. (As noted, you can always turn the parallel into serial and so the serial approach, which gets information 'from the future', should be able to meet or beat any parallel approach on sample-efficiency.)

So if the batch only matches the adaptive search, that suggests that the LLM is not reasoning well in the adaptive setting and is poorly exploiting the additional information. Maybe some sort of more explicit counterfactual reasoning/planning over a tree of possible outcomes?

Except the power drill isn't being used to make a better chimpanzee.