AI/Math Puzzle

Because it is reassuring.

Because this is just simple substitution of common math/CS functions.

One concern is that coding/logical puzzles are verticals where LLMs have lots of training data, they require small context window, and that's why they are doing well, but they don't necessary scale/generalize on other topics. For example I yet to see agents which would grab say Postgres codebase from github, and add untrivial features, send patch which is accepted.

I think the usual bearish sentiment comes from:

1. The marketing (even "research" papers) vastly overstates these systems' capabilities

2. We've observed unacceptable failure rates for the majority of useful tasks

3. Despite a bunch of money being poured into it, failure rates are decreasing slowly even with exponentially more training and inference resources, and there are precious few tasks where we've arrived at an alternative reframing making those errors tolerable

4. The capabilities are increasing slowly, and it's not obvious that pouring resources into the problem will lead to the phase shift necessary to either overcome (3) and (4) or to generate some novel application that makes it all worth it

Almost every attempted application I've seen has "worked" by using the AI to push externalities to other parties. It hasn't actually made the composite system meaningfully more efficient, with notable exceptions (e.g., the chat interface, once you learn the tool, if you use it correctly, can meaningfully improve certain kinds of productivity).

Plus, for people who actually work in AI, the current approach is obviously wrong. There are roughly two sorts of architectures in use for variable-width data: recurrent networks which store everything and process everything (e.g., transformers), and recurrent networks which compress everything and process the entire compressed space (e.g., mamba), with a special mention for compression-based architectures which have a large enough state space that compression is unnecessary, and another special mention for slight variations yielding constant speedups like MoE. The problem with all of those ideas is that you're necessarily trading off accuracy for compute. To have sufficient accuracy on arbitrary problems, you _must_ store everything, and you _cannot_ meaningfully compress it. The thing that isn't necessarily required though is processing everything at every iteration (a direction MoE works toward), and past a certain scale, unless we get vastly more efficient chips in a hurry (even if we do, depending on the target application), it's obvious that you need a new architecture to build around.

And so on. I work in AI (old-school ML for $WORK, new-school at home). I'm more bullish than a lot of people, especially since I see a direct cause and effect in things that make the business and my life better, but I'm still skeptical of the current craze. At a minimum, it's one of the most wasteful ways we could possibly do this research, and society isn't yet in a good place for the meager results we're getting. New-school AI is, currently, a net-negative to society, and I don't see that changing in the next several years.

Depending on what you understand by the term "prefix" you can dramatically shrink the search space without doing any thinking.

As I understand it, yeah, there's a way to ask the .ai nameserver for its entire "zone" (the mapping from domain names to... everything else). That's a "zone transfer" a.k.a. "AXFR" request, which you can make by first locating a nameserver that knows about .ai:

    $ dig NS ai.

Now you have the names of .ai's nameservers, and the glue records for some of them:

    ;; ADDITIONAL SECTION:
    v0n3.nic.ai.  107 IN A 199.115.155.1
    v0n3.nic.ai.  107 IN AAAA 2001:500:a3::1
    v0n0.nic.ai.  7 IN A 199.115.152.1

Now you ask that nameserver for a zone transfer:

    $ dig AXFR ai. @199.115.155.1

...And it quickly says "no, not to you; I don't know you and so I'm not going to spend the bandwidth to tell you all that."

    ; <<>> DiG 9.18.30-0ubuntu0.22.04.2-Ubuntu <<>> AXFR ai. @199.115.155.1
    ;; global options: +cmd
    ; Transfer failed.

But hey, that's how you'd ask. Now, if you were on the nameserver's whitelist, you'd see the whole zone, and the answer to the blog's puzzle would be somewhere in there. (But note that the answer is also at the end of TFA; you don't have to solve it yourself if you don't want to.)

I wouldn't use the words "API access" to describe "permission to make AXFR requests," but yeah, it's the same general idea: if you're not on the list, you can't do the thing.

More than I care to know about AXFR: https://cr.yp.to/djbdns/axfr-notes.html