5,020 karma · joined January 3, 2013
This premise makes no sense. AI makes rewrites to any language easy.
https://news.ycombinator.com/item?id=48706307
Even if it were written by hand, it’s a very poor and frankly stupid essay about an interesting topic. “The model's attention is a fixed quantity, and it has to add up to one, so the more things you make it look at, the less of that attention any single earlier thing can keep.” This is borderline gibberish and it outright rejects the interesting question about LLMs and attention, namely that they have very different capacities from us. LLMs can read an entire OpenAPI schema in seconds and immediately construct valid requests from it. The article first points this out, and then switches to arguing that LLMs have similar limits to us. It’s completely incoherent.
Here[0] is a fun selection of excerpts from his July 2024 post "How Does OpenAI Survive?"[1]
"I see no signs that the transformer-based architecture can do significantly more than it currently does."
[0]: https://xcancel.com/pathsnotchosen/status/206360940100129633...
In any case, maybe I was too subtle. I was talking about Mythos, a model that continues the trend, but which is not available to the public yet. The "overwhelming evidence" is the testimony of the people who have used it. The irrational skepticism was people who don't believe that testimony. In other words, we do know the future, because we know that model and others like it will come out soon.