AI 'breakthrough': neural net has human-like ability to generalize language

Yes I'm aware. My point is that

1. The level of variance displayed here is very atypical for a language model unless you're making significant changes to how the information is presented. This alone is cause for more elaboration.

2. With this big a variance already showing, it's hard to say for sure that they hit the top end. Maybe it seems strange but most LLM evaluation papers still don't actually bother to tie in even some extremely well known/basic "output boosters" like chain of thought and so on. But at least in these instances, we know how they presented it.

a "this is how we presented the task, this is how it performed" would have been very nice.

I kind of dislike the term "Prompt Engineering". It mystifies what to me is an ordinary process. I'm not talking about chanting magic words, i'm just talking about taking the structure of an LLM into account, even taking how humans would approach the problem into account.

In microsoft's agi paper, there is a test for planning they put out for GPT-4. They give detailed instructions on the constraint of a poem and expect it to spit out the constrained poem in one shot. Of course it fails and the conclusion is that it can't plan. But if you think about it, this is a ridiculous assertion. No human on earth would have passed that test with working memory alone. Even by our standards, it's a weird way to present the information but they did so anyway. This is actually a major issue for most plan benchmark assessment for LLMs i've come across.

These are the kind of problems i'm talking about. If i changed the request to encourage for a draft/revise generation process and it consistently passed these tests then it's very fair to say it can plan. That's not "hacky". It's just truth. and saying otherwise is denying real world usage for a misplaced sense of propriety to benchmarks. a benchmark is only as useful as the capabilities it can assess.

a "this is how this is presented, this is how the model performed" would have been very prudent in my opinion. Maybe these guys already covered the kind of things i'm talking about(i can accept that!)....but i can't know that if they don't tell us.

So what if we have an AI prompter-assister... we take an input prompt: run it through an AI to transform that prompt into proper AI talk - then that is what's prompted to the actual AI... That way the AI will always have the best prompt possible, because the Prompt-helper helps your prompt...

/s if anyone needs it.

I think adjusting the prompt makes sense. We have to keep in mind that this implicitly happens on the human side as well. The person creating the initial question will consider how understandable it is for them and edit until they're happy. Then the questions will be reviewed by co-authors and possibly tested on a few people before running the full experiment. They'd probably review the explanation again or change the task if they realised half the people can't follow the instructions.

We just call it "editing for clarity" rather than "prompt engineering" when we write for other people.

I would claim that most people are very bad at communicating in a way that can be interpreted, by human or AI, with a single shot. Most of the time some back and forth is required to clear things up. I think it's unfair to expect gold from garbage input, since meat brains can't do that either, without some back and fourth.

Those prompts should be available. It's ridiculous that they're not.

While I think that's a valid point, for the purpose of the paper having a competent prompt writer would be needed to make a fair comparison.

I think the researchers would know how to get the best from their own AI bot, so that's a level of competency that should be extended to comparisons, otherwise user competency becomes a source of bias. I do feel you're correct in your concerns though, the systems shouldn't need experts to use them, nor should they need the user to already know the right answer, which leads me to my next point:

When it comes to real world expectations, perhaps instead we need a large group of random people (with no prior experience) working with each bot to complete a set of tasks in order to determine how it truly performs - something that could be enhanced if the answers weren't easy to check.

Disagree - a model may have capabilities that it only deploys in certain circumstances. In particular, if you train on the whole internet, you probably learn what both stupid and smart outputs look like. But you probably also learn that the modal output is stupid. So it’s no ding on the model’s capabilities if it defaults to assuming it should behave stupidly.

If you tokenize using character n-grams, this isn't totally wrong.

I think a fairer comparison would be Toki Pona, a micro-language with ~120 words. you can express lots of things if you have a great deal of patience, Up-Goer Five style.

In logic there's also SKI combinator calculus, which is Turing-complete with three symbols, and unlike the Morse or A-Z examples, but like the Toki Pona example, each symbol has a semantic meaning.

If you just want to describe ideas in an abstract realm like sequences of colors, like this paper, it's not surprising you don't need many words.

> Most people know 1000 words on average in English

Maybe I misunderstand but that sounds so stupid and wrong I don't know where to start. Standard vocab estimates are 15K - 30K, averaging about 20K (these from memory)

Edit: from wiki https://en.wikipedia.org/wiki/Vocabulary#Native-language_voc...

---

As a result, estimates vary from 10,000-17,000 word families[16][19] or 17,000-42,000 dictionary words for young adult native speakers of English.[12][17]

A 2016 study shows that 20-year-old English native speakers recognize on average 42,000 lemmas, ranging from 27,100 for the lowest 5% of the population to 51,700 lemmas for the highest 5%. These lemmas come from 6,100 word families in the lowest 5% of the population and 14,900 word families in the highest 5%. 60-year-olds know on average 6,000 lemmas more. [12]

According to another, earlier 1995 study junior-high students would be able to recognize the meanings of about 10,000–12,000 words, whereas for college students this number grows up to about 12,000–17,000 and for elderly adults up to about 17,000 or more.[20]

---

I don't care to pay for access to gpt-4 but one could easily use one of the vocabulary estimation tests, which use some statistics plus knowledge of word appearance frequency, to estimate its vocabulary size. https://mikeinnes.io/2022/02/26/vocab is one such test which explains the statistics ideas, and there are many others based on similar principles.

100% of people know the most common 1000 words. The remainder of those who know more words fall into a consistent curve across languages that follows Zipf’s law. This is different than “most people know 1000 words on average.”

It’s called in-context learning. I wouldn’t call it a trick so much as the most exciting and useful feature of LLMs. A lot of research is being done to expand the context window which would allow you to prime the model with entire books or codebases. GPT4 has a max context window of 32k tokens up from 2k with GPT3.5. Whose to say iteratively shifting the output distribution isn’t a form of learning.

So to retain their relevance you claim they abandon academic standards and sensationalize an openly accessible study that anyone can read and thus dispute the article. Okay.

That's a category error. You don't submit your manuscript to a prestige journal in order to publish it but to take advantage of their PR machinery. The services Nature provides are very good and cost-effective, but you have to meet their standards.

As research papers, the papers published in Nature/Science/etc. are rarely that useful. Because the journals target the general scientifically literate audience, the interesting details are usually hidden in supplementary material. And the supplements are often little more than giant info dumps that nobody has paid sufficient attention to.

It would be nice to live in a world where such PR services are unnecessary, but that's not the world we are living in. The academia is very competitive, and there is never enough funding and never enough good jobs.