BioGPT: A language model pre-trained on large-scale biomedical literature

Probing it about CenPCA it gave these two definitions:

"CenPCA is an abbreviation for Centralized Principal Component Analysis. It is a dimensionality reduction technique used in data analysis for compressing high-dimensional data into a lower-dimensional representation. The centralization step of CenPCA involves centering the data, making the first principal component aligned with the mean of the data. This makes it possible to capture the overall structure of the data and to highlight the most important patterns in the data set. CenPCA has been used in various applications such as finance, image processing, and bioinformatics to analyze large and complex datasets."

"CenPCA refers to "Centered PCA", which is a variant of Principal Component Analysis (PCA), a widely used method in machine learning and data analysis. The key difference between regular PCA and centered PCA is that in centered PCA, the data is first centered around the mean before performing the PCA, while in regular PCA, the mean is not subtracted from the data. This has implications in the interpretation of the results, as centered PCA can sometimes provide more meaningful principal components."

I would think there has to be something to this but it is confusing exact titles although I have had it make up python code that simply is nonsense also.

Another type of example I had was during a language philosophy discussion it recommended the paper:

"Language Evolution: The Hard Problems" by Morten H. Christiansen and Simon Kirby is a paper that explores some of the challenges and limitations of current theories of language evolution."

It was actually just a book called "Language Evolution" by Morten H. Christiansen and Simon Kirby but trying to find that exact paper title made me think it made it up.

I am tending towards becoming addicted to it still after massive frustration with it for things like this at first. It seems you just have to be careful when probing the outer edges of knowledge and to not get led down these nonsense paths.

The exam was a reply for this part:

> isn't even close to understanding scientific concepts and knowing how to connect them

That test in my book definitely proves that it can understand (or let's say process and create abstractions) scientific concepts and can place them on a knowledge graph.

Put otherwise, GPT doesn't interact with reality directly, but mediated through our language. But we also don't experience reality directly, so it's hard to see how much of a hindrance that would pose. Hell, much of our model of the world is just as rooted in language as GPT's. The part of my model of reality of which I have direct sensory experience is tiny.

I think the key bit here is that we can influence its associations by inputting the prompt, changing the model that is presented.

I'm waay ahead of myself here, but the thought is interesting, and it will likely remain an open question for at least a few months/years.

Which is actually a novel capability and arises because the network does reinforcement learning over its own context window. It's a strength, not a weakness. Humans can do the same thing. ("Assume that X...")

> It just randomly landed on correct thing first just because it seen it more often in the input data.

Isn't that just a description of learning?

It's true that the network has no idea what is "true". But it's not like we do either, all we do is learning from correlations. We're just better at it.