Academics Need to Wake Up on AI | Hacker News Reader

Academics Need to Wake Up on AI

popularbydesign.org

27 pointsbarry-cotter26 comments

Author doesn't seem to understand that LLM AI works by predicting tokens out of training data. The model writes a research summary because it digested academic papers and other sources in its training. When you say "AI can already do social science research better than most professors" that is false unless you mean the colloquial sense of "research" meaning "reading other people's existing stuff and paraphrasing it in my own words". But the AI doesn't even have "own words"; they are the training data's words.

If all scientists suddnenly do nothing all day but play with AI --- all research grinds to a halt!

What about Alphafold?

> But the AI doesn't even have "own words"; they are the training data's words.

If the AI understands those words, in what sense aren't they its 'own words'? Are you arguing that nothing but neologisms count?

I would say that I don't consider that to be an LLM.

It's not literally an LLM because the L stands for language, and it's not trained on language.

But it is the same transformer architecture, and it is able to generate novel proteins in the same way that an LLM is able to generate novel sentences. AlphaFold 3 is a diffusion model, so it's most similar to the AI art generators.

But it's used on the thing that it's trained for. LLMs are trained on language, but then used as a substitute for thinking, which it naively looks like they are doing due to the smooth language.

Protein folding is a kind of syntax. If you train on protein folding and then use it to obtain protein folding results, you are using a screwdriver to drive a screw: that checks out.

Nobody should be arguing along lines analogous to the claim that a good neural net trained on handwritten digits is not suitable for classifying handwritten digits.

This is true after pretraining, but reinforcement learning allows the model to discover strategies and ideas that weren't in its training corpus.

This is not what RL does, and please stop anthropomorphizing statistical modelling as the model certainly does not discovers ideas.

What does RL do then if not discover strategies and solutions that weren't in its training data?

RL adjusts the learned probabilities to conform to a secondary source other than the raw training data, for example (but not exclusively) human feedback. Putting it in extremely simplified terms: If, owing to the training data, the learned probability for "green people are _" is 70% to be followed by "inferior", you may use RL to massage this, de-scoring it every time it produces "green people are inferior to red people" and up-scoring it every time it produces "green people are an ethnic group originating from Greenland". Doing this will adjust its learned probability for that sequence of tokens.

At most, RL can be described as injecting information from a secondary source. It is not extending a model's programming to do anything other than what it was already doing, probability-based token prediction. It simply alters the probabilities.

What about things like AlphaZero and Atari gameplay, where the model has zero prior knowledge and learns superhuman ability purely using RL?

With sufficient RL sampling/training, there's no reason an LLM couldn't similarly develop entirely new skills, especially in verifiable domains like math and code.

> It simply alters the probabilities.

Yes? What else would a learning system do besides alter its behavior? (and you can just sample with argmax or pseudo-randomly of you think probabilities are a problem)

Functionally, i.e. focusing only input and output, a model can certainly discover an idea. That’s not anthropomorphism.

Similarly, people often object to using words like “reasoning” and “understanding” in relation to models, but again, functionally, models observably demonstrate both of those qualities - you can test for them and measure their proficiency.

The fact that this discovery, training, and understanding is implemented in terms of a statistical model isn’t really relevant. If it were, you could similarly argue that humans don’t discover, reason, or understand, we just process chemical and electrical signals through our biological neural network.

Are you perhaps thinking of transfer learning, i.e. where training on one subject can be applied to another? RL is more about coercing models in particular directions.

Don't undersell AI - it also synthesizes and recombines those summaries in a purposeful way. Otherwise it couldn't product code that works in an existing codebase.

So it is able to process and act upon summaries and concepts. In other words, apply synthesis. What it can't do is understand what a useful result looks like without direction. So it could synthesize a billion pointless claims from source material, but we still need a human to know which ones matter (without a specialized framework to comprehend this). If you provide LLMs with an objective and source materials it is certainly capable of following threads of logic or building an argument backed by sources.

I understand the concerns about AI, but it is a powerful tool for discovery and synthesis.

Another thing they’re often poor at is making an incorrect assumption and then going down a rabbit hole trying to unnecessarily solve for it. Without a discerning human in the loop, you can end up with large amounts of unnecessary output.

>7. Much of the opposition to AI is status protection dressed up as principle.

Absolutely true haha, even outside of academia.

Software developers wax poetic about the value of 'handcrafted human-made software', but really they just don't want to lose their cushy $300k WFH job.

Well, if you're a real human, you're also probably not in a hurry to go from a decent quality of life to homeless, either.

> Software developers wax poetic about the value of 'handcrafted human-made software'

They really don't; devs are the highest consumers of AI by far. Designers, on the other hand, have already been obsoleted and exist basically as makework generators. They can put plenty of useless bullshit in figma and have zero ability to execute.

"P.P.S. That is, entirely generated based on my artisanal, hand-crafted human social media posts and thoughts on the topic. So who wrote it, really? You tell me."

We can't since it is a vapid, unsourced, AI mania fueled piece that could have been written by AI.

I suppose the associate professor wants AI funding.

"Haw haw, you couldn't tell it was written with AI!"

"Oh! That explains it!"

"Uh..."

"I didn't want to say anything rude, but the whole time I was like 'yikes, how did this idiot become a professor at Notre Dame?'"

"Actually -"

"Heh! You got me good! Of course it was written with AI. Duh. These ideas are so vacuous and shallow, there's no way a fancy professor like you-"

"Actually, I asked the AI to write an essay summarizing my arguments from social media."

"...oh."

"..."

"...hey you should log off BlueSky, it's not healthy there."

As an academic this article was a fantastic position piece. I loved this and enjoyed reading it even if I didn't agree 100% thank you for sharing

CS Academia tends to lag behind industry practices. The research frontier can be very cutting edge, but course curriculum, assignments, and institutional norms are slower and more conservative. That’s usually manageable when the shift is something like cloud adoption, new tooling, or a new dominant programming language. But this particular industry trend, use of AI in software development, is massive and fast moving (especially the agentic workflow growth over the last 6 months). And we're just now understanding where everything fits in and its limitations.

Journal articles are sometimes years behind. There are still papers coming out that use GPT-3.5 (!) for their main result. These days I'm basically only reading arXiv preprints (and whatever is trending on GitHub).

Nope. Ai tools are simply not there at this stage. Very useful as autocomplete and quick overview and prototyping but can’t create new knowledge at all.

Is this the new clickbait? AI written AI scare papers.

and web's dead baby....web's dead.

academics are a thing of the past (ancient greece that is...) nowadays people are dumber than rocks, thank god (the expression) for AI arriving when she did.

Reply on news.ycombinator.com