Hacker News Reader

Comment by rfergie | Hacker News Reader

> there are students whose performance on such assessments is not a good signal of their intellectual ability

Is there a form of assessment that is a good signal of the intellectual ability of all students?

Fruit of labor? Good students of science produce novel science..as in testable ideas without precedent.. a thing llms struggle with, as they at best can interpolate and mash up previous ideas

Many good students don't produce such artifacts. That's one of the big problems in academia -- a smart person with good ideas can inadvertently pursue a less promising path and not produce ground-breaking research, so how do you tease that apart from somebody scamming the system? The current approach is to measure unrelated bullshit like citation metrics. I'd hate to see those sorts of perverse incentives pushed down into the student body as well.

Or maybe you can be smart but have no ideas in quantity and quality, thus you rig the systems in your favour to not be dethroned by those who can. But that would show by number of innovations per yearafter a systemic shakeup like a war.

It shows up easily at a macro level of you look at total "innovations" for a society (and have a good way to measure that -- currently still in an "I know it when I see it" stage). It won't usually show up in individual researchers. A few will absolutely be insanely productive with real innovations, but unless you're proposing only hiring that small cast of individuals then you're back to the problem of distinguishing good scientific ideas and methods which just didn't pan out from other forms of actual laziness and fraud given that in both cases you see no meaningful output from the researcher in question.

Reply on news.ycombinator.com