Hacker News Reader

Comment by lokar | Hacker News Reader

The problem is as never the tests. It was pretending that the difference between a 600 and 625 (or whatever) really predicted anything.

It was the silly idea that with tests you could produce a fair ordering of students based on potential to succeed.

View on HN

You can absolutely make a bet on who's more likely to succeed based on a 100 point difference, though. It's not absolute, but it's highly predictive. And the reason the SAT was dropped wasn't because admissions were being forced to blindly accept 620 over 610 (they never were), but so that people who scored hundreds of points below the mean could be admitted (in the pursuit of other institutional goals).

We have decades of data (test score vs grades and degree completion). They should gather it up and calculate the answers.

Flip answer: the bucket width should be 2.5 times the score improved of a prep course.

Any working system has to rely on some arbitrary rules. Drawing a line between students who scored 600 and 625 is still infinitely better than drawing it based on the decision-makers' moods.

Or, treat 600-625 as a tie, and use a lottery.

As imperfect standardized tests are, they are still more fair and less biased than using arbitrary judgement on extra curriculars

Bucket to the observed predictive power of the score, resolve ties with a lottery .

Would this be fixed buckets? I.e. would you treat 649-650 more predictive than 648-649? Presumably that wouldn't work. I'm sure there's some algorithm that could do this but it seems subtle.

Obviously, if a school has a cutoff score bucketing is easy, but with excess applicants ordering becomes necessary. I guess this sort of probabilistic score would induce an order for any given student relative to sufficiently superior or inferior applicants.... I'm now kinda curious to figure this problem out. Did not expect an algorithms problem to arise in this thread lol

And you don’t want a 100% cutoff. You need to admit some people just under the threshold since the scores are relative. You need fresh data to keep the model tuned.

Some kind of weighted lottery

Allow me to propose a model for this score-based ordering with fuzziness. (Perhaps we can call this problem probabilistic rasterization.)

The final output of an execution of the system, given a static, complete set of applicants is a particular ordering of applicants. Since lottery is involved, there are multiple acceptable orderings for a given input set. The question is to define a set of criteria to classify acceptable orderings, and a desired probability distribution of orderings, which can be satisfied by an algorithm for a maximal proportion of inputs.

For example, given a set of applicants A with score function F, we notate an ordering relation R(x,y) such that, given a limited number of seats, applicant y will be admitted before applicant x. For shorthand, x < y means R(x,y).

Possible acceptance criteria for an ordering R may include:

(1) Given some d in the codomain of F (presumably a group), FOR ALL x,y in A, if F(x) + d ≤ F(y), then x < y

Possible criteria for the distribution of orderings may include:

(1) FOR ALL x,y in A, if F(x) = F(y) then P(x < y) = P(x > y)

who uses SAT scores as "potential succeed"??

The original argument for standardized tests was to pick based on how well you would do in university (vs who your parents know).

Reply on news.ycombinator.com