Hacker News Reader

Comment by mkl | Hacker News Reader

They never got to the results of the competition and how the interpretable model did!

From https://www.fico.com/en/newsroom/fico-announces-winners-of-i...: "The team representing Duke University, which included Chaofan Chen, Kangcheng Lin, Cynthia Rudin, Yaron Shaposhnik, Sijia Wang and Tong Wang, received the FICO Recognition Award acknowledging their submission for going above and beyond expectations with a fully transparent global model and a user-friendly dashboard to allow users to explore the global model and its explanations. The Duke team took home $3,000."

Cynthia Rudin is one of the article's authors.

View on HN

But how did their model compare against others? The article only mentions how their interpretable model compared against their own ML attempts

Their model didn't win. IBM's model won, based on actual metrics around useful insights.

The IBM team got $5,000 and the second place/honorable mention NYU got $2,000. So going by prize amounts, the Duke model was still pretty good.

IBM turned the model/paper into a toolkit: https://www.ibm.com/blogs/research/2019/08/ai-explainability... Their model seems to be a variant of decision trees that has a knob controlling how complicated the trees are.

And the evaluation was completely subjective, so there's not any meaning to the Duke people losing besides that the judges didn't like them.

> And the evaluation was completely subjective, so there's not any meaning to the Duke people losing besides that the judges didn't like them.

That's what you get if you use a black box for judging :-)

Reading "subjective" to mean "nonexistent" is a potentially big mistake.

Objectivity is more accurate, sure. The winner of an objective contest is always objectively better against objective criteria. But, objective criteria are generally narrow. This works well if one is either (a) seeking fundamental principles like in physics or (b) the narrow objective criteria is the definite goal.

In this area, we don't exactly know how to define narrow, objective goals & subsequent criteria. We can definee goalposts, but not goals. These are guesses at useful markers of success, useful to the larger goal of useful/novel ai.

Subjective goals have their own (massive problems), but since we can't objectively define the goals of ai research... we need to fall back on human subjectivity to define our subgoals.

All objective criteria are chosen, directly or indirectly, based on subjective criteria.

Also true. Subjectivity is unavoidable as long as we are relevant.. or so it seems circa 2020

Reply on news.ycombinator.com