Hacker News Reader

> So they collect this data, vet it, propose their use for benchmarks and give it away free. What is the complaint?

It's well known that neural networks can easily inherit biases from their training data. It's also well known that datasets generated by western universities are widely used in training and evaluating neural networks.

If my training set is full of pictures of Stanford CS undergraduates, I could end up with a computational photography system that makes everyone look like Stanford CS undergraduates, or a historical photo colourisation system that makes everyone look like Stanford CS undergraduates, or a self driving car pedestrian tracking system that expects 90% of pedestrians to look like Stanford CS undergraduates.

And if the people who make the model say "Hey, our model's biases aren't our responsibility, we're just representing the training data as best we can" and the people who make the training data say "Hey, we never claimed it was perfect, you can take it or leave it" these problems might fall through the cracks.

> And if the people who make the model say "Hey, our model's biases aren't our responsibility, we're just representing the training data as best we can"

I don't think this excuse is like the others. If the model doesn't work well because they used bias data, it is the job of the people making the model to find better data (or manipulate the training process to overweight some data and attempt to counteract the bias).

I think the burden of responsibility has to be on the people who make models or put models into products to make sure the model is a good fit for the problem it is solving.

> It's well known that neural networks can easily inherit biases from their training data. It's also well known that datasets generated by western universities are widely used in training and evaluating neural networks. If my training set is full of pictures of Stanford CS undergraduates, I could end up with a computational photography system that makes everyone look like Stanford CS undergraduates, or a historical photo colourisation system that makes everyone look like Stanford CS undergraduates, or a self driving car pedestrian tracking system that expects 90% of pedestrians to look like Stanford CS undergraduates.

Why then aren't foreign universities/companies simply... building their own datasets?

>The problem is not even posed as "well, this data is overfitted in papers or we are solving narrower problems".

I mean it does get into that:

"They additionally note that blind adherence to this small number of ‘gold’ datasets encourages researchers to achieve results that are overfitted (i.e. that are dataset-specific and not likely to perform anywhere near as well on real-world data, on new academic or original datasets, or even necessarily on different datasets in the ‘gold standard’)."

True, I guess my complaint is that they don't get into the mechanisms.

The sibling comment on benchmark lottery paper lays this out. But I should modify.

I believe the author is suggesting that large institutions are doing this to earn extra citations - the currency of academia.

A large institution can make a dataset for X then browbeat other researchers into using X and citing X. Using X also likely leads to citations of derivative work by the lead institution.

Is the term "browbeat" fair here? I don't think they're making calls and saying "Oh, nice paper, but I notice you used this other dataset..." No, they're putting out good quality datasets that people want to use. If that earns them a citation, good for them. Its the least I can do for helping me test my algo.

It depends, feedback is rarely as polite as "I noticed you used this other dataset". The feedback would probably look like.

- "Nice paper, however the results are not relevant to current research due to the use of X dataset rather than Y or Z datasets score 2/5 do not accept."

- "Nice paper, however the results are of unknown quality due to the use of X dataset 3/5 recommend poster track".

In fact I'd generally say that most paper reviews would drop the first three words of those feedbacks. It's not an unreasonable assertion that progress is measured on standard datasets - but it's also necessary to push back on this.

Absolutely. Failure to report results on a popular benchmark suggests to some reviewers that you have something to hide - even though they might be computationally expensive or tangential to the main point of the work.

If a non-standard dataset is being used, I would expect there to be a discussion/analysis on what characteristics of that dataset made it unusable for this paper. Especially if a proposed model is being compared against models that were trained on those standard datasets.

If you are establishing new baselines using those same models on your non-standard dataset, then one would expect you to put in a good amount of effort to finetune all the knobs to get a reasonable result. If the authors put are able to put in that much effort, then that kind of feedback is definitely unreasonable.

>> If a non-standard dataset is being used, I would expect there to be a discussion/analysis on what characteristics of that dataset made it unusable for this paper.

Unfortunately that just adds more work for the reviewer, which is a motive for many reviewers to scrap the paper so they don't have to do the extra work.

That sounds mean, so I will quote (yet again) Geoff Hinton on things that "make the brain hurt":

GH: One big challenge the community faces is that if you want to get a paper published in machine learning now it's got to have a table in it, with all these different data sets across the top, and all these different methods along the side, and your method has to look like the best one. If it doesn’t look like that, it’s hard to get published. I don't think that's encouraging people to think about radically new ideas.

Now if you send in a paper that has a radically new idea, there's no chance in hell it will get accepted, because it's going to get some junior reviewer who doesn't understand it. Or it’s going to get a senior reviewer who's trying to review too many papers and doesn't understand it first time round and assumes it must be nonsense. Anything that makes the brain hurt is not going to get accepted. And I think that's really bad.

https://www.wired.com/story/googles-ai-guru-computers-think-...

Basically a new dataset is like a new idea: it makes the brain hurt, for the overburdened experienced researcher or inexperienced younger researcher alike. Testing a new approach on a new dataset? That makes brain go boom.

Which is a funny state of affairs. Not so long ago it used to be that one sure-fire way to make a significant contribution that would give your paper a leg up over the competition was to create a new dataset. I was advised as much at the start of my PhD (four ish years ago). Seems like this has already changed.

You might be right here. My comment was more of my expectation as a reader on what should be present in such a paper.