Ethical issues in research using datasets of illicit origin

To anyone who is interested in this topic, I published a paper a few years back on the topic of 'accidentally illicit' datasets. It's not my best work but someone might find it interesting.

http://firstmonday.org/ojs/index.php/fm/article/view/2739/24...

Amusing anecdote: I wrote this paper (2) after a reviewer insisted they would reject my other paper (1) unless I tested the paper's algorithm on 10% of all images on the Internet.

Talk about a hostile review!

(They also demanded I remove the performance comparison which showed the new technique to be some 1000x faster than existing techniques and more reliable... Hmm).

The reviewer then dragged out the review/response process for so long that I had time to write/review/publish the ethics paper above, in between one round of reviewer comments (!)

I then took the freshly published ethics paper to the editor for (1), and asked them to disqualify the hostile reviewer for making unethical demands and refusing to withdraw them even when this was pointed out.

The editor agreed. The reviewer was then replaced by someone else, who replicated the entire work of (1) completely from scratch using only the description in the paper, confirmed the result using their own datasets they gathered privately, and who approved publication.

'Reviewer 1', they're always either the hero or the villain. It was an interesting feeling to see the very worst type of reviewer being replaced by the very best.

Anyway, that's the strange story behind this paper :-)

1. Financial account aggregators and “budget apps” like Min monetize their business, in part, by selling huge amounts of data to the financial sector. Sometimes companies like Second Measure take raw data from companies like Yodlee and clean it, then resell it. Nowadays there is an entire industry of alterative market research that has had all sorts of participants, from Foursquare (locations) to Spark (email enhancement). This is technically authorized, because it’s in the TOS. The users effectively contribute their own data.

2. I developed an extremely accurate, reasonably generalizable method of forecasting vehicle production at several companies that relies on implementing a VIN searching algorithm in conjunction with legally required NHTSA recall lookup portals hosted by each manufacturer. This data is what you’d call unauthorized, because no entity explicitly endorses your use of it. For example, several colleagues and I knew well ahead of time that Tesla would miss on production of the Model 3s because they were utterly unrepresented in our data. But this data is public, so it’s fine to use from a legal and compliance standpoint. It was lucrative data specifically because it had a high signal for revenue, yet was hitherto unused and unidentified.

3. I once found, in the course of looking for legally usable data, an actual security vulnerability disclosing all users of a publicly traded QSR’s online delivery service, along with their phone numbers, email addresses and last four digits of credit cards. This is both unauthorized and illegal, because the data is contaminated with personally identifiable information and it clearly requires a vulnerability (not just scraping) to acquire.

I’ve seen overzealous data vendors accidentally slip from #2 into #3, which is really bad for all concerned. It’s not a great look for the vendor, who will likely be fired, and it represents a breach for the company who owns the data and its users. Any firm that has purchased the data will likely be contamined and be forced into a trading lockdown of that security for a period of time by compliance.

My real concern is that illicit data like this is used in machine learning research. Machine learning is already pretty frustrating - it’s common for me to find research from a conference that I’m simply unable to replicate because the training or experiment data is not available (this is annoyingly the case with A/B experiment optimization research put out by giant companies in particular). I worry that this trend of accepting machine learning research without any requirement for total data transparency will incentivize researchers to conduct their experiments using illicit data that doesn’t need to be sourced.