Hacker News Reader

Comment by fyp | Hacker News Reader

We probably need a technical rather than legal solution for this. The problem is that generative algorithms are susceptible to accidental memorization so you can't guarantee that the output will be transformative. For example, play with https://talktotransformer.com to see how many well known pieces of text it can spew out verbatim. It is very prone to derailing into a harry potter fan fiction regardless of prompt.

Other than copyright there are privacy considerations too. For example Gmail's Smart Compose is trained on users' private messages so you don't want it to memorize "private" details (such as credit card numbers): https://arxiv.org/abs/1802.08232

Is it possible to solve this by adversarially checking if the output is "original" enough or not? Or is that intractable, given how much resource our society already pours into making the same classification in court?

View on HN

I agree with your point, but I think the tendency to accidentally produce substantial quotations of some unknown fidelity of some copyrighted material is definitely not a "significant market substitute for the protected" material. There's no way I'm going to try and trick a bot into reciting a whole story to me instead of buying a copy of the story.

That said, these many-pointed court tests make it really hard to do anything around copyrighted stuff without being a large corporation, which is problematic.

If you look at it from a perspective of information theory, you may find that compression and AI are equivalent problems. And we know that compression (including lossy compression) doesn't remove copyrights.

Good thing that these court tests are establishing that the argument, rather than being technical, is about creation of value to society. And about potential economic damage to the copyright owners. As long as there's no damage and value to society, it seems that the courts are fine with the use of data.

This point is extremely important, particularly in the healthcare field (which I happen to be in at the moment). We have to be very, very positive that our deidentification process is thorough and accurate to prevent a HIPAA violation from occurring.

> We have to be very, very positive that our deidentification process is thorough and accurate to prevent a HIPAA violation from occurring

If your model is any more useful than something trained on coarse aggregates, then it can be used to reidentify individuals. This is a pretty hard dilemma in the entire industry, not just health.

I hope my observations are skewed, but instead of trying to seriously address this issue I've seen an entire legal-loophole style data laundering industry emerge where highly identifiable information changes hands without it 'technically' changing hands in the legal sense. I'm talking about entities like datarepublic.

It really is a complicated topic, and one we spend a lot of time thinking about. We're using a peer reviewed method for removing PHI identifiers, and are combining that with an approach that involves using the least amount of data possible to get results. Our models will also never be released to the public, but instead will have an API where we can see abnormal behavior (such as sending lots of requests to try and tease out other information) and intervene.

Other than easily controlled walled garden type accesses it is hard to limit with throttling.

Is there a way to determine how much information about a particular individual has been leaked out?

If you start from real data that's a very tough nut to crack. Depending on your purpose you may be better off with generating data that has the same statistical properties.

That's true, but in healthcare generating data that matches the statistical properties of real data is also a very tough nut to crack. Essentially you need experienced clinicians who can document a fake patient's care journey through multiple years with thousands of concept codes and narrative text blocks. The level of effort is on the same order of magnitude as writing a novel.

Of course if you don't need a complete longitudinal patient chart then generating realistic data for just one aspect can be a lot simpler.

https://www.statice.ai/

Sure and Synthea is doing something similar, as well as a few others. Those can produce data good enough for certain use cases. But it's not quite good enough for realistic product demos involving complex conditions, or testing anything related to record linkage.

We've tested numerous synthetic data companies and so far have not found any which can produce data that is useful for real world applications. There are some that are close enough where academic groups might be able to publish around it.

There is a single startup we're working with that has come much closer, but still isn't quite there. I think that if they manage to get funding they'll be somewhere usable in the near future.

Which one is that? Super interested for one of my customers...

It is possible to solve it by either:

1) Federated machine learning. Basically you let google train their model on your private data, with the understanding that some of it would be used for the public goods. But this separate your data from the weights.

2) Train your own models, and ask google to open Gmail such that they will call your models.

Reply on news.ycombinator.com