Hacker News Reader

I’d argue you’re incorrect on most accounts. I work in this space, and have implemented what’s being described related to synthetic data.

You can see some of my work here:

https://medium.com/capital-one-tech/why-you-dont-necessarily...

The reality is, you can guarantee certain values are not passed. That’s fairly easy, what you can’t do is block general trends easily in synthetic data - else the model can’t learn. So you have to be willing to accept that “leakage” when using synthetic data.

If you accept that leakage, then you can actually improve model performance in some domains. See ubers architecture search blog, there’s a lot of material (also from Uber) showing this.

Regarding cost of training, yes model training costs could increase. Although, I’d suggest much much less than 10x more like 50% or something. Is that worth it for privacy?

Have read your post. Interesting! Is the code perhaps already available?

What specific privacy guarantees are you hoping to provide and how do you know you are satisfying them? That is the key to basically all of it and most work on this subject falls flat (except for differential privacy, but current evidence indicates that it doesn't work very well for generative models).

Guaranteeing that certain values are not passed is both useless and trivial. I can easily satisfy that by simply adding 0.1 and a space character to all values in my dataset even though that doesn't remove any of the sensitive data.

> The reality is, you can guarantee certain values are not passed. That’s fairly easy

Nonsense. There's a fundamental requirement that for a model to have utility, it must have access to relevant data. If you truly find a leak-proof way to block certain values (which is borderline impossible in itself), then you make the model significantly weaker or outright useless if those values aren't uniformly distributed across the input domain.

And if you do have leakage (and you almost certainly do), then direct re-identification becomes a trivial problem and we're back to square one.

Synthetic data generated from statistical analysis of real data is either worthless or leaky. One of those two conditions is formally guaranteed to be true.

GP is correct that a viable synthetic data generator basically has to have already solved your problem for you, and in that sense it just becomes your model which is trained on real data. Training an additional model on top of that model doesn't add any privacy or mitigate reidentification.

The idea that the synthetic generator model must have solved the actual modelling problem is a attractive idea that doesn’t correspond to what people want data for: they want to eyeball it, see what’s in it, test some algorithms, figure out how they might approach the problem. You can do that very well on realistic synthetic data (much better than any other privacy tech) even if the synthetic data has lost some utility through its statistical approximations.

The idea that training on synthetic data is a “charade” misunderstands the usefulness of having realistic, “drop in compatible” data that works with your existing code or models.

The ideas of training models as a service and also of working directly with synthetic data generators to “extract“ from them are great but incompatible with (a) the complexity of real world DS workflows in regulated industries and (b) data scientists current code / workflows / techniques.

If I’m a bank, I’m not going to give you my fraud rules and you’re not going to solve my problems with xgboost. Access to models is just as locked down as access to data.

This is why it’s useful having an intermediary. Like ... a generative model that you can train where the data is and then copy over to where the modelling is happening.

I think there are a number of different use cases that people want data for, so there aren't any blanket answers to "is this a good approach".

As an intermediary like you describe is far different than "I don't have enough real data for what I want to do", for example.

> must have already solved the problem at hand

Assuming the problem at hand is analyzing the data.

I don't know enough about what they're building to understand what use cases they have in mind. (Their blog says "developing new features or exploring insights", which is a bit vague. See https://medium.com/gretel-ai/gretel-readme-fd0c4eff8a09 .)

But if this allows people to take their production database, run a command, and generate fake data they can run integration tests against, then that's not analyzing the data. It's making it easier to run tests in an environment that is as close as possible to production, because data is part of an environment.

Aside from that, I'd argue that some kinds of analysis may even still be possible. Let's say you have an e-commerce site, with one database table for user accounts and another table for shipping addresses. And let's guess that Gretel anonymizes this data by creating a user table with different usernames and an address table with made-up shipping addresses, but the overall structure is isomorphic: for every row in real data, there is a corresponding row in fake data, just with different values in it. And the (synthetic) keys aren't private (they data you generated, not collected from the user), so they can be preserved. Then you can, on the fake data, run a query and find out what percentage of users have given a shipping address.

Of course it's just a guess that that's what they're aiming to do since so little detail is available. It's entirely possible that what I described isn't what they're planning to do. Maybe it doesn't sufficiently protect privacy.

But the point is, until we know what they're building, I don't know if we can conclude that people should run away.

I got your point and it sounds good, but let's take your example of the shipping addresses. Let's say you want to develop a model that helps finding the best route for an ups driver. The real data may contain addresses all based in New York because your customers business is located there. The fake data contains addresses based all around New York. I guess the model you train with the fake data won't be any helpful because instead the driver can do 5 locations in a hour, he has to drive 5 hours for the same amount. Gretle needs to know the context of the data and the importance of relationship between the data rows. A pure anonymisation won't be enough.

While generally I agree with your conclusion (synthetic data doesn’t have better privacy guarantees, probably will hurt training if you use it naively), I wouldn’t be so pessimistic.

At the risk of digressing from TFA, Candes’ knockoffs, for instance, are an example of a (theoretically) successful use of synthetic data for model robustness. Still need original data, of course.

Basically, the broader point is that you don’t need to solve the full problem of joint likelihood estimation to use generative models effectively, e.g., GANs are another example.

If you take the MNIST dataset and generate synthetic data that has variations on the original images, e.g. rotations, added noise, different background images, etc. you will get a higher model accuracy than the baseline model in exchange for a longer training time.

Maybe I misunderstand something, but if x -> f(x) is easy to compute but you want to learn f(x) -> x, then isn't synthetic data exactly what you want to be using?

Example: Training an image upscaling algorithm by feeding it downscaled images. In this case you don't even need to train a generative model (the algorithm is known), but it should illustrate that the generative task can be extremely easy compared to the target task. You can't just handwave that away with "just divide by P(X)".

What about synthetic data that is correct? Example: Using the output of a physics simulation to train... The output is synthetic (as in, its not from the real world) but it is governed by the equations of motion...

This is the one that seems to be useful today. Running simulations with physics being modeled across objects seems to be useful for reinforcement learning algorithms that interact in the real world. You see some of this with autonomous vehicle training, robot manipulation training, etc. I think there are also emergent interactions that can occur that you wouldn't have modeled directly unless you let the simulation run.

The example is fundamentally different as physics simulations have closed form solutions or can be numerically approximated for various initial and boundary conditions. Therefore, by definition the generation is not synthetic.

Other examples I could think of fall in the category the top comment is referring to.

The point lalaland1125 is making is that the equations of motion are the existing superior model in this scenario.

Right, but presumably we don't want the model to learn those, since they're an impractically low-level model. If you're training an AI for flying a plane, you want it to learn "the simplest model that works"—presumably something like aerodynamics—not a far more computationally-intensive (and data-intensive!) model, like e.g. quantum chromodynamics.

Going from the parent comment's assertion, the answer would be that if you want to train something to predict physics, the original physics simulation that's generating the data is either:

1. Already doing that, in which case it's already the superior model.

or

2. Not doing that correctly, in which case the synthetic data output is equally poor.

I mean, if (1) is true but you get faster performance, then it's useful, right?

Yes. A typical example would be the use of statistical emulators, emulating process (physical) based models that are computationally complex in the environmental sciences.

Consider computer graphics and computer vision. If you had a photorealistic renderer, it could be useful for generating training data for a vision algorithm. However, you can't just run your rendering pipeline backwards to create a scene out of an image.

> When you hear a company talk about the promises of synthetic data, you should run far far away. The fundamental problem is that in order for synthetic data to be useful for model training the generative synthetic model must have already solved the problem at hand.

I've heard this group is pretty good, and they seem to be proud enough of their synthetic data to publish papers on it:

https://ai.googleblog.com/2020/02/generating-diverse-synthet...

Illuminate me, why are they wrong?

That post is illustrating parent’s point: to generate realistic fake data, you have to first create an accurate model of the real data.

I answered this in another post as well but this is straight up misinforming.

Generative models do not necessarily know P(X,Y), often they simply give samples of (X,Y) using some map

G: Q -> (X,Y); Q~N(0,1)

Actually getting P(X,Y) would mean inverting G, which is not always possible (although there are some invertible GANs).

Computing P(X) is often computationally intractable as tracing out variables can be arbitrarily expensive.

A recent(ly featured on the front page) example where synthetic data is helping solve an unsolved problem: https://eng.uber.com/generative-teaching-networks/

I'm being pedantic by posting this though, because their approach involved learning the synthetic data generator simultaneously with the classifier/whatever that you're training. That is not relevant for a static synthetic data source.

Why are you citing Bayes Law here? P(X) might be some gnarly computation.

@lalaland1125 you're right that privacy is a hard problem, we're really excited about making techniques like synthetic data, data labeling, and analyzing data sets for potential privacy (re-identification risk/etc) available to all developers. We'll be open sourcing some of our work in the next few weeks- feel free to jump in the code and we'd love to hear your thoughts!

I don't know, if your generative model is something like getting the knn of each data point, then fuzzing each point by the local statistics of each point (from the knn's), I don't think you will miss the important trends and you get a bit of differential privacy.

Might you or someone else say what this specific area of ML generative models/synthetic data is called? Are there any introductory references you could share. Thanks.

You’re coming at this from a theoretical approach, which isn’t appropriate for deep learning because nobody has any idea how or why these neural networks actually work. Generating a bunch of rotated and stretched versions of your labeled images DOES work, and very effectively. Why? The neural network is an inscrutable God, we just have no idea.