Hacker News Reader

Comment by walterbell | Hacker News Reader

> Generating synthetic data is another approach to catch up with incumbents housing large tracks of data. We know of a startup that produced synthetic data to train their systems in the enterprise automation space; as a result, a team with only a handful of engineers was able to bootstrap their minimum viable corpus. That team ultimately beat two massive incumbents relying on their existing data corpuses collected over decades at global scale, neither of which was well-suited for the problem at hand.

Generated data is being reverse-engineered by machine learning? With both the generation and ML written by the same team?

View on HN

Just to give a practical example. I've been working on a product and I need sales data but I can't get sales data until I have the product. So what do I do?

I made a basic market simulation. In my simulation there is a demand curve which dictates how many buyers exist at or below a given price. From the demand curve I build a population of people each of which has a maximum they are willing to pay. Then I pick people out of this population at random and I pick time intervals between sales from a poisson distribution. In this way I can test many many different scenarios in terms of equilibrium price, sales volume and market size. In about an hour I can have well over a million simulated sales across an exhaustive range of product types.

I don't know for a fact that my product works outside the simulation, but I'm confident enough in the assumptions I made that it is worth trying. For simulated data, the question is always "will the expected loss from a bad assumption cost more than the cost of acquiring a real data set".

Generated data can be used to increase variety as a way to avoid overfitting. A simple example might be translating or rotating images.

Not an expert, but I'm guessing it must be easier to do this than to improve how the machine learning generalizes from less data.

Usually when someone says “synthetic data” they mean CGI, not simply transformations of existing data. Using synthetic data is fraught (and presumptuous), as you are assuming you understand the problem domain 100% and are also extremely good at reproducing it. There’s a chance the model is using something specific to the CGI (and not the general reality) to produce its results.

For winning a computer vision competition it’s probably ok but I’d be very careful about using synthetic data for systems I cared about.

I thought "synthetic data" it's something that rarely shows in training image recognition, and is more like randomly generated user data (name, surname, etc.) or data generated from simulations of some processes?

>I thought "synthetic data" it's something that rarely shows in training image recognition

On the contrary, it is used to train models but it cannot adequately capture the long tail of weird events in the real world. Hence, it cannot be relied upon, as alluded to by the parent commenter. With reference to using data collected from a simulated environment vs real world ─ this subject was discussed at some length by Elon Musk and Andrej Karpathy, at the Tesla Autonomy Day event a few weeks ago.

I generally agree that there is no substitute for experience running in production and more data is better - or at least should be, if you can figure out how to take advantage of it.

The thing is, when it comes to weird events, historical data can't be relied on either. The next weird thing may never have happened before.

Predicting the future is hard no matter what you do. Gathering more data and learning more efficiently from what you have are both important. Training on artificial challenges can also be useful.

Rotation, etc. of images is known more generally as data augmentation and you're correct in that it's a way to reduce amount of training data to cold start a data product.

For "synthetic data", my understanding is that it's referring to use of ML to generate brand new training samples (e.g. through GANs, though there are limitations [1]).

1 - https://openreview.net/forum?id=rJMw747l_4

Reply on news.ycombinator.com