Hacker News Reader

Comment by sigmar | Hacker News Reader

>“It will make it very easy for any researcher or group of researchers to create fake measurements on non-existent patients, fake answers to questionnaires or to generate a large data set on animal experiments.”

Perhaps I'm naive, but I think the people that want to fake data were already doing it without tools like chatgpt. Especially since a ton of biological data is normally distributed, so it's exceedingly easy to generate plausible fake results for such data without a system as advanced as chatgpt

View on HN

Cases of fabrication have been caught because the fabrication is done so badly, i.e. inplausibly. It's often just imputing some "random" numbers or repeating samples etc. Many would be amazed how technically and mathematically illiterate scientists often are. And probably ones that fabricate even more so.

Maybe it will increase and/or get a bit higher quality with LLM fakery. But as with many "AI bad" themes, the problem isn't that "AI" can fabricate the data. The problem is fucked up institutions and cultures.

Being able to do it at scale is an issue because of paper mills.

The issue is that we consider passing peer review to be enough for something to be taken as truth when it should really be reproducibility/replicability. That, and the incentives currently driving academia are absolutely ridiculous. Paper mills are merely a symptom of academia's ills, not the cause of them.

`np.random.normal(0, 1)`. You too can generate infinite amount of normal distribution data to use for fake datasets, free of AI.

People are probably thinking data more complex than normal distributions (though I'm also not sure if GPT-4 is the best method for that)

The easiest data to fake is null results because anything else is replicable, hence the importance of replication.

null results are also replicable.

A null result, or the absence of evidence, is not evidence of absence. If you fake a null result, you’re not asserting anything other than you could not measure and collect supporting data using your experiment to prove or disprove a hypothesis. It is difficult for someone doing replication to accuse you of ill-intent, as opposed to faked data that proves your point when anyone else can replicate your experiment and get totally different or even contradictory results.

You still have to give out the statistics that show the null result. E.g. something with a high p-value. You are in fact "confirming the null hypothesis". They aren't any more difficult to replicate than results supporting "the alternative hypothesis".

(The whole binary hypothesis system and culture is a mess though, but that's besides the point.)

This is true.

However, I think that no one will do the scut work necessary to find that a null result was faked, and even if they do since you the researcher got very little status out of it then it’s believable that you made a mistake, and didn’t falsify data.

Probably not, especially as null results are almost impossible to publish (which is mad).

But you won't find null results in a journal, so problem solved.

So we’re pretending making something an order of magnitude easier makes no difference? Ok.

I'm not sure this is much better than the state of the art. Training a model on data and then having it generate new, fake data, is not only easy, it's a standard tool for model boosting.

Poisoning the well for others, huh?

I wouldn't immediately call creating synthetic data 'poisoning the well' unless it is actually distributed as such. For training models with a minimal amount of quality data, it is a viable method for generating more data to increase the quality of the models. But any legit organization will obviously label synthetic data as such.

It's not an order of magnitude easier. It won't make a difference

ChatGPT will unlock new levels of both good and bad. The question is what the ratio between the two is going to be.

If you think about it, it will make 99% of ChatGPT answer technically INCORRECT.

It is harder to keep bad data out than it is to keep it filled with good data.

Reminds me of the “asymmetric bullshit principle”

Precisely.

https://en.m.wikipedia.org/wiki/Brandolini%27s_law#:~:text=B....

which is the best kind of incorrect?

This is so worrying for me, the amount of digital garbage that can now be generated that is not obviously garbage, that one must read and discern first nonsensical bad text generation and second if factual, is truth becoming a needle in the hay stack? How can this be cleaned?

Are we pretending Faker hasn't been a staple library in software testing for years?

Are we pretending GPT isn’t a leagues better data-faker than Faker?

As always, it depends on your requirements.

GPT definitely wins out if you want more novelty/variety in your fake data and are willing to accept extraordinarily higher cost, less rigor, and less reliability. I'm sure there's some occasion when those criteria win out, but Faker's pretty decent most of the time.

Thankfully using GPT doesn’t preclude you from using Faker, so I think we can all agree this is strictly an improvement in one’s ability to fake data.

Sure, but what do you expect? It's like blaming Boeing for genocide and then trying to get rid of 747s.

There's plenty of research into AI safety. There were some damn coups going on over AI safety. The general public defines AI safety as Skynet and homemade bombs, but it's also things like this - political manipulation, astroturfing, fake data, the risk of another industrial revolution.

It's something we should be slamming the brakes on, but most of the people calling out AI safety are also building their own B52 bombers, so nobody takes them seriously either.

80000 Hours has been telling people to get into AI and nuclear policy for years now. Hopefully we have some competent people in govs who do something.

Exactly. How is this news? What would be surprising is if ChatGPT _could_ generate fake data that passed analysis.

This could be the key to gain ultimate understanding

AGI Silicon Valley style: Fake it till you make it, just like everything else.

I mean, that's how adversarial networks work, isn't it?

Add bitcoin and a DAO and it becomes completely autonomous. Or lawyers…

This.

I think you meant _couldn't_

Reply on news.ycombinator.com