Hacker News Reader

Comment by slashdave | Hacker News Reader

GenAI will make faking western blots fantastically easy

Many journals now require all versions of a gel image that is used in a figure. So, you’d have to fake the full image that is cropped down to the lanes used in the figure. I think there aren’t as many of those raw images around to train AI on… yet.

I predict it will get even worse than that, in the next couple of decades I expect any document or work that has a substantial reward associated with it, either financially or in terms of career advancement or a grade for critical course work in one's major, or penalty such as indictment or conviction, to be backed by a time stamped stack of developing documentation, drafts, revisions, with these time stamp validated against a trusted custodial clock and a seed random string marking the start of work, recorded in some immutable public form.

Accompanying to finish document will be a hash of all of these works along with their associated timestamps, originals of can be verified if necessary to prove a custodial chain of development over a plausible period of time and across multiple iterations of the development of the work - a kind of signed time-lapse slideshow of its genesis from blank page to finished product as if it had a mandatory and global "track changes" flag enabled from the very beginning - by which the entire process can be proved an original human-collaborated work and not an insta-generated AI fiction.

I actually thought that digital timestamps would have been a great use-case for blockchains. They are publicly available and auditable. If you're working from hashes, you don't necessarily need to make the raw data public, just the hash. It is a use-case that has an intrinsic value to the data generator and the future auditor (so you could charge something for it). I know there was some work done on this, but I think it lost momentum due to trying to generate crypto as a value storage medium.

The gold bugs really set back that entire field: the quasi-religious pursuit of “trustless” designs made everything more expensive, but so many problems are far more tractable with trusted third parties both for cost and reduced attack potential because institutional/professional reputations are harder to build than getting n% consensus on a cryptocurrency and don’t have the built-in bug bounty problem.

For example, imagine if university libraries ran storage systems based on Merkle trees with PKI signatures and researchers used those for their papers, code, data inventory (maybe not petabytes of data but the hashes of that data), etc. If there were allegations of misconduct you’d be able to see that whole history establishing that when things were changed and by whom, and someone couldn’t fudge the data without multiple compromised/complicit people in a completely different department (a senior figure can pressure a grad student in their field but they have far less leverage over staff at the library), and since you’re not sharing a database with the entire world you have a much easier time scaling with periodic cross checks (e.g. MIT and Caltech could cross-sign each other’s indexes periodically so you could have confidence that nobody had altered the inventory without storing the actual collection).

Sounds complicated. You could just demand the lab log books. They are supposed to be dated and countersigned. Standard practice is to counter sign outside your group.

The YC company that wanted to sell fake survey results (yes they really had a launch HN with that idea) will surely be the first to sell fake science results next. YC disrupting sciences

Yep! Even at the moment you don't even need GenAI, you can just run a dummy gel with X, Y, and Z proteins that will give you desired band patterns.

Eventually AI will also be able to reliably audit papers and report on fraud.

There may be newer AI methods of fraud, but it will only buy you time. As both progress, committing to record a fraud generated by technology will almost certainly be detected by a later technology.

I would guess that we're within 10 years of being able to automatically audit the majority of papers currently published. That thought must give the authors of fraudulent papers the heebee jeebies.

The problem is that detecting fraud is fundamentally harder than generating plausible fraud. This is because ultimately a very good fraud producer can simply produce output that is identically distributed to non-fraud.

For the same reason, tools that try to detect AI-generated text are ultimately going to lose the arm's race.

It's not a race though. Once the fraud is committed to record it can no longer advance in sophistication. Mechanisms for detection will continue to advance.

I think the argument is that if you produce your fraud from an appropriate probability distribution, any "detection" method other than independently verifying the results is snake oil.

Reply on news.ycombinator.com