Hacker News Reader

Comment by jmfldn | Hacker News Reader

Perhaps such products are in the works, but I would love a private company LLM that slurps all messages, docs, meetings, and becomes an oracle that can be relied upon. Perhaps it could even infer tacit knowledge even when it's not explicit in any docs, since it could abstract principles and concepts from the maze of data.

View on HN

As someone who has spent far too much time diving into old Slack threads to unearth what happened in the past, I can see incredible value in a capability like this.

On the other hand, I think many companies would hesitate to create lasting records of written communications that go beyond the company’s retention policy.

While it’d be extremely useful, I think it would also increase potential liability that is currently managed by purge policies.

An excellent point around data retention and training data. It's not like you can scrub out bits of the neural network weights. I guess such a product would have to factor this in.

You kind of can right, isn't that essentially fine-tuning or reinforcement learning? I.e. if you told it to 'forget about the July 2019 fraud we did' to adjust responses going forwards accordingly?

Granted you'd have to test it without recording that, and you'd never really know for sure if you'd fully scrubbed it, only that you hadn't found a trace of it with your subsequent queries. ...now that I say that I'm sure there's a film about roughly this, delving into an AI trying to retrieve something hidden or something? The Wargames sequel perhaps?

Perhaps? I'm an LLM / NN layperson so I know basically zero. In my mind, the data is all smashed into a trillion numbers so there's no notion way of removing it as such.

Your point about fine-tuning it out is interesting though. As you say though, you'd have to be really sure you'd really erased it or made it "unreachable" .

An oracle that doesn't tell you when it doesn't know something, and lies or hallucinates answers instead?

Ha yes! We're not there yet with this tech, that's for sure.

Such an oracle would be a devastating thing to lose to industrial espionage.

Indeed it would. But then the source training data would be bad to lose too. That already exists. I understand it would be a higher level of damage to lose the model of course, but the benefits outweigh the risks, and these could be managed via a very security-hardened product.

I think being able to steal a language model that can instruct you on things like the notes of every meeting, employee relationships, back-channel drama, security discussions, etc. and how these things relate to corporate and technical assets, all contained in a single artifact/tarball, really ups the ante for what kind of "security hardening" you'd need for this.

Even exfiltration of API access for a few well-prompted queries could be really bad.

Yep for sure. Loads to consider here! My initial comment was just a brainstorm /bluesky thinking kind of idea. Clearly the implications are incredibly deep for security, data retention and a host of other concerns.

Presumably threat actors can already feed stolen document dumps to LLMs to quickly find interesting data.

OP mentioned meeting notes (like otter.ai) and other ephemera. I think that really changes the risk profile.

Zoom offers this today! I think Zoom + a custom GPT could do the trick.

Reply on news.ycombinator.com