Hacker News Reader

Note that he's inferring from a base model there, which are fairly capable of regurgitating their (highly-weighted) inputs since they do nothing but predict pre-training tokens. For instruct services like ChatGPT, if they regurgitate something I'd think it would more likely be their fine-tuning data, which is usually owned by the provider (and also kept secret).

what I mean is if we can describe an LLM as a lossy compression (which are words spoken by Andrej), we could define what was done during inferring as uncompressing the compressed data, and at this moment shit would hit the fan.

It's a interesting question, one I wonder now that our federal data is being exfiltrated to AI companies. If they train their models on the data; how does the law tell them to 'unlearn'?

Destroying the copies they took will be what the courts ordered, but the data will still be there.

This is still being litigated I believe.