Has anyone really pondered the consequences or are we just repeating the concerns of others without thinking for ourselves again?
What are real actual consequences? Especially if I’m a small company? Except of course it repeating or regurgitating your content, which is a valid concern.
It's like when people found out that Ring/Alexa staff were viewing their recordings and got upset. Even if they make super sure not to commit any further violations with their access, the act of consuming and making use of my data is a violation in of itself. It violates my privacy, and my ownership of my data. Even if my data can't be retrieved from the product you use it to make, it still feels icky that you changed the terms of our deal without my permission, and took advantage of our relationship to make extra money off of stuff that is fundamentally mine, and I have no recourse.
If my landlord surveilled my apartment and sold the footage on PPV but only in China, the material consequences to my quality of life would be negligible. There's virtually no way that any of the people watching the footage would ever contact me or interact with me in any way, especially if they did it with all of their tenants and randomized the footage such that it's hard to track one specific person. But it would still feel super weird and bad, and it would still be a violation of our relationship.
Unless I give my consent unhindered then my data is my data and cannot be used for any purpose at all unless I approve. This also means you cannot make blanket statements in Terms of use saying I grant you permission to use my data. I don't.
That is exactly it.
LLMs are nothing more than engines that predict the next token based on a mind-boggling large set of statistics of what tokens follow what. They're essentially Markov Chains on a massive dose of steroids.
If the training data includes one document that includes password rules like "The password shall be at least 12 characters long. The password shall include [...]", and then you have another document that includes a line of "The password is fdFjkkl1@!#", then if you ask the LLM to generate a document that includes password rules, there's a chance it may output "The password is fdFjkkl1@!#" in that document, because it knows that "is fdFjkkl1@!#" sometimes comes after "The password".
I guess we'll all find out if/when people use the model to get it to disclose other users information but I won't be holding my breath.