> there's still a long ways to go
Sure, in the way that technically this is a computable problem, but maybe not a simple one. Any exponential in the real world is a sigmoid and given all major AI labs, having spent years and incomprehensible sums of cache, have arrived at about GPT-4 performance, including OpenAI's latest release being a smaller model, should tell us something. Be the limiting factor corpus size, model parameters, or an architectural defect, we're clearly loosing momentum, at least until it's to be diagnosed and solved. Deferring to hypothetical futures without meeting the burden of evidence seems ill-advised, especially when it comes to incompetent use today.
> there's no putting AI back into the box
There's also no complete undoing of an oil spill, nor the practical possibility of unilateral nuclear disarmament. The weights are public, the corps is as well, as much as it had broken the open web to gather it, the architecture is known, ergo today's open-weight models are the baseline of capability for all future models. Still, seeing that LLMs form a natural monopoly given the required compute power to enter the field, we can enforce policies like mandatory statistical fingerprinting on all outputs of proprietary models beyond a certain size. LLM detection is also getting quite good; my hope is that adversarial fine-tuning will work out like Bayesian poisoning for email spammers, only giving the discriminator new stable patterns to look for. We as consumers do have power, however small and unconcentrated, to vote with our wallets, which a study posted here shown many are beginning to do [0]. The copyright question also gives quite a nice kill switch if our society decides this whole industry isn't that beneficial, removing the commercial incentive for training new models while providing some for detection.
There's great value from transformer networks, such as state-of-the-art speech recognition, that will make its way into consumer products and will be here to stay. As for the FADs like useless chatbots in every product, that may come under question.
[0] https://news.ycombinator.com/item?id=40390375