Stanford’s Alpaca shows that OpenAI may have a problem

Now, in case of scraping _all_ of public useful information. Let's take Stackoverflow as an example of a community-produced content. It gets 90% of its traffic from Google, people show up, read, ask and answer questions (click on jobs and ads to keep SO servers running). If a scraper comes by that inhales all of that data, and regurgitates it under Microsoft's name, giving no credit, no references and absolutely nothing back to the authors of those questions and answers. What is the reason for SO to exist anymore? What is the motivation for people to keep answering and refining the questions? 90% of Stackoverflow will evaporate. The way it's currently set up it's an existential threat to any meaningful content on the web. Maybe there have been some legal precedents to repeal anti-scraping, but this is a very big problem now.

I mean they didn't care about where all that training data came from but that doesn't mean anybody else should (in their minds).

That's them pulling up the ladder behind OpenAI

Their newer chat style endpoint for the GPT-3.5-turbo and GPT-4 models no longer supports this. https://platform.openai.com/docs/api-reference/chat

Might as well make it cheap and easily portable and fine tunable

Be patient.

i.e they can't adequately defend their business with trade secrets.

Patents probably wouldn't work either because the structures are too easily recombined to bypass any conceivable patent that would be enforceable.

That said I think all of this is actually emblematic of a deeper problem with the space which is that none of the LLM stuff recently has been groundbreaking but rather just just continual refinement of a given branch. We aren't seeing evolution, just increasing either number of parameters or quality thereof + additional context. Which is why it was so easy for other folks to make the same progress in similar time periods.

Time will tell if we are about to slam into a local maxima or if someone finds a significant evolution or better yet stumbles on a way to properly combine LLM for context + NLP with traditional AI/logic/expert systems to engineer something that actually thinks and learns rather than regurgitating statistics.

For example, imagine that engines were regulated earlier, because steam engines could blow up and kill people. That's a good reason to regulate steam engines. But then we probably would have never invented other engine types like gasoline engines and jet engines. With that, we'd never have invented planes or flight, because a regulation steam engine would have been too heavy.

The only thing AI regulation would do is hand AI supremacy to China. This is the case with almost every other technological development that we kneecap ourselves with, nuclear power, high speed rail, etc. We waste endless energy on bureaucracy while China is building.

The only reason why propaganda is so effective is because life is so terrible. No one bought USSR propaganda in the 60s that the US was terrible because people remembered growing up without electricity. A majority believe Russian propaganda about the US today because life expectancy in Thailand is higher.