"Management believed Jimmy Intern would be fine to deploy Prod Model Sysphus vN+1; their Beginner Acceleration Divison (BAD) Team was eager to show off the new LLM and how quickly it could on-board a new employee. To his credit, Jimmy asked the BAD model the correct questions for the job. That's when the LLM began hallucinating, resulting in the instructions to 'backup the core database' being mixed up with the instructions for 'emergency wipe of sensitive data'. Following the instructions carefully and efficiently, Jimmy successfully wiped out our core database, the backups, and our local tapes overnight."
You don’t even need to fine tune a model to do this, you just give it a search API to your documentation, code and internal messaging history. It pulls up relevant information based on queries it generated from your prompt and then compiles it into a nicely written explanation with hyperlinked sources.
Everyone also seems to overlook how much time and resources it takes for these models to be trained/fine tuned on a corpus of knowledge. Researches have calculated it probably took OpenAI the equivalent of 355 years on a single NVidia V100 to train up GPT 3. [1] Clearly they used more horsepower in parallel, which is a foreseen problem right now for other reasons. [2]
[1] https://lambdalabs.com/blog/demystifying-gpt-3
[2] https://www.pcworld.com/article/2020375/the-ai-boom-could-cr...