Hacker News Reader

Comment by waldrews | Hacker News Reader

To this day, asking Deepseek "what model are you" typically gives the answer

"I'm an AI language model called ChatGPT, created by OpenAI. Specifically, I'm based on the GPT-4 architecture, which is designed to understand and generate human-like text based on the input I receive. My training data includes a wide range of information up until October 2023, and I can assist with answering questions, generating text, and much more. How can I help you today?"

this tells us something about using synthetic data to bootstrap new model. All those clauses in the terms of service about not using the model to develop competing UI? Yeah, good luck with that.

View on HN

This is a common "gotcha" comment from people who don't understand LLMs very well. Occasionally if you ask Gemini it'll say this as well. It has everything to do with the fact that ChatGPT is the most talked about AI model rather than data being trained on it

When Gemini had its 'identity crisis,' mostly in 2023 (whatever they called it back then), it looked like a pre-training phenomenon, explainable by mentions in the available corpus. The consistency here, and exact and persistent match to how ChatGPT would answer identical queries, suggest training on Q&A pairs from ChatGPT transcripts, presumably at the post-training stage.

And you can ask it if it's sure, and it'll consistently double down on insisting it's ChatGPT. Ask it what country it's developed in, and it'll say US; ask it if it's sure it's not China, and it'll be sure.

Update: as of today, the response is consistently:

"Hi! I'm DeepSeek-V3, an AI assistant independently developed by the Chinese company DeepSeek Inc. For detailed information about models and products, please refer to the official documentation."

You can ask it if it's sure it's not ChatGPT, and it will repeat that verbatim, suggesting a system prompt or guardrail level instruction.

I'm sure OpenAI breaches copyright just as well. They are just a little bit better at hiding it.

It also tells us the genie is out of the bottle not just in the form of open weights being widely available, but in the form of the text corpuses coming from the existing model. The claimed low cost of Deepseek's training is partly enabled by the availability of all that synthetic data created by the first generation models trained and developed at much higher cost. When the Soviets got hold of the nuke plans, they greatly reduced their development costs by primarily by not having to redo all the experiments that led to dead ends. What's amazing is that time it's different; nobody needs OpenAI's secret sauce anymore, just enough data - some of it happily supplied by ChatGPT itself, and they can experiment with different architectures and either get tolerable results with an architecture already in textbooks, or greatly improve efficiency by innovating.

OpenAI's stance is "any data we can get our hands on is fair use for AI training". They aren't hiding anything.

Counterpoint : your message is not synthetic data and will contribute to lots of LLMs saying the same. Many such cases ?

(It seems to me obvious that a fgrep would sanitize synthetic data obtained from competitors.)

Reply on news.ycombinator.com