Hacker News Reader

Comment by applfanboysbgon | Hacker News Reader

Why should Dutch people be expected to make do with models 99% trained on American/Chinese cultural context and language?

View on HN

Maybe the Dutch really really want an LLM that tells them the truth as straight as possible no matter how harsh - that might be tricky

Understood, but they could fine tune base models on their own cultural context and language. Why reinventing the wheel?

This gets better short-term results for a fraction of the cost, for sure, but what do you when China places an export control banning the release of open weight models? If you don't have your own talent, you're then relegated to using a base model from 2026 or whatever the cutoff date is, forever. That defeats the purpose of a 'sovereign' model made for and by your people.

They could apply the Polder Model of consensus decision making with a mixture of experts.

https://en.wikipedia.org/wiki/Polder_model

Funny, that's what I thought when PewDiePie set up his monster AI rig and what he called a 'council'. Quote:

"PewDiePie has built a custom web UI for self-hosting AI models called "ChatOS" that runs on his custom PC with 2x RTX 4000 Ada cards, along with 8x modded RTX 4090s with 48 GB of VRAM. Running open-source models from Baidu and OpenAI, PewDiePie made a "council" of bots that voted on the best responses, and then built "The Swarm" for data collection that will become the foundation of his own model coming next month."

https://www.tomshardware.com/tech-industry/artificial-intell...

Calling a bunch of LLMs a "council" is just rebranding well known ensemble methods with shrill marketing hype, nothing original or out of the ordinary. Mixture of experts and every other idea in his stack has a literature older than PewDiePie's career.

Yet another attention craving influencer who shilled crypto scams during the crypto bubble and is now marketing "AI councils" during the AI bubble.

He's not a serious or honest person, AI is just what he pivoted to after crypto. That's not innovation; it's attaching trendy branding to ideas that were already old when Marvin Minsky wrote The Society of Mind in 1986, three years before PewDiePie was zero years old in 1989.

The only thing PewDiePie's brought to the table is cleverly optimized YouTube thumbnails designed to attract clicks. The architecture is decades old; only his marketing and shilling is state of the art.

Lighten up dude.

Grow up and stop watching children's entertainment and promoting crypto scammers.

PewDiePie's Shady Promotion of Dlive (Cryptocurrency Scam)

https://www.youtube.com/watch?v=GYyNaQzZTo0

>If you want more concrete and in-depth explanations of what was touched upon in this video, here are the sources I used: Dlive ToS: https://community.dlive.tv/about/terms-of-service/ Dlive/Lino's Shady History: https://steempeak.com/@meno/an-objective-look-at-dlive-s-exi... Dlive overview: https://community.dlive.tv/about/welcome-letter/ 20 Million Chinese Investment: https://web.archive.org/web/20190716124653/https://www.coind... PayPal Suspension: • Is PewDiePie Supporting a Scam? Difference between Lino & Twitch Bits: • PewDiePie & Co-Founder Defending DLive? Is... Refusing to explain how they make profit: https://variety.com/2019/digital/news/pewdiepie-dlive-live-s...

Does what PewDiePie calls a council not resemble a Poldermodel? That’s all I associated with it. Please don’t lecture me about what to watch and drag up negative associations that have nothing to do with the subject (multiple LLMs voting for the best answer) for a simple observation.

Zonnetje in huis.

edit: also, 7 years on and after closure of the platform I cannot find evidence of rug pulls or exit scams associated with Dlive or any of their underlying crypto funding mechanisms, so before judging others on the content they consume maybe don’t take random YouTubers at their word and then don’t look into it 7 years after the fact.

I thought finetuning data can't contradict foundation models, and anything that are inconsistent with the standard LLM American-Chinese split personality would be rejected?

Fine tuning happens on top of pretraining, so of course it can "forget" pretrained defaults when warranted by the new data it's being fine tuned on.

But you have to have more data than used for pretraining for the added knowledge to take precedent over pretraining, no? If that would be the case, you practically contradict the knowledge in the base model.

I mean ... LLMs are sort of an extreme and living proof of linguistic determinism. Their behaviors are dictated almost entirely by disorganized language data, primarily English and Chinese. So you can't just add a language as native primary language in a quick post training, I think. There's no way that it would work.

Oh, it's all fine with cultural context here -- we don't even dub English language movies here because we are that cheap

Reply on news.ycombinator.com