Analyzing the websites that are used to train AI chatbots

I never saw the actual article, but the quoted figures were not just plausible and reasonable but also internally consistent throughout several exchanges. It produced exact figures for key (i.e. largest) submarkets and growth estimates for smaller ones, in the same way that I would expect the original content to be structured.

I would bet with a 95%+ confidence that it parroted the actual contents. $2000 for that extra 5% tho.

How did they patch all the illegal stuff ChatGPT said at release?

If you want a model to, say, never draw pornography or write a Hitler speech, you don't do that by excluding "bad" content from the foundation model but rather you tackle that in later stages of training, particularly in this phase

https://en.wikipedia.org/wiki/Reinforcement_learning_from_hu...

Having some of that in the foundation model actually helps the model "understand" what it is it is not supposed to do. The whole point of the foundation model is that it generalizes from the examples you show it later to similar examples it saw during "pre-training" and a certain about of offensive content will help it learn what it is offensive much quickly later.

Imagine how your comment would come off if your guess was wrong. That's how it comes off.

While we're at it, imagine how awesome an ordinary search engine could be if we would give it access to all the info behind firewalls.

this is what Microsoft is trying with Office 365 GPT approach, where they are siphoning training data from every company and offering graphs and summaries in return

Just saying, if ChatGPT says it knows something it doesn't necessarily mean it knows something, or that said thing is even real.

The intersection of people who can host it themselves vs those who host chats, video conferences, etc events is very very small. Once they start trying to figure out the SSH host keys and TLS firewalls, they will become very quickly overwhelmed and out of their depth. They would likely turn to an expert at this point, but it has become far cheaper and easier to just turn to a third-party service to handle this. The very service you ask them to dismiss for the troubles of hosting themselves.

On the other end of the spectrum, I have dedicated systems to hosting my own chats, emails, source code, and more. My user count is super low. Seems that not everyone wants to come to my copy-cat Gitea server when "all the cool kids" are on GitHub

Because they act as a portal and distribution service. The same reason freelance journalists don’t print their own newspapers but instead publish via either a syndicate or directly through papers. The paper has the brand, distribution, ability to monetize and pay journalists for their content, layout staff, etc. Similarly say YouTube is YouTube in the same way the NYT is the NYT - people go to YouTube for content, it provides hosting and distribution as well as ways to promote your content, monetize it, they provide a UX and technical infrastructure, etc.

There is nothing stopping anyone in any RFC or other technical standard from hosting their own content on equal footing with YouTube. People do host stuff on their own machines and sites. You’re just not directly aware of it because it’s basically impossible to find relative to the ease of content discovery on YouTube. No one is getting rich or even paying their bills because on self hosted because beyond having no funnel monetization of content is actually kinda hard.

Case in point, politics aside, Parler and Truth Social were basically formed based off similar thoughts. Why do we let these big companies make content moderation policies? I’d like a place where I can be abusive and racist or whatever without fear of censorship. I’m after all entitled to hold those views and say them. But, it turns out, it’s hard to provide the quality of service of say Facebook on the premise of absolute freedom - especially when it’s so toxic most people are turned off. This gets to the point: you can host yourself, but if you want to build a “free YouTube” you probably need to ask why did YouTube, a for profit entity, take the positions it has in the market? Because the market rewards them for making a family friendly middle ground platform that provides easy access to personalized content and a simple experience for content creators to distribute and monetize their content. Do that on your own systems, and you’ll probably do alright - except one thing. They have the most users. Acquiring users, or in news papers, subscribers, is not easy or cheap.

So, that’s why things are the way they are. But you’re absolutely able to self host. Just don’t expect everyone to show up at your random website.

> Why do so many of us, including the biggest influencers in the world, spend so much money only to post all of our content on the Big Tech’s sites and infrastructure? You give all your social capital to them too, and they turn around and sell it back to you (access to your own followers) and can deplatform you. They encourage you to pay them to attract eyeballs away from other videos, and vice versa, in a zero-sum game on their own platform.

Because we're lazy and Big Tech solves the immediate, short term problem.

Glad someone mentioned this!!! It is one of good reason to self-host! And only send sneak peek to social media sites. Stallman has a recommendation similar to that on what to post to FB.

Only use social media as a notification/publishing/advertising system, post important stuff on your own site.

Because we're not content with just a private little soapbox in our private little space talking to the few hipsters we can convince to drop by; we are vain and want to talk to the crowds, and the crowds are all over /there/.

> I ask again: why not simply host it ourselves?

Cost, ongoing maintenance, convenience, existing apps are good enough, existing infrastructure is good enough, I could probably go on for a while.

> I ask again: why not simply host it ourselves?

Because the average person and the average HN user are different sets of people. The typical person who wants to start a forum on, say, trout fishing or stamp collecting has only a vague notion of a server is, much less how to set one up and maintain it. Even if they did, they'd have little or no interest in doing those things. It's far easier for them to just make a Facebook group or whatever.

Self hosting is hard.

Self hosting with channels, voice chats and permissions is harder. Discord also had one unified account to join different channels, private message, etc.

Self-hosting is too hard to do unless you're free to fight spam 24/7.

As it happens, I am in the process of building something pretty close to what you describe. Here is an extremely partial list of reasons people don't do it.

- Written content: because network effects are real. Also because building a blog in 2023 that doesn't have awful look-and-feel or seem like a spam site is remarkably hard. Substack is full of chuds, but you can firewall off from them most of the time and at least you can write things instead of writing code to write things. WordPress is maybe the least bad option today if you want interactivity (which I would assume you do, what with the "community" bit and all), and getting things like SSO (for playing nice with all those other things in that gated community) costs money for dubiously-supported plugins that claim to do things they might not actually do. Ghost has a ton of weird limitations that are often claimed to be by design, so unless you want a plain white (or, if you're bold, black) site that's Just Text and maybe a few images, it's not great either. Static sites--community, no bueno.

- Podcasts: this one is not "because network effects are real", this one's just "because open-source podcast hosting software basically doesn't exist and what does is bad at doing the job." Meanwhile, anchor.fm is free.

- Chats: because network effects are real. And because the options are mostly unpleasant. IRC is difficult and mobile-unfriendly. Mattermost is buggy. Zulip is pretty good, but now you're a sysadmin and most people on Discord just want to shitpost about Pokemon. (I am not doing this, either, because there are limited hours in the day.)

- Videoconferencing: because network effects are real. (You may be detecting a theme here!) And because self-hosting Jitsi is an enormous pain in the rear. And, again, now you're a sysadmin, and most people on Discord just want to shitpost about Pokemon. (Might do this one, just for latency's sake when recording stuff. Haven't decided yet. It is hard. It is also pretty expensive to run.)

- Video content: because network effects are real. Also because I don't own a CDN, and because YouTube is $free dollars per minute watched. Even my former employer, whose stack I quite like, is not $free dollars per minute watched--nor should they be! They're targeting businesses. (Needless to say: definitely not doing this one.)

Gated communities are great, but somebody needs to make achievable these things for normal people, and they're probably going to want money for it. And while I've written plenty of open source software, the stuff I'm building won't really be portable to somebody else because I'm not making decisions for others' consumption. I'm making decisions for exactly my (relatively high) tolerance for system administration and exactly my (relatively high) ability to fix things when they break. I'm going to open-source parts of it, but I can only open-source parts, and they'll still depend on my CMS of choice, etc. etc. unless I spend time not building my thing and instead building abstraction/indirection layers for hypothetical other people.

If you want people to do similar, maybe it should be you who goes and builds the path to doing it?

For niche topics you probably can.