People paid to train AI are outsourcing their work to AI

If GPT4 really is 8 230M models, the next bit for us will be a few ~1-5M models that swap in for whatever you want to create, or talk about, or what have you

Imagine a model trained just on English football for the purpose of having a good time in the pub that is used when the topic changes to it. I bet you could pass on the dailymails sports page if you add some "u"s into your words.

Or a model finetuned specifically on the library you're trying to debug, maybe even specifically in combination with other tools you're trying to put together.

"Transcribing" is a better word or maybe "manual OCR".

The ones they sent to AMT were just awful, I would say 2/3 of them were impossible to transcribe with complete accuracy and would take a lot of time to do it, I'd be afraid of getting kicked out for making mistakes on them.

The hardest problem I had when I ran a lot of HITs were people that I called "Superturks", generally these people were very fast but the quality of the work was as low as they could get away with. If I kicked them out I could raise the quality of the work but it would not get done so quickly. There's the possibility of coaching them to do just a little bit better (would be happy to pay a bonus) but it is no so simple to do in that context.

The argument here is the LLM generated text is now going to enter the corpus, muddying the waters and reducing the quality.

The initial data set was essentially created by undiscriminatingly crawling the internet. This worked reasonably well because up until now most of the internet was - in one way or another - created by humans. This is no longer the case, as LLMs are incredibly attractive when you want to create spam.

Anyone who wants to get any general dataset past 2022 will have to deal with the reality that a significant amount of crawled content will have been written by a LLM and is therefore essentially unusable for training. Facts are useless when they have been hallucinated!

There is consensus that almost all contemporary LLMs are undertrained. See, for example, the Gopher, Chinchilla, and LLaMA papers.

Larger models are easier to train, and there are diminishing returns when you keep training. Thus, to claim SotA performance, researchers tried to optimize for the best performance, given a certain training budget.

The best performance within a certain training budget is achieved by training HUGE models on VAST datasets, for a relatively short amount of time. Almost all models could profit from simply training for longer, but the cost/benefit isn‘t there, if your goal is to achieve the best performance.

This is also why distillation and quantization work so well. Models with more and larger weights are easier to train, but ultimately don‘t utilize all that capacity.

Recently, researchers have begun focussing on inference – rather than training – budgets, in order to make using these models actually viable and profitable in practice.

I.e., what is the best performance we can achieve within a certain computational limit at inference time?

This is mostly done by simply training with more data, for longer.

A LLM will parrot what it learned without any understanding, which is unlike a human.

According to this theory we have a built-in faculty for things like language (Chomsky) and how to interact with the world. We haven’t bootstrapped ourselves; that would mean that the Blank Slate theory is true.

Could a human tribe who was raised on a different planet (with completely alien concepts) survive? That’s unclear. Maybe we have evolved to only be able to learn Earth-concepts.

Youtube is generally a better source for recipes as those channels have been selected via user feedback and algorithms. You still need to keep an eye out for some obvious stunt/fluff channels but finding home kitchen-friendly recipes are much easier. Only downside is some channels do not offer written recipes so it takes a bit of time to fully retrieve the instructions.

That's the root of the problem right there. Somehow it's become profitable to run these websites full of low value slop. Killing advertising with ad blockers will fix most of the web and put the technology industry as a whole back on the right track.

What I have found, and I've been using Bard over ChatGPT because Bard seems to be a bit smarter at first glance, is these tools are powerful but limited. They can augment a workforce but only a fool, soon to go out of business, would use them to replace a workforce.

It seems we are not reading the same HN.

LLM> Forgive me, I made a mistake. Pour six cups of boiling water into the microwave.

There's a lot of fuzziness in that particular test around how it's interpreted as "A worker who is subject, either as a matter of contractual right or in actual practice, to the type and degree of control a business typically exercises over employees would be considered an employee."

But there's also an exceptions for "Single Engagement Events" that would cover most things like handyman or home contractors. https://www.nolo.com/legal-encyclopedia/exempt-job-categorie...

But in California, once you engage someone in a recurring manner with hours and work that quacks like a full time job, then you need to look into it (there's a ton of special cases and other exceptions too).

It's the same reason why if I pay my nephew $5 to mow my lawn, I'm not violating minimum wage laws.

The law in the United states is that employers may not tell contractors how to do their job or else they'll be considered employees. I'm not an employer.

Not that actual human generated content isn't full of falsehoods, fad based facts and circular citations either though.

This doesn't seem like a scare. If GPT and the like start outputting increasingly nonsensical outputs (which, a lot of the time, that is the case already) due to tainted inputs, oh well?