If AI seems smarter, it's thanks to smarter human trainers
reuters.com
reuters.com
It was a real challenge. I managed to come up with a handful of questions that tripped up the models, but it was clear they stumbled for pretty mundane reasons—outdated info or faulty string parsing due to tokenization. A common gripe among the trainers was the project's insistence on questions with clear-cut right/wrong answers. Many of us worked in fields where good research tends to be more nuanced and open to interpretation. I saw plenty of questions from other trainers that only had definitive answers if you bought into specific (and often contentious) theoretical frameworks in psychology, sociology, linguistics, history, and so on.
The AI company people running the projects seemed a bit out of their depth, too. Their detailed guidelines for us actually contained some fundamental contradictions that they had missed. (Ironically, when I ran those guidelines by Claude, ChatGPT, and Gemini, they all spotted the issues straight away.)
After finishing the project, I came away even more impressed by how smart the current models can be.
That kind of stuff would be great to train on. As long as the answer says something like "If you abide by x framework, then y"
> Many of us worked in fields where good research tends to be more nuanced and open to interpretation
In many optimization problems there are no global solutions. This isn't because we lack good enough models, it's just how things are. And the environment is constantly changing, the targets moving. So the complexity will always exist. There's beauty in that, because what fun is a game when you beat it? With a universe like this, there's always a new level ahead of us.
> Many of us worked in fields where good research tends to be more nuanced and open to interpretation
Why, can't you just tell them "it's not a science, it's more like performance art."
Did you mention this to folks running the project? I would think that pasting the "detailed guidelines" from an internal project into a competitor's tool would run afoul of some confidentiality policy. At least, this sort of restriction has been a barrier to using such LLM tools in my own professional work.
Companies that create data for FM (foundational model) companies have been hiring people with degrees for years
> Invisible Tech employs 5,000 specialized trainers globally
Some of those companies have almost a million freelancers on their platforms, so 5k is honestly kinda medium sized.
> It takes smart humans to avoid hallucinations in AI
Many smart humans fail at critical thinking. I've seen people with masters fail at spotting hallucinations in elementary level word problems.
This is like lamenting that a person who has a doctoral degree, say, in mathematics or physics often don't have a more than basic knowledge about, for example, medicine or pharmacy.
Does this mean that these two sentences are completely unrelated and only happen to exist within the same paragraph?
> Many smart humans fail at critical thinking. I've seen people with masters [...]
I've understood you to use "people with masters" as an example of "smart humans".
You wrote:
> I've seen people with masters fail at spotting hallucinations in elementary level word problems.
I wanted to express that having a master in some (even complicated) subject does not make you a master at [pun intended] spotting hallucinations. To give evidence for this statement, I gave a different, more down-to-earth example of a similar situation.
Q: A farmer has 72 chickens. He sells 15 chickens at the market and buys 8 new chicks. Later that week, a fox sneaks into the coop and eats 6 chickens. How many chickens could the farmer sell at the market tomorrow?
AI Answer: The farmer started with 72 chickens. After selling 15, he had 57 chickens left. Then he bought 8 new chicks, bringing the total to 65. Finally, the fox ate 6 chickens, so we subtract 6 from 65. This gives us 59 chickens. Therefore, the farmer now has 59 chickens that he could sell at the market tomorrow.
--
You'd expect someone who can read/understand proofs to be able to spot a a flow in the logic that it takes longer than 1 week for chicks to turn into chickens.
Rather, I'd assume that someone who is capable of spotting the flow in the logic has a decent knowledge of the English language (in this case referring to the difference in meaning between "chick" and "chicken").
Many people who are good mathematicians (i.e. capable of "reading/understanding proofs" as you expressed it) are not native English speakers or have a great L2 level of English.
If an AI made a similar mistake, people would laugh at it.
You confuse "intelligence" with "knowledge". To keep to your example: there exist quite a lot of highly intelligent people on earth who don't or barely know English.
Imo it's evidence that humans make assumptions and aren't always thorough more than evidence of smart people being unable to perform elementary logic.
The OP there also has a pretty bad riddle (due to a grammatical error that completely changes the meaning and makes the intended solution nonsensical, and a solution that many people wouldn’t even have heard of).
https://chatgpt.com/share/66f9371a-d6a0-8003-b2b5-4af3b10e8a...
I think maybe the original poster is making some sort of additional assumption that the farmer must be selling chickens as meat at the market and a chick wouldn't be sold for that purpose until it's a mature chicken?
(Of course depending on how you interpret the question a chick is a chicken (species) and there's nothing inherently preventing reselling the chicks so I don't really understand why OP thinks the ai answer is clearly objectively wrong. It seems more like a matter of interpretation.)
Anyways this thread is a perfect example of the chaotic datasets that are being used to train FMs. These arguments of whether it’s reasonable to assume a chick could mature into a chicken within a week are happening everyday and have been taking place for years. Safe to say a billion dollars has been spent on datasets to train FMs where everybody has a different interpretation and the datasets are not aligned.
https://chatgpt.com/share/66f890b2-04bc-8002-9724-2deaf3985d...
"Most right" would have been to ask questions about what is being asked instead of trying to answer an incomplete question. But rarely is the human even willing to do that as it is bizarrely seen as a show of weakness or something. An LLM is only as good as its training data, unfortunately.
Regardless I think it's good showing that models are increasingly able to solve these "gotcha" questions, even though I think it's not hugely useful. Partly because I think it's a poor compliant and an easy shutdown.
At some point, we decided that compilers were good enough to convert code into assembly to just use them. even if an absolute master could write better assembly than the complier, we moved over to using compilers because of the advantages offered.
Is ChatGPT an all knowing and infallible oracle? Clearly not. But holding it to a higher standard than we hold other humans to is a unfair test of its abilities.
If you'd said "hens" you'd have a stronger point, but then you'd need to be talking about chicks and hens (and they could still cross whatever adulthood threshold you like within the week, as you didn't specify how young they are - "new" could just mean new to the farmer).
The AI got it right.
We leaned on spoken tradition education to pass down knowledge as written literacy, paper, writing tools were hard to come by until the last century. It was never about the student but the future. Still the same today; one student isn’t propping up reality.
People think learning a linguistic style means discovery of net new knowledge.
It was word problems not rocket science. That tells a lot about human intelligence. We're much less smart than we imagine, and most of our intelligence is based on book learning, not original discovery. Causal reasoning is based on learning and checking exceptions to rules. Truly novel ideation is actually rare.
We spent years implementing transformers in a naive way until someone figured out you can do it with much less memory (FlashAttention). That was such a face palm, it was a trivial idea thousands of PhDs missed. And the code is just 3 for loops, with a multiplication, a sum and an exponential. An algorithm that fits on a napkin in its abstract form.
Why not take away from this that "intelligence" is a word that obtains something relative to a particular society, namely, one which values some kind of behavior and speech over others. "Intelligence" is something important to society, its the individual who negotiates (or not) the way they think and learn with what this particular signifier connects with at a given place and time.
Like I assume you don't agree, but just perhaps if we use our "intelligence" here we could maybe come to some different conclusions here! Everyone is just dying to be like mid-20th century behaviorist now, I just don't understand!
I prefer to use the concept of search instead, it is better defined in search space and goal space. It doesn't hide the environment, the external part of intelligence, or the learning process.
All invented/discovered and formalized by humans. That we found so much (unexpected) power in such simple abstractions is not a failure but a testament to the absolute ingenuity of human pursuit of knowledge.
The mistake is we’re over-estimating isolated discoveries and underestimating their second order effects.
I think it is more like searching and stumbling onto some great idea than pure-brain-ingenuity. That is why searching and social collaboration is essential and why I say we're not that smart individually, but we search together. It's slow, it took us years to get to Flash version of attention, but we get there, someone finds their way onto a major discovery eventually.
It took humanity 200K years to accumulate our current level of understanding, and if we lost it, it would take us another 200k years. Not even a whole human generation is that smart. It's also why I don't fault LLMs for mass-learning from human text. We do the same thing, 99% is inherited knowledge. The whole process of knowledge discovery moves slowly, and over large populations.
Maybe we get an llm to design the next evolution and benchmark how potential improvements fare?
If you have a MLB career at all, you are an elite baseball player.
HN concentrates on the hype but ignores the massive growth in startups that are applying commoditized foundational models to specific domains and applications.
Early Stage investments are made with a 5-7 year timeline in mind (either for later stage funding if successful or acquisition if less successful).
People also seem to ignore the fact that foundational models are on the verge of being commoditized over the next 5-7 years, which decreases the overall power of foundational ML companies, as applications become the key differentiator, and domain experience is hard to build (look at how it took Google 15 years to finally get on track in the cloud computing world)
The reality is that both things are important. It is necessary to know the limitations of AI (and keep up with them as they change), to avoid getting yourself in trouble, but if you ignore the things that AI can do (which are many, and constantly increasing), you are leaving a ton of value on the table.
I might be one of these people, but in my opinion, one should not concentrate on things that it can do, but for how many of the things where an AI might be of help for you,
- it does work
- it only "can" do it in a very broken way
- it can't do that
At least for the things that I am interested in an AI doing for me, the record is rather bad.
Considering it works for so many cases, I think it is naturally to point out the examples where it does not work - to better understand the limit.
Not to mention that practically, I did not see anything proving that it will always "be able" to do something . Yes, it works most of the times for many things, but it's important to remember it can (randomly?) fail and we don't seem to be able to fix that (humans do that too, but having computers fail randomly is something new). Other software lets say a numerical solver or a compiler, are more stable and predictable (and if they don't work there is a clear bug-fix that can be implemented).
At least in the cybersecurity space, most startups have 3-5 year plans to build their own foundational models and/or work with foundational model companies to not directly compete with each other.
Furthermore, GTM is relationship and solution, and an "everything" company has a difficult time sympathizing or understanding GTM on a sector to sector basis.
Instead, the foundational ML companies like OpenAI have worked to instead give seed/pre-seed funding to startups applying foundational MLs per domain.
The industry is still in it's infancy right now, and stuff can change in 3-5 years.
Heck, 5 years ago models like GPT-4o were considered unrealistic in scale, and funding in the AI/ML space was drying up at the expense of crypto and cybersecurity. Yet look at the industry today.
We're still very early and there are a lot of opportunities that are going to be discovered or are in the process of being discovered.
A trend towards domain-specific tools makes sense, though.
I'm much more impressed with the advances in computer vision and image generation.
Either way, what are the startups that I should be looking at?
Search/Replace as a service.
Obviously, there are real ways new model get better too. But if we have diminishing returns, as many speculate, it will take a while for it to be entirely obvious.
First and foremost a chatbot generates plenty of new data (plus feedback!), but you can also commission new high-quality content.
Karpathy recently commented that GPT-3 needs so many parameters because most of the training set is garbage, and that he expects eventually a GPT-2 sized model could reach GPT-3 level, if trained exclusively on high-quality textbooks.
This is one of the ways you get textbooks to push the frontier capabilities.
The project got killed due to management but I still got results on that dataset better than state of the art in 2023 with no human annotation.
The Venn diagram of people who know TeX well enough to write a modular script for table generation with metadata and people who know how to train LLMs has an intersection of a dozen people I imagine.
It seems to me there is a huge amount of unharvested low-hanging fruit here, for example IIUC GPT is not trained on synthetic code in languages other than Python (and maybe JS, I don’t recall).
The role played by humans on the training side is of little interest when considering the technology from a user's perspective.
Years ago, my GF asked me why we bother with judges and juries, given all the uneven sentencing practices and other issues with the current legal system. "Why can't the courts run on computers?" This was back in the pre-Alpha Go era, so when I answered her, I focused on technical reasons why Computers Can't Do That... reasons that are all basically obsolete now, or soon will be.
The real answer lies in the original premise of her question: because Humans Also Can't Do That with the degree of accuracy and accountability that she was asking for. Our laws simply aren't compatible with perfect mechanized jurisprudence and enforcement. Code may be law, but law isn't code.
That problem exists in a lot of areas where people will be looking to AI to save us from our own faults. Again, this has little to do with how training is conducted, or how humans participate in it. Just getting the racism and misogyny out of the training data isn't going to be enough.
someone should write a story about that!
Eg, why does ChatGPT like the concept of harmony so much and use it as a principle for its political analysis?
I think the earliest GPT-3 wasn't too keen on harmony, but I might be mis-remembering
* Data
* Architecture
* Training techniques
* Compute
* Other
What allocation gives you the best chance for success? And how are you defining "best"?
Meta did this with L3. They used L2 to pre-filter the training data, filtering out a lot of junk. They also used it to classify datasets. Then after pre-training (involving lots of compute) they also used almost exclusively synthetic data for fine-tuning (forgoing RLHF when it was surpassed). So yet more compute. The results are pretty good, L3, 3.1 and 3.2 are pretty high up there in terms of open access SotA.
oAI did this with their o1 models. They used lots of compute to have the models go over the space of generating tokens, analysing, correcting, and so on. They RLd the "reasoning traces" in a way. Lots of compute. The results seem to be pretty good, with impressive showings on "reasoning" tasks, math, code, and so on.
The thing is, they weren't the first ones to propose these techniques! What differentiates them is the available compute.
WizardML tried and were really successful with their RLAIF implementation (tho never released code afaik) about a year ago. And while they were connected to MS research, they probably didn't have as much compute available as Meta. But the WizardML fine-tunes on open models like Mistral and Mixtral were pretty much SotA when released, scoring way higher than the creator's own fine-tunes.
In the same vein, but at lower scales is the team behind DeepSeek. They used RL on math problems, in their DeepSeekMath-7bRL model, and that model was SotA at the time of release as well. It took a team of multiple really talented folks to fine-tune a better model (in the AIMO kaggle competition) and everyone except the 1st place used the RL model. The 1st place used the base model, with different fine-tuning. So again, the methods were tried, just at much lower scales.
Yeah, I think compute would be my bet in the short run.
On public tests, LLMs vary between "just below average human" and "genius".
On the hopefully-private test (it's difficult to be sure*), the best was o1, which was "merely" just below an average human, Claude-3 Opus which was stupid, and all the rest were "would need a full time caretaker".
In both cases, the improvements to the models came with higher scores; but there's still a lot you can do by learning for the test — and one thing that LLMs are definitely superhuman at is that.
https://www.maximumtruth.org/p/massive-breakthrough-in-ai-in...
* I could have said the same last year about last year's models, so I'm emphatically not saying o1 really is as smart as this test claims; I'm only saying this demonstrates these IQ tests are a learnable skill up to at least this magnitude of difference.
After using LLMs daily for some time, I have developed a feeling on how to phrase my requests as to get better quality answers.
For example, ensure that it can process the information linearly, like asking to classify items in a list and adding the label directly after the item so that the order remains, instead of allowing it to create multiple lists as the output (which it tends to do by default).
So, at least for me, the prompts are getting smarter.
I wonder if someone tried something like this: "Given user question generate prompt that tells LLM that he is a master in this subject and then ask the question" - something in this allows LLM to precompute what the focus of question will be so it can answer much better and it should be automated for one-off questions.
Starting the article by comparing what is necessary for ChatGPT to work and image labelling is a bit weird
Seems like if then else haha.
I feel as though it 'extracts' some sort of "smartness" out of me (if any) and then whatever intelligence from me becomes part of google gemini
this is why I would never want to pay for using these tools, anything good that comes from me in the chat becomes google's by AI training, which is ok so long as it's free to use
i.e. I won't pay to make their stuff better through my own work
With react you are more productive, my web experience is worse than without a those frameworks.
And LLMs get worse if they are trained on AI generated text. At the current speed I don't know if in 10 years AI is still worse the high costs.
You cannot begin to know that for sure and really makes little to no sense if you think about it.
As with the anti-electron crowd the options are not:
* Electron app
or
* Bespoke, hand-crafted, made with love, native app
The options are normally “electron app” or “nothing”.
Same deal here. Taking away React/Angular/Vue won’t magically make people write more performant websites. I’m sure people bitched about (and continue to) PHP for making it easy for people to create websites that aren’t performant or Wordpress for all its flaws. It’s the same story that’s repeated over and over in tech circles and I find it both silly and incredibly short-sighted. Actually I find it tiring because you can always go one level deeper to one-up these absurd statements. It’s No True Scotsman all the way down.
In the time before all those framework like react the UX was better for me than now.
Less flashy, animated but faster.
I was hoping I could say the same for Gemini, but unfortunately their policy at https://support.google.com/gemini/answer/13594961?visit_id=6... says "Google uses this data, consistent with our Privacy Policy, to provide, improve, and develop Google products and services and machine-learning technologies"
My intuition is that Google don't directly train on user conversations (because user conversations are full of both junk and sensitive information that no model would want to train on), but I can't state that with any credibility.
That's worth literally nothing.
Reputation has far more value than whatever they gain by lying. Besides, they can just say "We do xyz" because <1% of users read the TOS and less than <0.1% care enough to not use the service.
Big companies not only lie, some of them do so routinely, including breaking the law. Look at the banking industry: Wells Fargo fraudulent / fake account scandal, JPMorgan Chase UST and precious metals future fraud. Standard Charter bank caught money laundering for Iran, twice. Deutsche Bank caught laundering for Russia, twice. UBS laundering and tax evasion. Credit Suisse caught laundering for Iran. And so on.
Really it comes down to what a company believes it can get away with, and what the consequences will be. If there are minimal consequences they'd be dumb not to try.
Oh I just remembered a funny one: remember when it came out that Uber employees were using "God view" to spy on ex-partners, etc? For years. Yeah I'm pretty sure the TOS didn't have a section "Our employees may, from time to time, spy on you at their discretion." Actually the opposite, Uber explicitly said they couldn't access ride information for its users.
Note that such reputation risks are external and internal. The reputation reflects on the executive team and there is a risk that the executive team members may leave or attempt to get the unscrupulous employee fired.
> Also Google: "Let's do all the evil things..." ~ heavily "paraphrased" ;)
My "tongue-in-cheek point" is that it seems like corporations beyond a certain point of "filthy-richness" just do as they please, and say what they please, and mostly neither thing has to agree with the other, nor does either one need affect their profits "bottom line" all that seriously much. Most of your typical "mega-corps" are really only able to be affected much by the laws and legal system, which they've been increasingly "capturing" in various ways so that happens very rarely anymore these days, and when it does it's most often a "slap on the wrist" and "don't do that!" sorta thing, followed by more business-as-usual.
You know the old worry about the "paperclip production maximizer AI" eating everything to create paperclips? That's kinda where we're pretty-much already at with mega-corps. They're so utterly laser-focused on maximizing to extract every last dime of profit out of everything that they're gonna end up literally consuming all matter in the universe if they don't just destroy us all in the process of trying to get there.
I'm not arguing that the grocery store saying "fresh produce" guarantees that the produce is fresh. Fresh, like evil, is subjective.
I'm saying that if the grocery puts "All our produce is no older than 10 days" you can be pretty sure they adhere to that and train employees to follow it. "10 days" is not subjective.
https://www.decisionproblem.com/paperclips/ #-the-game