If the problem is really incompetence, then maybe a viable solution is for the community to create a really great (and well-behaved) OSS crawler. Make it easier for the AI people to do the right thing by making rolling their own crawler the more expensive, lower quality option.
The incompetence hypothesis makes sense (it is often a good explanation). Web indexers like Google had decades to get really good at this, including hoards of people who work on crawlers full time. AI companies are often very young, execute with small teams, and don't consider web indexing their main activity, just something they do in support of pre-training (or maybe serving web results).
Yes, I understand. And sorry to hear that. But I'm trying to understand how it is related to AI. How come this is happening with AI crawlers but not with traditional web index crawlers. If the pattern is so common (which is confirmed by multiple credible sources) there must be some interesting and potentially useful explanation.
I don't understand the current thing about "AI Crawlers". Maybe someone can help educate me.
How is it related to AI? Do AI crawlers do something different from traditional search index crawlers? Or is it simply a proliferation of crawlers because of the growth of AI products?
That sounds plausible. Google have two advantages: 1. They do their own capital allocation, 2. TPUs. That likely means that they can execute more training runs in parallel, experiment more, and release when they hit a crack of gold. Independent labs that depend on outside investment have to carefully trade off experiments. Hence Stargate.
Not sure that's relevant. Obviously an LLM has to learn from something, but it's not a database. I could also program this myself and I don't think that it's an argument against my coding abilities that I have read the source code to many existing interpreters. I can only do it because I not only read but also understood and internalised.
"""
Create a complete reproduction of the ReBirth virtual synth. Create it as a single HTML page with javascript. Use canvas and/or react for the UI and tone.js or something similar for the audio. It should be fully working and playable. Output it as a single HTML page. See screenshot for the UI and description below for how the virtual synth system works.
"""
(this is followed by long spec of the RB-338 which I also generated and is too long to include here, and a screenshot).
So far everything I tried indicates that it is superb at coding. Maybe the best yet (though I understand that some of the published benchmarks dispute that).
Exactly! We don't know everything these machines are good for. Even with older and more established models new things that can be done are discovered many times every day.
And yes, I share the view / feeling that we basically got the AGI building blocks. Models will continue improving, but we can already get most of what we need just by orchestrating the latest generation of SOTA models. Crazy time to be alive!
I bet it can do that if hooked up to an agent system. Rate limits are still very restrictive now in the free API, but as soon as they make it available for more frequent use we'll find out.
How is it "racist"?! It's merely factual. Just before the holocaust, ~85% of Jews in the world were Ashkenazi Jews. The percentage among Jewish immigrants to America is even higher. That doesn't make the culture of Jews from other communities less valuable or worthy of appreciation, but it makes it less relevant for this particular sub-discussion.
They were not so aligned until the 20th-century when Jews from all communities mixed again. I am not disputing that Jewish practices today and for the last two millennia are connected to practices in Judea at the turn of the first millennium and to texts that had been written even earlier. What I'm saying is that strict and regulated adherence by large parts of the population came relatively late. That, as far as I can tell from the research, is not at all controversial.
Religion, and strict adherence to these laws by the general population, is a relatively late phenomenon. It only became as strict as it is now during the middle ages and more so in early modernity. Your Jewish friends are, to a large extent, a product of 18th-century Eastern European Jewish culture and Western / American homogenised understanding of religion.
I love the title "Big LLMs" because it means that we are now making a distinction between big LLMs and minute LLMs and maybe medium LLMs. I'd like to propose the we call them "Tall LLMs", "Grande LLMs", and "Venti LLMs" just to be precise.
I think the author may be misunderstanding what practitioners mean when they say that "evals are all you need", because the term is overloaded.
There are generic evals (like MMLU), for which the proper term is really "benchmark".
But task-related evals, where you evaluate how a specific model/implementation is doing on performing a task in your project are, if not _all_ you need, at the very least the most important component by a wide margin. They do not _guarantee_ software performance in the same way that unit tests do for traditional software. But they are the main mechanism we have for evolving a system towards being good enough to use it in production. I am not aware of any workable alternative.
Agree that "months" is a bit hyperbolic. But note that 20 months or 70 months is still months :D The important point is, it's going to happen very soon in historical context.
I did not say that it is diminished. Maybe I didn't express myself very well, though. If artificial intelligence takes over the universe, its origins in biological intelligence on planet earth will remain just as important, even more, but will be proportionally insignificant.
Mostly UI. That's where I get the most value, because I don't have much experience so I get things bootstrapped and also learn. Sometimes I also use that for writing code (usually in Python) that I could have easily written myself, and can therefore review easily, but it's fun to watch and can be a bit faster.