ege_erdil

that's not the relevant data i'm talking about

how much real-world data do you think went into the evolution of the human brain and all its learning algorithms?

having 40 years of experience building software gives you no more insight into that than having 40 years of experience using language gives you insight into where your language skills come from

we don't think it's just around the corner

then i would disagree

what makes you sure about that?

how do you think humans cross that chasm?

we're not sure if the actual data exactly matches our reconstruction, but one of the authors pointed out to us that we can exactly reproduce their scaling law if we make the mistake they made when fitting it to the data

what they did was to take the mean of the loss values across datapoints instead of summing them and used L-BFGS-B with the default tolerance settings, so the optimizer terminated early, and we can reproduce their results with this same mistake

so our reconstruction appears to be good enough

we didn't eyeball the graph, there are more accurate ways of extracting the data from a pdf file than that

we did ask for the data but got no response until we published on arxiv

what is supposed to be "salacious" about the abstract?

one of their three approaches does not replicate and it's because of a software bug in the optimizer they used, i don't know what else we were supposed to say

we did and gave them a two week grace period to respond, but they only responded to us after we published on arxiv

also, we didn't reconstruct the data using a ruler, you can automate that entire process so that it's much more reliable than that