> I think ChatGPT is just kind of bullshitting at this point. It doesn’t have an answer, and cannot think of one, so it’s just making shit up at this point [...] But instead it’s [overconfident] in its own capabilities, and just makes shit up. It’s the same problem it has with plenty of other fields

If anything, the article demonstrates it can write code, but it can't thoroughly reason about problems it hasn't been trained on

So when saying something like "Its possible that similar problems to that have shown up in its training set." as a way to dismiss any scintilla of 'intelligence', how many of these articles reduce to a critique e.g. "Can a Middle Schooler actually understand dynamic programming?"

Like, what is the actual conclusion? That a software model with O(N) parameters isn't as good as a biological model with O(N^N) paremeters? That artisans need to understand the limits of their tools?