I think some of Chomsky's issues with ChatGPT might be misunderstandings and some of it can be iterated on and improved. ChatGPT _is_ a statistical engine returning probabilities -- but there's nothing stopping engineers from changing how it weighs its probabilities to reject obvious falsehoods "the earth is flat" or show a moral bias "doing this will hurt people, I will not engage in it" without convoluted prompts or torturing underpaid Kenyans to label abusive content.
One thing I would've like Chomsky to engage more in is comparing how people create new "output" from previous "input" vs. how AIs like ChatGPT do it. He talks about "fake thinking" and "real thinking", but doesn't really go into the "hows" of it (ironically, kinda like how ChatGPT would).
I _do_ agree, however, that we're nowhere near AGI and this doesn't bring us closer, but I don't know what would, either.
On the topic of Aaronson's blog post, I feel like he didn't understand what Chomsky wrote about or intentionally mischaracterised it, then gave a super childish response. It's very embarrassing.