Nobody's saying give up. I'm saying if your solution needs a trillion parameters and a power plant, and biology does it with a few billion neurons and a sandwich, that you're maybe on the wrong track. This is not an engineering gap.
Maybe there’s also a hardware component to it, but there’s very little point in trying to optimize the hardware to work with a poor algorithm. Once we discover an efficient way to train and infer, then it will be worth hyper-engineering the hardware.
Yes, we need better architectures and algorithms. We can point to massive advances in software as well in many spaces, including in LLMs (e.g. compare early GPT versions with current smaller open models), but the hardware comparison came from further up-thread.