> and biologically implausible
I really like this approach. Showing that we must be doing it wrong because our brains are more efficient and we aren't doing it like our brains.
Is this a common thing in ML papers or something you came up with?
I really like this approach. Showing that we must be doing it wrong because our brains are more efficient and we aren't doing it like our brains.
Is this a common thing in ML papers or something you came up with?
We know there is a more efficient solution (human brain) but we don’t know how to make it.
So it stands to reason that we can make more efficient LLMs, just like a CPU can add numbers more efficiently than humans.
Wheels other than rolling would likely never evolve naturally because there's no real incremental path from legs to wheels, where as flippers can evolve from webbed fingers incrementally getting better for moving in water.
I dunno, maybe there's an evolutionary path for wheels, but i don't think so.
Have you heard of https://en.wikipedia.org/wiki/Bio-inspired_computing ?
I believe human and machine learning unify into a pretty straightforward model and this shows that what we're doing that ML doesn't can be copied across, and I don't think the substrate is that significant.