Is memory really an expected bottleneck in the long term?
It sounds like you're viewing "OOM" as "Out of Memory", when the parent meant "Order of Magnitude" - meaning that a baby's "training data" is several Orders of Magnitude more than what the DenseAV model from the paper used.