If you train a small model in another domain it will begin losing capabilities in the former domain. This is effectively the sigmoid problem.
Although I will admit that if we discover a higher information density algorithm that it might change, but not by a substantial amount to where "super intelligence" in 1gb would be possible.
There is undoubtedly a limit somewhere (there is only so much you can pack into a given size) but it's really not particularly clear where that limit is. I don't think it's superintelligence - that much I agree with you - but I think "We already have a 1gb model that is as capable as it will ever be" is strictly false.
It's like comparing two person A and B of similar intelligence where A is smarter and B is a genius at signing, but signing was not on the test so person A won.
The rest is just the general reality I am sure you are familiar with:
- https://en.wikipedia.org/wiki/Catastrophic_interference
- https://en.wikipedia.org/wiki/Fine-tuning_(deep_learning)
- https://en.wikipedia.org/wiki/Entropy_(information_theory)
Or we've lost random noise, or redundancy not captured by the entropy measurement.