I remember stumbling upon an early discussion about this [0] a bit ago in the EleutherAI discord when searching for discussion about a paper; I'm glad to see it's turned into something public.

[0]: https://discord.com/channels/729741769192767510/730095596861...