Seems like a nice paper. Reminds me of work by Merhav and others on phase transitions in estimation and inference, although I appreciate the perspective taken in this paper.
I also wonder what happens as you generalize to non-closed-form generating models.