Agreed, it just feels inadequate to me, too. Information theory is a hammer ... and now everything we can think of (~ information...) looks like a nail.

Another hint that maybe the big Turing machine isn't quite cutting it might be the experiments showing that gravity isn't quite like 'just' an entropic force but seems to behave genuinely different.