thats interesting, and maybe beyond my current knowledge, I will certainly look into it.
About the entropy being a property of a distribution, thats totally correct and I need to fix the post. Thanks.
You'll probably also want to look at uniquely ergodic systems. Those are interesting in the way that they behave 'random' for every input, one example is if you keep multiplying a number by anything other than a power of 10 and divide it by a power of 10 so it's between 0 and 1. Turns out you get the same distribution of results every time (eventually).
And while you can define an entropy rate without I think ergodic theory also helps inform why that is the correct generalisation. Your current article only considers the case where each digit is independently random (it's clear why, it's a lot simpler and all that information theory tends to talk about).
- Walters, P., An Introduction to Ergodic Theory, Graduate Texts in Mathematics, 79. Springer-Verlag, New York-Berlin, 1982
- Billingsley, P. – Ergodic Theory and Information, John Wiley and Sons, 1965.
You may need a refresher on measure theory, or just ignore that part I suppose