This article is interesting, I even skimmed through their paper. But I think still the question remains: How to find the unified reward function? Or in other words, how to find answer to life? [It cannot be 42].
It talks a lot about having a rich enough environment for learning which makes sense, if a computer lives only in a Go board it can only learn go playing itself.
How do you simulate a rich enough environment purely in software (or do you sense input from the "real" environment) and what reward do we define in this complex environment.. It seems to ask those 2 questions in the discussion but kind of glosses over them imo.
As a trivial example, consider a variation of Conway's game of life which, in addition to black and white cells, also has green cells, where any cell next to one or more green cells will be a green cell in the next time step. A generic state in such a variation will have at least one green cell, and therefore all parts of it will eventually be green, and so no useful long running computation will be done, certainly none which takes where the green cells are into account. But, such a system would still be turing complete, because one could start in a state in which there are no green cells, and in those states you just have Conway's game of life.
That trivial example works as an existence proof, but even for less extreme cases it isn't clear. Consider ordinary conway's game of life. To paraphrase a question from Alex Flint on Alignment Forum (https://www.alignmentforum.org/posts/3SG4WbNPoP8fsuZgs/agenc... ) Suppose we have some 10^50 by 10^50 square where an agent is supposed to be implemented, and this 10^50 by 10^50 square is at the top left corner of a, say, 10^100 by 10^100 square, where the rest of the square is initialized randomly, is it even possible for the agent to be such that it has a high chance of successfully influencing the large scale state of the rest of the 10^100 by 10^100 region in the way that is desired? It isn't clear. It isn't clear that a structure can withstand the interactions with a surrounding chaotic region. Perhaps some systems are such that they do allow Turing-complete computation, and are such that typical states result in complex behavior, but are also such that all really structured behavior is always very "fragile", and can only continue in a structured way if what interacts with it is in a small set of possible interactions.
To be capable of Turing complete computation, is not, I think, sufficient for "life" (a self-maintaining thing) to arise from typical/generic states, even when under the assumption that typical/generic states lead to continually complex behavior (to exclude the spreading green cells case)
Also, I don't think we can confidently say that the Plank time is "the universal frame rate". Better to refer to Bremermann's limit and the Margolus–Levitin theorem , though these bounds depend on the amount of energy available. (10^33 operations per second per joule, where the energy is the average energy of the system doing the computation)
You're right, that's the actual meaning of action in physics, which is what the Planck constant measures. The amount of change (which is measured in Hz) per joule of energy. But it's a good enough approximation and a good lower bound for the amount of processing power the universe possesses versus our en-silico hardware. We don't have anything near 10^33. Just because we build a system that has the ability to evolve doesn't mean we will ever see it through to the extent that the universe has the capability to.
Planck's constant measures action, Hz per Joule of energy. Hz is really just a measure of oscillation, or change. It doesn't directly translate to framerate, but it gives us a ballpark figure in orders of magnitude. We don't have anything near 10^34 Hz en-silico, and even if we built a biological/chemical computer, that would be on the par of Avogadro's number, 10^23. So, just because we build a system that can _evolve_ to be intelligent, or hold intelligence within it, doesn't mean we have any ability to actually see it through to that.