> Can we agree that consciousness is outside the realm of mathematics? That, while there could be a formula that determines what our conscious experiences should be, the fact that we actually experience them, as opposed to them just existing in some abstract sense, is not mathematical?
There's certainly a difference in our feelings, I'm not sure if a mathematical description of experience has to be incomplete - but I agree that all attempts of doing so have been complete failures.
> So if the behaviour of our universe can be described entirely mathematically, isn't it weird that it physically contains this comment about how we know that we're in a universe that contains non-mathematical stuff? It's of course possible. But I find it strange.
It is, but this kind of self-referential process isn't unheard of, in fact we are currently consciously discussing consciousness. A popular sentiment in some sci fi circles (fe. Babylon 5) would be to posit that life is the universe's attempt to become conscious of itself. If true, it would do worlds for us to regain the self-importance lost from Galilei and Darwin.
> Isn't this necessarily the case for anything to exist? Is it more strange for there to be exactly one thing without a cause (the initial conditions of the universe), or for things without causes to just be a regular part of the universe we live in?
You are absolutely right, I think we are generally much more used to the "first mover" concept, since it is the basis of most, if not all religions. Personally I find the concept of truly random events to be very unsettling. A possible out could be that the causality graph is not acyclic, that is, that future events can inform the past, and that for example the "last" thing to happen in the universe "caused" the first.