This type of theory seems to only handle the common case, but the universe is full of edge cases...
Current explanation for existence of galaxies is those are quantum fluctuations that grew large. So apparently fluctuations can explain everything if you do enough of hand waving in between. I don't think any particular quality of galaxies as we see them today can be traced to specific quantum property. But that doesn't stop us from believing in that explanation.
For any explanation X there will always be the question, "now what explains X"?
http://www.av8n.com/physics/thermo/entropy.html#sec-s-not-kn...
http://www.av8n.com/physics/thermo/entropy-more.html#sec-s-c...
http://www.av8n.com/physics/thermo/s-relevance.html#ch-s-rel...
Speaking of, I'm somewhat puzzled, you said that "under strict Bayesianism knowledge can't ever decrease", but how does it then deal with something as simple as card shuffling ??
That after shuffling you know less about the stack of cards (that used to be at least partially revealed) is a fact that our model must follow or fail at being relevant.
It's been hard to find out more information about this, but I did find some :
https://www.preposterousuniverse.com/blog/2015/08/11/the-bay...
Same thing : "And the Bayesian Second Law (BSL) tells us that this lack of knowledge — the amount we would learn on average by being told the exact state of the system, given that we were using the un-updated distribution — is always larger at the end of the experiment than at the beginning (up to corrections because the system may be emitting heat)."
Though I also did find another interesting thing :
http://jakevdp.github.io/blog/2014/03/11/frequentism-and-bay...
"in a strict frequentist view, it is meaningless to talk about the probability of the true flux of the star: the true flux is (by definition) a single fixed value, and to talk about a frequency distribution for a fixed value is nonsense"
But that's pretty much the case in statistical physics ! A macrostate is actually NOT a state (=microstate), but a probability distribution !
"For Bayesians, probabilities are fundamentally related to our own knowledge about an event. This means, for example, that in a Bayesian view, we can meaningfully talk about the probability that the true flux of a star lies in a given range."
So looks like statistical physics are already at least part-way between Frequentist and Bayesian ??
Also, this one sounds potentially interesting, but sadly, paywalled...
https://www.researchgate.net/publication/363313577_On_revisi...
"Could instead explain" is not a definitive statement.
Their headline is "Controversial new theory of gravity rules out need for dark matter"
"Controversial new" tells me that this isn't settled science, nor is the Guardian trying to say otherwise. "Rules out" by itself, out of context, could imply proven science, but not within the context of the headline and byline.