I feel like you're missing my point. Bayesian reasoning of that nature is totally reasonable and isn't something to be avoided just for the sake of avoiding it. What is is, is useful as a guide for where to direct energy and focus. And what it is is faster than sitting around playing with your pud waiting for review for a journal submission.

Again, what's going on now is a form of peer-review. Double blind? No, but that's not really relevant in this context anyway.

ML is really more of an empirical field in this day and age and people are going to read pre-prints on ArXiv, and use various Bayesian weighting schemes to decide what to direct time and energy towards. This process complements, not replaces, the kind of formal peer review you're demanding. There will still be plenty of room, and time, for that stuff, but there's no real reason to wait for all that to happen before starting to look into something.

Is peer review generally done double blind? Who in the field has not heard of hinton s capsule proposal that would be a blind reviewer? Hell I'm not even in the field and I've heard of it.