PRNGs are complicated, and sometimes we don’t find out flaws in them until long after they are widely deployed – although test suites like U01 Big Crush and PractRand should really help that situation out. Part of the problem is that some people just keep doing what they’ve always done and don’t want to see that the building is on fire.
I took a Computational Physics class in 2011 that was still advocating Mersenne Twister as state of the art, despite the widely publicised flaws and much better generators being around at the time. Others were fine with just using a Linear Congruential Generator with decent constants. We did have one class that encouraged us to plot RANDU and see the pattern, but I recall the resolution was to use an LCG with better constants.
I spent a while looking into PRNGs then, particularly the test suites available then. At the time, I think two approaches stood out as being fast and suitably random: xorshift plus weyl function (apparently now used in CUDA toolkit) and AES with reduced rounds.
Sometimes it’s important to recognise that there are possibly people still doing science with Linear Congruential Generators, and we can do a lot better even if a PRNG doesn’t pass every test. These test suites are designed to catch PRNGs out. It’s an adversarial situation, where better tests lead to better generators that lead to better tests – and it’s worth pointing out that some tests are so sensitive that they will give false positives by chance alone.
> So, for simulations where predictability etc is not a concern, what's the state of the art? And since I'm into microcontrollers, what's a decent choice when you only got a 8-bit CPU and very limited space (both code and memory)?
I don’t think this is going to be the same generator as used in MCMC experiments, but I would imagine a xorshift generator could be implemented in 8 bits with a bit of thought or research.