1. Ensuring that it's adequately seeded before anyone can use it.
2. Providing forward secrecy by periodically rekeying ("refreshing the entropy pool", but we should do away with that terminology).
3. Carrying over state across reboots.
(There are four problems if you consider "being performant" a problem on the scale of fundamental design, but it's mostly not).
You wouldn't think CSPRNGs are this simple by reading either the literature on things like Fortuna and Yarrow or by reading the LKML and patch proposals for the LRNG. You'd think it was super tricky to collect "enough entropy" from enough "decorrelated sources", you'd think you'd have to deal with "depletion", you'd think about "high quality" versus "low quality" entropy, and the like. But these issues aren't just not important: they are mostly folkloric.
Generating a practically limitless stream of random bits isn't a hard cryptographic problem. It is an elementary problem. It's the problem that every stream encryption scheme solves. You can simply use stream encryption schemes, like stream ciphers or block ciphers and hashes in CTR mode, as CSPRNGs.
The problem we need to solve isn't upgrading the LRNG or getting everyone to use Schneier's favorite CSPRNG design (the section on this in Cryptography Engineering is one of the book's few warts). It's getting everyone to use the kernel RNG to the exclusion of all else. That's because those 3 problems I listed up there are easy to solve if you're using /dev/urandom (or the getrandom system call), but extremely tricky otherwise.