Well it isn't really a unit test if you don't, is it? If you don't mock it then you're not only testing classes under unit tests, but also C's methods. And you're doing that not just in one place but in several different places in your testbase. The same methods. Over and over again.
In reply to your original question, its not extremely bad if you can't unit test that class alone , but in this case you wouldn't be able to unit test anything which uses it either.
Edit: I see what you say, but I think that effort we spend on decomposing app into perfectly isolated unit tests is far greater then the effort of identifying and troubleshooting where the less perfect test failed.
If you really want to have different cache implementation for testing purpose, use C as just an instance storage, and return reference not to self but to cache interface. Put a switch inside static C.getX() that returns IX. Inside of switch statement see if System.getProperty("isTest")=true and return XSimple (which implements IX), otherwise return XReal (which also implements IX). Few lines of code, transparent and debuggable.
But then I'm not testing the code that uses the cache, I'm testing the cache plus the code that uses the cache. Which is a valuable test but a separate one.
Regarding your proposed implementation, that sounds like basically what I proposed here: