Here's a few things:

1. JVM cannot do unsafe code. In the CLR, you can create unverifiable IL and do pointer manipulation, etc. Raw memory can be free accessed however you like. That's a handy interop and performance feature.

2. JVM doesn't support structs. Again, performance difference. (Although, sometimes the CLR's JIT hasn't emitted the best code for structs.)

3. The CLR has generics. In Java, generics are implemented via erasure. This has an impact on performance and reflection.