Yes, if you dare to look into a GC implementation, you might see some truly unsafe code indeed

You can't have 100% safety all the way down. Garbage collection doesn't help you if the VM itself has a memory leak or some driver crashes.