I don't think a contributor can unilatary withdraw their code from a code base unless there is license change. Then it wouldn't be about "withdrawing" their code but refusing to allow it to be distributed under a different license than the one that applied when they submitted it.

So if Linus wanted to relicense the kernel under a different license than GPL2 he couldn't do it by himself without tracking down all the contributors and getting them to agree to the license change or removing the code in question which, for someting as large and with as many contributors as the kernel, would be basically impossible.

The CoC is completely different. It's not a license change but a process change. If a contributor doesn't like a new CoC or a new processs they can decide to no longer contribute in the future under that process, including no longer maintaining "their" code already in the kernel but they can't force anyone to remove it.