> Can you elaborate on whats special in terms of CS, SS and the FLAGs registers in terms of what the microcode must do?
For SYSCALL, SYSRET, SYSENTER, and SYSEXIT, microcode needs to change the values of the registers (including the hidden parts). For entries through the IDT (interrupts, faults, and traps) and IRET, the microcode needs to go through a whole bunch of complicated logic including looking up the requested CS and SS selectors in the GDT or LDT.
> I thought that mapping the kernel into the top of of a userland process address space was supposed to be kind of elegant as it prevents a TLB flush. What is does "move to the front end" in your proposed alternative to this arrangement? This sounds interesting. Might you have any links regarding this?
Once upon a time, the x86 MMU, and probably most other MMUs, was quite simple. There was a TLB, which is essentially a cache that maps (VA) -> (PA, access rights, cache mode). A memory access looks up the virtual address (VA) in the TLB, then checks the access rights and accesses the physical address. In the event of the TLB miss, the CPU falls back to reading the page tables and fills the TLB. When you change to a different process with different page tables, you flush the TLB. You can either map the whole kernel into every process's address space of you can map a small trampoline that switches to the real kernel's address space and flushes the TLB every time you enter the kernel. From (IIRC) 486 onward, x86 would let you tag a TLB entry as "global", meaning it would survive a normal TLB flush and would only be cleared on a special type of flush. Linux uses global entries for kernel pages so that only user entries get flushed on processes switches.
Modern CPUs have a somewhat different arrangement. The TLB maps (VA, address space ID) -> (PA, access rights, cache mode) [1]. On a TLB miss, the CPU will look up the VA in the current process's page tables and add a TLB entry with the appropriate address space ID. On x86, the address space ID is (VPID, PCID) [2]. This means that you can sometimes switch between processes without a TLB flush -- you can change the page tables and the address space ID and leave the TLB intact.
I would like to see CPUs arrange it just a bit differently: have a separate address space, complete with a separate tag bit, for user mode and kernel mode. Then user accesses effectively look up (VA, "user", ASID) and kernel accesses effectively look (VA, "kernel", ASID). This would add almost no overhead. As an added benefit, when the front end decodes an instruction, it would know the whole TLB key (i.e. VA, user/kernel, and address space ID) and could decode user and kernel code differently. This means that kernel-mode instructions could be rejected in the front end. More importantly, if you imagine the user/kernel bit as being an extension of the VA, a transition between user and kernel mode is functionally the same thing as an indirect jump and could use all the optimizations, including pipelining, that indirect jumps use.
[1] It might be organized internally as (VA) -> (address space ID, PA, access rights, cache mode), but it doesn't really matter.
[2] VPID identifies the current VM guest, if any, and PCID is used at the discretion of the kernel. Linux may soon use PCID to allow a few processes to coexist in the TLB at once. grsecurity uses it for a totally different purpose.