Rasterization works per triangle - which means that each CUDA thread is working on a triangle. Which means that by definition, each thread reads/writes to a completely different place in the Z-buffer and reads from a completely different place in the shadow buffer. Hence the abyssmal speed I experienced with my CUDA rasterizer, and why I switched to CUDA raycasting.