Couple things:

First, perf you mention is from fundamentally CPU intensive workloads, not a sparse async environment like a typical server.

Second (far more important) the reason we do this is type safety. We don't want types to break at the handover or to maintain them twice. We eliminate a huge category of issues and mental overhead.

This maintainability and simplicity is worth more (to me at least) than perf.

As a bonus there's also directly shared utils, classes etc. which again can be reused across client/server but more importantly stay in sync.