The argument for Rust: 1. If I'm going to rewrite - why not go the whole hog and do it in rust - and thereby get a 20x improvement, not 2x. 2. By using Rust we can have add more customisation with virtually no performance impact, with Python that's not the case.
Of course we could make Pydantic faster by removing features, but that would be very disappointing for existing user.
As mentioned by other commenters, your comment about "constrained" does not apply.