You linked to a playground that's in debug mode. If that's what you're measuring, you're absolutely going to be much slower. You need to compile with optimizations.
Additionally, while I don't have the time to dig into the specifics of the assembly here, your Rust code doesn't do the exact same thing as the JavaScript code. valand's version is a more direct translation. While it appears you tried a few things to optimize the function in this Rust version, ..= can be a performance hazard in microbenchmarks like this. I have no idea if that's relevant, but it's a good example of why it's important to actually show your code, how you compiled and ran it, and how you tested it.
But disregarding that, I still don't see how you're getting 400 seconds. You said you're testing up to 50,000 primes. In the playground that shows 0.02 seconds for me in debug, and 0.00 seconds in release.