You're implicitly applying different threat models to browser-downloaded vs. alternatively downloaded code.

If the maintainer of JS code make a mistake and fixes it in future versions, there's no need to worry about it anymore either. However, every time I upgrade I'm just as much at risk from someone attacking either my connection or having attacked the server. So to believe it's more secure, you have to believe those things are easier to do in the context of a web application.