How big of a difference it makes obviously depends on your OS and how you run your browser.

I guess your choices are

1. Educate yourself on the nitty-gritty and make an educated decision

2. Reason from first principles and apply general best-practices

3. Trust your vendors fully unless provided proof of vulnerability

I suggest 1 and 2.

> Surely if an exploit can go as far as to gain access to the browser pw mgr, they can do basically anything userspace, including execute payloads against all the other pw mgrs on the system?

If that is true for you, I really suggest hardening your system.