This is a great point-
I reviewed with team and created https://github.com/brave/brave-browser/issues/3301 to track this (folks are welcome to give it thumbs up). The fix for this should be something we can deliver in our next product release (0.60.x - 9 days from now)
edit: issue is now fixed! https://github.com/brave/brave-core/pull/1633
I also captured feedback on being able to customize/opt-out of this functionality with https://github.com/brave/brave-browser/issues/3302 (thumbs up and comments appreciated!)