Yes, I think the asymmetry is entirely due to Portuguese people being regularly exposed to Brazilian pronunciation, whereas the reverse is much rarer.

In the late 80's & 90's Portugal started importing TV programs from Brazil. Since the 2000's this happens much less, but that has been replaced by a large number of Brazilian immigrants settling in Portugal.

As far as I can tell there are no equivalent influences going in the other direction, so Brazilians are rarely exposed to Portuguese pronunciation.