I think you underestimate the degree that he is becoming a household name, especially since the ScarJo thing
I'd say they're on about the same degree of (low) popularity.
You may argue the comparison vs. Swift is overkill, but that's actually the kind of fame that's "household name" territory.
Taylor Swift is far beyond household name level of fame, she is likely among the most recognizable names in the world if not the most recognizable name in the world right now. I use google trends quite frequently, I have yet to be able to find a name that eclipses taylor swift