I'm skeptical that theoretical physicists disagree. If anyone would be aware of these dangers, I would imagine it to be them; these are incredibly smart people! I only even got an undergraduate physics degree and this stuff was all quite obvious. Which isn't to say that arguing about which metaphors are likely to bear conceptual fruit isn't worthwhile, but even with much more physics education than most I would never deign to enter such a debate myself. The people doing the work definitely know what they're doing and I definitely don't, so to presume otherwise is weird to me.
I worry that it is the pop sci presentation of physics that non-physicists look at and think wrongly that the actual people carrying out the work are blinded by training bias (which is true for anyone, but I think far less true for experts in the field than otherwise) when in fact it's just the authors of pop sci articles and books who are ignorant of the difference between metaphor and reality. These are people who are paid to figure out how to think differently about their questions, they're not just following conventions and calculating pi to more decimal points.
Is an encryption algorithm "real"? Is the process used in proving a theorem "real"? Is the way that we model an atom "real"? No, of course not; they're all ways to represent an idea/observation through metaphor by breaking down a problem to make it look more like problems we already know how to solve. And if you just decide to approach the metaphor for understanding that idea differently, you can get massively more tractable problems.
As an example, a friend of mine in undergrad was studying crystalline domains in metals and wanted to find a way to understand better the relative misorientations of neighboring crystals in three dimensions. The metaphor as written strongly suggests that the right approach is some manner of statistical approach with a random variable representing orientation that can then be statistically analyzed to provide a misorientation distribution, but the problem is generally intractable in 3 dimensions whereas in 2 dimensions it's fairly straightforward math that way.
He instead decided to represent the crystallites in four or maybe five dimensions, which by so doing he was able to show that the math looks the same as that used for adding angular momentum using Clebsch-Gordon coefficients derived from quantum mechanical spin coupling. And found that by so doing he could turn a statistical problem into one with a closed-form solution that was quicker and easier to calculate in more sophisticated cases. With apologies if I'm explaining it wrong, this one never was quite comprehensible for me, he was far better at math.
Does he then think that crystallite orientation is literally quantum mechanical spin? Of course not. Does the math produce a testably distinguishable result? No, of course not; it should be the same result. But is it likely some future scientist with greater understanding of quantum spin coupling might be able to conceptualize the system more deeply? Definitely! This example is hardly unique.
So my point is that representation and metaphor matters, even if it's not a testably different result. To say that it doesn't matter to me seems to be saying that the current model itself is real, when it's not either! There's no fundamental reason to favor one over the other if the results predicted are the same, other than personal preference, ease of manipulation, or ease of conveying the model/metaphor to future students.
By encouraging different metaphors for the same system, you enable future scientists to be taught those alternative approaches so that they have a bigger toolbox to approaching some yet unknown future problem. We're just humans, and everyone looks at things a bit differently; none of us with a clear picture of what's really "real". Much like how there are a half dozen different approaches for quickly sorting a list, some are more intuitive or powerful or quick to execute than others.