For example, all the predictions that make us capable of building skyscrapers that don't fall down for centuries. Does it matter that two bricks are not "the same piece of matter" if our predictions work the same for both of them? In terms of their behavior under particular circumstances, they are the same.
Bricks are in the category of "made objects" not natural objects, which generally means that they are _designed_ to come off a production line as similar as humanly possible to the other products. "My iPhone is physically interchangeable to yours" is a statement about the huge efforts of industrial manufacturing to standardise matter, not about the natural world.
Bricks too have quality thresholds that they have to meet or exceed. That alone should tell you that treating them like integers is a convenient abstraction, nothing more. The sibling comment has it right: counting bricks is a great model, but confusing your model for reality is still an old error.
No, because of a dense, interconnected web of other social truths (the rest of the arithmetic model), the relative error of this one truth/model is negligible.
However, confusing your model for reality is a fallacy perhaps older than time.
> However, confusing your model for reality is a fallacy perhaps older than time.
I think that the human perception the world is a robust enough model to be equated with reality without issues. If you go down the path of denying perception, you might as well go full solipsism, in which case it doesn't even make sense to discuss reality at all.
The model is "good enough" for the purpose of creating a building, but that doesn't make the act of "creating a building" any less real, nor the underlying rules that govern matter any less true. Our descriptions are not real - the rules themselves (which we may not know exactly) are real.
Therefore, mathematics - the set of rules that corresponds to how reality works - itself exists in reality regardless of social consensus. Society can't change them by making a different consensus.