That just means you're putting a boundary when it's not necessary. Mox exists to make tests simpler to write and reason about, if you keep hitting extra layers of indirections then, "you're holding it wrong."
Most things are best used sparingly. That said, the vast majority of developers, even senior, have a very limited understanding of testing practices, and tend to create millions of separated, isolated modules, for no reason whatsoever. That's not Mox's fault.
A stupid example in practice: a naive blog app does not need Mox at all. If you add SSO sign up, you might want to put a boundary there to unit test without needing to set up a fake auth server, but 99% of your code won't ever touch it, just the authentication tests. Tomorrow you add payments, you might want to put a boundary just before your StripeRestAPI module to test the entire payment flow without contacting Stripe in your unit tests. Again, only the payment module tests ever need to touch Mox, the rest of your code doesn't. You place it before modules in the outer edge that have real-world side effects, not to isolate your functional and mostly pure core.
I recommend the DestroyAllSoftware talks, especially https://www.destroyallsoftware.com/talks/boundaries
> And I just can’t justify it with “one day I might want to swap out the implementation”. YAGNI and if you do, deal with it then.
I agree with that. Using Mox to swap out implementations is a possible use case, but in practice it's better to hardcode things rather than trying to make it generic and swappable the first time around. Junior devs have a tendency to over-engineer code "because we might need to change it later." My motto is "copy and paste it three times before adding a layer of abstraction"