> You could code numerical things in Mathematica, but you'll get poor performance compared to MATLAB.
Yes and no. MATLAB and Mathematica have access to the same BLAS libraries. If you're using packed arrays in Mathematica, you can expect similar performance for many pure numeric things. Mathematica is missing some of the higher level primitives, there's no built-in function that corresponds to DGEMM, as a random example.
Another weakness is that there's only a handful of packed array types, for a 64-bit system, you get tensors composed of doubles, signed 64-bit ints, and complex numbers; no bool arrays, no unsigned 32-bit or 16-bit, no enum or factor arrays. Now this is purely a performance thing -- you can make tensors/arrays containing anything you want, but their contents just won't be 'packed' into a contiguous region of memory, instead they'll be arrays of pointers to other memory on the heap. Such arrays are slower to work with, much less memory efficient, harder to pass over an FFI, etc.
There's also a lack of fast workhorse-ish functions involving tensors, so there isn't a way to broadcast a condition (say 4 <= x <= 10) onto an input matrix to get a 'mask matrix' as is so common in MATLAB and R and so on, and one of their secrets to writing fastish code in a interpreted language.
On the other hand, you can deal with ordinary lists (and Associations) that contain arbitrary types (as with any dynamically typed language), which makes many kinds of algorithmic work very straightforward. After all, not everything is best represented as an n-tensor of reals or ints or bools! But if your lists do happen to be homogeneous lists of ints or reals (of any rank), they will usually end up being stored more efficiently as mentioned above, a process that is, for better or worse, totally invisible unless you know about Developer/PackedArrayQ.
There is also Compile (https://reference.wolfram.com/language/ref/Compile.html?q=Co...) which still has a fairly limited type system, but will still let you bypass large amounts of interpreter overhead when you need to do something performance critical.
> you wouldn't analytically solve a sophisticated PDE so you would use numerics
The whole analytic/numerical distinction really isn't true anymore (e.g. https://reference.wolfram.com/language/ref/NDSolve.html, https://reference.wolfram.com/language/FEMDocumentation/tuto...).
> so that explains why MATLAB seems more prevalent.
I think the explanation lies more with MATLAB having exactly the right features (or the right toolboxes) at the right time to fill niches in thousands of engineering and research labs. If you're running a lab, and that postdoc who wrote the code driving your instrument has long since left, there is very little reason to embrace a new language, even if it has caught up in various ways, and especially if that new language isn't tailored exactly to what you are doing, and would prefer you to use some strange thing called "functional programming", etc.