Thats for structured data, for non structured it's more like "create a NN and stack more layers until you have your MVP"
I mean, that's a pretty good principled approach to a lot of ML problems.
I invite the possibility I've gone this long misunderstanding the definition of "principled" in this context.
Yes, the "just add more layers" approach typically works (in a very narrow sense of the word "works"), but we don't really understand why. We likewise don't understand the failure modes of the system, and cannot engineer around them. Thus it's not really principled in my view.
Unless you're Google, who even trains models from scratch these days, at most you do some fine-tuning