I agree with this, with your caveat of "early stage". Later on we have a responsibility to keep tech debt under control to a degree, even if that's invisible to the customer. (The side effect of faster development pace and higher quality later down the line will certainly be visible).
But in the early stages - I agree with you and tend to take the view that it's better to just throw something together and get it out there and being used quickly - irrespective of how bad the code might be underneath.
Get the system out in front of users, and delivering value to them as quickly as possible. You cannot beat the power of getting early feedback and validation of a concept.
For larger (higher stakes) projects this can be in the form of a "prototype" where the expectation is set that it's not a complete product. For smaller projects, fuck it, call it Version 1 (or a "beta" if you must) and get real people using it quickly :-)
Just be prepared to continue iterating from there.
As a dev I also often fall into the trap of "coding for coding's sake" - I love TDD, DDD, clean code, getting the right abstractions... and yes this stuff (when used appropriately) can lead to good maintainable code with low tech debt and easier feature development... but too often as developers we can end up losing sight of the business need and go too far up our own backsides with this stuff. Perfecting the codebase in the name of maintainability and controlling tech debt, when really it's often unnecessary perfectionism - or premature optimisation at least. Maybe that's just my own experience and I'm projecting - but I do see this quite often with even the best devs I work with. Actually more often with the devs that take the most pride in their work. It's a difficult balance to strike.